Read the whole article here - I like the pasta theory, below.
To get at why that time begins typically after age 3, Nautilus references the "pasta theory" of memory and the work of Patricia Bauer from Emory University:
Children are actively forming memories even at a very young age, but they lose them faster than adults do. "I compare memory to a colander," Bauer says. "If you're cooking fettucine, the pasta stays in. But if you're cooking orzo, it goes right through the holes. The immature brain is a lot like a colander with big holes, and the little memories are like the orzo. As you get older, you're either getting bigger pasta or a net with smaller holes." . . . In a forthcoming article in Journal of Experimental Psychology, Bauer reports on interviews with 100 children and 20 of their parents, which showed that the forgetting patterns of children and adults are quite different. Children forget things at a steady rate that doesn't depend on the age of the memory. By contrast, adults tend to hang on more to their older memories. After many years, the memories become sticky, like the fettucine that won't slip through the holes. Adult memories also come with more contextual information, more "who-what-when-where-why," which may make them stickier.