Well, I have some few memories of my very early childhood, but it is undisputable (and the articles quoted by PEEJWEJ don't dispute it either) that most children don't remember most things from their earliest years. The number of events that adults remember from their childhood, and the memory span of a child, clearly increases with the age of the child:
Maximum memory span in weeks (on y-axis) by age in months (x-axis) for operant conditioned tasks. From chapter 5.2 in the German edition of Berk, L. (2010). Development through the Lifespan (5th ed.), Pearson.
As for why, the reason is very probably four-fold:
(1) At birth, the brain is not yet fully developed. Areas which play a role in memory (like the frontal lobe), are still growing.
(2) Memory needs understanding. We don't just memorize sensory input, but the results of the processing of these inputs. If we don't yet understand the world and cannot process the sensory input, it cannot be "elaborated" and memorized. Another similar opinion is that we need a sense of self that we can relate our experiences to, and small children lack this sense of self.
(3) Because small children cannot yet speak, their memory is purely non-verbal. Adult memory is in large parts verbal. This discrepancy might make it difficult for the adult brain to access those non-verbal childhood memories that are in fact still there. [I don't believe this, because much of my mental processes today are non-verbal. Maybe I am different than the average person in this, but verbal thought does not play such an overwhelming part in my brain.]
(4) During later childhood and adolescence large parts of the neural structure are being degenerated (around age 6) and restructured (adolescence). During these processes, information that is not used, gets lost. [Anecdotally: I clearly remember a childhood memory I still had during my teens. I could "see" the scene in my mind's eye. Today, that memory is either lost or unaccessible to me. I only have a memory of the memory.]
This is what I remember from and looked up in some text books on developmental psychology. All of them are a few years old and not completely up to date with recent research, so there might be more and different information in journal articles. Anyway, I guess this can serve as a starting point for other answers.
If you have the English edition of the book by Berk, feel free to replace that image. Wanted to post another graphic that shows the development of different functions of the brain, but there is too much German text in it to make it useful for an international audience. Again, feel free to supply images of your own and edit them in, if you have any.