All of the scenes where Harry and Dumbledore dive into the penseive are fascinating. So much more so when reading the books back to back because you are more amazed then. Rowling showing us young Riddle, making us understand where he comes from and why he is the 1/7-of-a-man that he is, is just a brilliant storytelling technique.
I certainly agree with her here about the storytelling, but also about the history and back story of Tom Riddle. In HBP especially, we get to learn the back story and the history which explains so much.
JK treats us to the other character's back stories, too. In my opinion, what we learn of Dumbledore's youth in Deathly Hallows ties for first place with the Riddle Pensieve scenes. For almost all of the series we are presented with Dumbledore as an ageless old man; it is quite impossible to imagine him in his youth, or even that he ever had one. Indeed, we only consider him as a middle aged man in Riddle's memory of The Aragog Incident in Chamber of Secrets, let alone as teenager looking forward to his gap year.
We also get to hear more of Lily's and Petunia's story although I wasn't as desperate to hear them as the others. Yes, they tied up some loose ends nicely (like the Dementors comment) but to be honest I always trusted JK to do that well. The backstory which was most revealing and exciting to me (after Voldemort and Dumbledore) was Snape.
My appetite for youthful recounts and prequels is not limited to the Potterverse. I relished in Hannibal Rising which also explained so much about our Chianti sipping dinner guest. X-Men Origins, Batman Begins and so many others explain why these iconic characters are who they are and I lap up every story, even when I wasn't a follower of the character before that. Of course, the most famous prequels of the last decade - The Phantom Menace, The Clone Wars and Revenge of the Sith also told us much about the back stories of so many iconic characters, although we did know a lot of that before and I must admit I didn't find the back stories as fascinating as I thought I would (even though many other aspects were phenomenal.) I would still love to learn about a teenaged Yoda, though.
I think my fascination with character's youths is related to the fact that as a high school teacher, I am forever teaching teenagers. Individuals graduate, pass through, go on to other things, have careers and children and lives, but I am perpetually part of 'those formative years'. To return to the beginning of my post, I think Dumbledore explained it best when Voldemort was seeking to teach at Hogwarts.