One of my mom's brothers was like your youngest. He was 4 before my grandfather ever heard him speak, and he had to hide behind a door to do that! He learned (and still learns) everything at his time his way and, even if he started late, he caught up and flew past expectations in a blink! If your son grows up anything like my uncle, you'll be bursting at the seams with pride and scratching your head over him at the same time, hehe.
Naw, my uncle is just a truly full and rich personality, an all around great guy and very much his own person!