" I think this probem is attributed to us not brushing his teeth like we should."
If only everyone else in this forum could see the obvious as you have.
And enamel does not grow after the tooth is erupted. When it's gone, it's gone. Please don't neglect this problem as many people do "because they're just baby teeth and will fall out anyway." These teeth must fall out at a certain time to allow for proper eruption of the adult teeth. Otherwise the child's adult teeth become crowded without the baby teeth serving as "space maintainers", and may look like this: