While a child with no dairy issues might do fine with some raw milk, if your son has shown allergies to it, I likely would not advise giving milk raw or pasteurized. Even small amounts can be troublesome for those sensitive to it. I believe raw milk is better because it contains enzymes that help digest it but cow's milk is still the most allergenic food on the list.
My son was dairy sensitive and now that he is 5 he can have small amounts. I still limit it though especially if I notice that he is mucousy (runny nose, cough etc). I can also tell when he is getting something that irritates him by the red angry ring he'll get around his anus. That has always been a good indication that something does not agree with him.
I make biscuits with water and mashed potatoes that are quite good with the potato water or some kind of broth.
That said, I have no access to raw milk and do not have much experience with young children consuming it. Maybe someone who has been raised in a community where raw milk is consumed might be able to offer more insight???