When I was a teenager I milked the family goats for a few years when we lived on a farm. When the raw milk was chilled in a large container it tasted as you describe. My mother found out that if you chill the milk in a very shallow pan so that it gets cold all at once it will not have the bad taste. In fact it tastes so rich and creamy it makes cows milk taste like chalk water. The whole family loved the milk.