Milk really IS NOT good for you! Milk companies pushed it as "healthy" and "necessary", but it has a lot of cow hormones (you're drinking breast milk of a cow!).
I found milk made me break out, also can make you gain weight. Seems gross to me. Also it wouldn't help anything? It's not that healthy, you certainly wouldn't lose weight consuming all that fat.
Sorry. Try juice, or water if you truly want to fast. OR why not just a healthy diet? =)