It is the bacteria that generate the acids that deal with the Candida.
From what you are describing it sounds to me like the cultures are getting contaminated. Maybe by bacteria in the milk itself, mold spores in the air or contaminated equipment. Make sure the equipment you use is all sterilized and I would keep the container sealed tight. As for the milk I recommend treating the milk with diluted food grade hydrogen peroxide first to sterilize it. A scum will form on the top with the dead pathogens. Just scrape this off then use the milk as usual. This is how milk is sterilized in Europe, but I don't know the exact peroxide dilution. So this will require a little research.