I have two thoughts for you about this:

1) I prefer to focus on eating foods that have been part of our (hominids) diet for longer than 100,000 years. This would mean no dairy products or derivatives.

2) It is difficult, sometimes, to adhere to these principles with anything like perfection. So if one eats these things occasionally I am sure it would be with minimal challenges. Just remember that we are not designed for these 'newer' foods and as such our bodies will need some time to cleanse/deal with these foods so regular consumption of them is not a good idea.

Also, anytime you are eating a less-than-healthy food you should remain conscious of the fact that eating unhealthy food also means, probably, not eating healthy or required foods. This is worse than simply eating 'bad' foods.

To your health,
Eric Edmeades
http://www.theHumanDiet.com