Do you have the Nourishing Traditions book? In the book the author talks about this reaction. She says it's not so much an allergic reaction as the body's reaction to enzymes, which are detoxifying. Maybe you just have to start out very, very slowly...
Is brewer's yeast different from nutritional yeast?