If your body has a true allergic reaction to a food then it will react instantly and swell up like people who can't eat peanuts.
I believe food allergies that take hours and sometimes days when continually eaten over several days are intolerances that are related to the state of our digestive system and out guts.
Check out the tests on Great Smoky Laboratories and how depression, fatigue etc relates to how our bodies react to foods