Forgot to mention some other ideas. You listed foods that seem to aggravate you. Have you done an elimination diet? They sound difficult but once you're on one and have nailed down a do-able diet they aren't too bad. Kind of simplifies your life actually! If that doesn't give you clues you could consider a sensitivity to sulphites or histamines (although your symptoms don't seem to indicate these).
There are stool tests that look at your commensal flora. If this gets out of balance your defenses will weaken and you can get chronically ill. I came across this idea while researching anxiety as there is a connection between anxiety (and depression and autism and ADD etc) and flora. I agree with Allopaths conclusion that candida only occurs in the immune compromised but have a much broader definition of immune compromised. If your intestinal foundation is out of balance...you are immune compromised. To my understanding there are two labs in the world that can do this test and the stool has to be FRESH. I emphasize fresh because it's a real trick getting it to them in time. One lab is at the Melbourne University in Australia and the other somewhere in England (more info on Dr Myhill's site). If this flora is out of balance you're fighting a losing battle until you correct it....it's not unlike the foundation of a house. If I'm not mistaken the correction is an enema concoction developed by Dr Borody (of Bad Bugs lore) that involves the flora of a healthy donor (think poo enema!)....again, Dr Myhill's site has got that info in detail.
Email me if you have trouble tracking down any of this info...I'll comb my bookmarks.