I sounds like you are getting overstressed in some way...

Since you think you are starting out moderately enough, is there something else you are doing that could be stressing your system out?

For example, does the gym where you go have loud music that bothers you? Maybe you have to drive out of the way to get there...
Do you change your eating or sleeping habits when you start working out? (do you eat more desserts because you started working out?)
Do you feel anxious about your progress?

I find that when I am running my life at maximum speed, it is really easy to tip the scales and get sick. It is often a sign that I'm involved with too many things that are stressing or irritating to my system and not enough things that are pleasurable.

Maybe a walk or a bike ride coupled with some strength training that you can do at home would be a better way to start, and then you can gradually add in short visits to the gym...

Another thing is to reconsider going to the gym. Do you enjoy it? Is there something else you could do instead. FOr example, I really enjoy learning new skills, so taking up karate was a great choice for me. My sister, on the other hand, enjoys team sports such volleyball, softball and tennis. If you find an activity that feeds your spirit, too, you may have more success.

I hope that gives you some ideas to work with

By the way, a friend of mine is offering a free online course in advanced hygeine that could give you some pointers on how to avoid catching viruses http://prevent.flu-bird.com/ (yep, I know it says bird flu but it works for viruses in general)

I wouldn't be so quick to blame it on exposure to germs.
It must be your own system. Either you over-tax its current stamina level or something else is amiss.

Do you get plenty of sleep? That is one of the keys for exercise recovery as well as allowing the immune system to do its thing. It is common to need more sleep when you are exercising, particularly in the beginning. It is easy to get sick if you haven't had enough sleep.

What is your diet like?

Apparently exercising is very cleansing, your sickness might be a result of your body cleaning itself out, change in pH, etc.

You may want to check into zapping/blood electrification to kill the viruses and keep yourself from getting sick but that is not something everybody wants to do.

to find something you like

I used to love Jazzercise, and some of th egals ran before the class- I joined them cuz' it kept the weight off, but I NEVER enjoyed it and I got injuries from it- tendons (which are not that easy to heal) and bone spurs- (no healing)

I refound yoga, and I love it- the trick for me is to have a group that I participate with or to teach- then I have an approach that keeps me going...and I am not concentrating on looking good stuff, I am concentrated on inside stuff ~the feeling- and most yoga "types" have some meditation offered which helps with the real bugaboo- the mind! or the attitudes that are produced by the old mind- the stories the bad tapes the opinions can be calmed-and the resistance to taking care of the physical lessens over time!