Different molds have different smells. Most of them are characterized as a musty smell. The smells present themselves during various stages of their growth cycle. Just because the smell goes away does not mean the mold problem is gone. The smells are produced by the MVOCs (Microbiological Volatile Organic Compunds) they produce, along with mycotoxins, to kill off their microbial competitors for the available nutrients in the building materials.
Bathrooms are the most common place for mold problems to start. A simple leak from a toilet which is not corrected, and the building materials completely dried out within 48-72 hours, with start growing mold. Since most people do not take these proper corrective measures, a high percentage of the bathrooms in homes have some mold problems.
With all the humidity in bathrooms from showers and baths, it provides excellent conditions for mold to grow. If that humidity is not exhausted out of the room within a half hour with a GOOD fan it will make the problem get worse and worse.
Chances are that if you suspect there is a problem, there is. If you now have health problems, that is another clue there is a problem. If your health problems get better when you are away from that house for a week or so, that is another clue you have a mold problem.
All building materials HAVE TO be replaced. Use protective gear while it is being done. Take precautions the spores don't get blown into the rest of the house, by putting up a containment barrier. Use a professional company who is REALLY familiar with mold problems and tests afterwards to make sure problem is gone.