This is a good idea, using a blow dryer after you shower is a good idea also because the dead skin absorbs alot of water and it could be feeding the yeast.
One thing i have noticed is that there is FAR less dead skin build up when i wake up in the mornings because im not breathing through my mouth (i sleep with my mouth shut)