I find it hard to believe people passing in cars with the windows rolled up can smell you. I think you're thinking too much about your smell and thinking it's worse than it really is. Since you can't smell it you blow it up to more than it really is. Try to get your mind off of yourself. I don't mean that to be mean but you're only thinking from your perspective.
By the way there are many people that drive with their hand on their face. You're noticing something from your environment and imediately thinking it has something to do with you.