It's so hard when you've had the IUD out and feel better and then feel bad again. It's a roller coaster, and it can happen again and again. I don't mean to be negative, but try not to let it freak you out when you feel better and then slide back. It's worse for some women than others, but it's typical. It's easy to start thinking that maybe it wasn't the Mirena after all, but it's not true. It's just that the Mirena can throw other body systems out of whack, and for some reason our hormones are intricately involved, so we end up on the roller coaster. Like you, I've sworn off ALL hormonal supps for now. Every woman has to follow her feelings on this. You will improve, but it might be 2 steps forward, 1 step back.
Hang in there! When weird things happen, and you've been to the doctor and told you're "normal", then just grit your teeth and hang on. The good days will come back. Try to think of it as the tide: it comes in, it goes out. Relax with the cycles, and take care of yourself. I'm convinced that stress and worry only prolong the recovery.