I had one really large fibroid (about the size of a large orange) on my uterus. It had to be removed because it would have grown larger and possibly affected my surrounding organs. Also, I had a lot of pain during exercise, very, very bad cramps during my period and heavy bleeding. I had a Myomectomy, which means my doctor was able to remove the tumor and leave the other female reproductive organs in place(which is good because I am 30 and I have not had children yet.) It is the same incision as a c-section so it was an uncomfortable recovery, but I feel better than I did before. The only complaint I have is a pouch in my lower abdomen that is dreadfully hard to get rid of. I am working on that though. :-)
I am able to exercise again and my periods are not as heavy. I have heard dairy products and birth control are not good for fibroids. If you can keep them under control and at a reasonable size, I don't think they cause much trouble, other than that, you have to go with your basic instincts and listen to your doctor or advisor.
Peace and Blessings