What a difficult decision. The good news is that much of your baby's brain development was in the first 3 months. The bad news is that your baby needs more overall nutrition now than before.
Lung development happens in the last trimester. Additional calcium, magnesium, phosphorus, boron, vitamin D, vitamin K ... is required for solidification of structural elements. You need a well balanced nutritious intake or you and your baby will suffer.
It is difficult to get balance with restricted input. You need a dietician to help you through this period.
Thoughts I have, not knowing your history or specific issues:
1) Balance fluids, solids and fiber. Fiber without sufficient fluids is a problem. Solids without sufficient fiber is a problem. If you take a fiber supplement, let it sit for 15 minutes. If it turned solid, figure out how much water you need to add to make it liquid again. Then, drink that much liquid everytime you take the fiber.
2) Get sufficient essential fatty acids. I like fresh ground flax seeds for fiber and omega-3 fatty acids. Molecularly distilled (pharmaceutical grade) fish oil is also terrific, though it has no fiber. The process eliminates mercury but provides pure omega-3 fatty acids for your baby's brain and nervous system. Perhaps in the winter you should go for molecularly distilled cod liver oil, for the added vitamin A&D. D is difficult to get in the winter and is important to bones and teeth.
3) Consider juicing, especially above ground veggies. Drink the juice and mix the fiber into soups. This gives you the nutrients from the juice and the fiber as well.
4) Excercise. Is there an aqua areobics class at a "Y" near you? Excercising in the water floats the baby's weight, taking pressure off you and letting you maintain fitness levels. The digestive system needs activity -- I always notice that when my dog doesn't go on frequent long enough walks, she gets constipated.
Best wishes. Let us know how you're doing!