I avoid the typical Health shops, they tend to be expensive. Also I avoid tablets as you pay too much for the fillers / binders and it may inhibit the uptake of the very supplement.

I would go for pure bulk, powders. Like ascorbic acid, magnesium citrate / malate. Some companies sell a good vit c in capsule form without additives, which for me is more convenient in a busy day.

The magnesium oil has been a very effective supplement for me. The concentrace I take has a good amount of other minerals too (sulfur,boron, lithium etc) + most traces. This is from the salt lakes; not the cheapest one, but all natural and has a fast uptake.
Alternative is to get magnesium chloride in powder / bulk and make this yourself.

For boron, borax is super cheap.
Potassium; chloride form that can be purchased in big bags at your store. V uses it to salt his soups with, salt alternative. I prefer the high mineral salts for this purpose, Himalayan and others.
MSM / sulfur; relatively inexpensive in powder form.

The ones I buy ready capsuled, is vit B and selenium; have a good look around to make sure you find ones without too much fillers. Powders are possible to purchase in the US.

Iodine; also possible to buy in bulk, make your own potassium iodide or Lugols. Also check out aquarium shops for iodine.

These mentioned has been the supplements I had most effect of.

Herbs, spices that I "fortify" the diet / supplements with; I buy powdered, bulk. Or pick them fresh and put them in a strong alchohol to make my own tinctures. In the spring / summer I of course pick and eat directly; flowers, dandelions, st. Johns Worth and many others. It is totally free!

Empty gelatine capsules, fill them with the ones you cannot palate / take in water / juice, other drinks, food.

Essential oils; possible to get them very cheap at e.g soap making companies. Same with coconut oil.

Other shops that are worth searching on the internet; "make your self shops" beer making, wine making, cream/cosmetics.. Aquarium / pet, horse shops (DE for instance, also herbs and vitamins, very inexpensive!)
In US, I like the mountainroseherb company; but unfortunately they dont ship to Europe.
In Europe we have indigoherbs, healthleads UK as good alternatives for high quality herbs and supplements.

Good luck searching!:)