I'm in the same boat as you my friend. I used to take EFAs daily for years. Had a major health crash almost 2 years ago and only recently have I begun to feel normal again. One thing I still notice is that oils of any kind drain my energy like nothing else.

Now I have had issues with adrenal distress my entire life so part of it is the fact that some days depending on various factors ie. air quality, quality of sleep, nutrition from previous day. My energy level overall is much more fragile.

But getting to the point I believe the reason for this fatigue is primarily due to the amount of work your body has to do in order to break down fats, and should the fats be consumed with any other foods then it's even worse. In addition to which people with toxic or weakened liver function also have issues with oil digestion for this reason which can cause fatigue in of itself.

I had considered that an allergy could have been developed to these oils, which could be a possibility in your case. But more than likely it's a combination of already weakened adrenals with sub-par liver function which is causing the fatigue after consuming the oils.