Again I'd say it doesn't really matter if its a 'gut based' or 'adrenal based problem', because both will need work.
I'd get on an antifungal and powerful probiotic. There is a relationship between dysbiosis and histamines but I am not really sure what it is. Diet change alone is often not enough to 'cure' dysbiosis.
Then I'd keep supporting adrenals. Sex problems for men is usually drawn to estrogen/testosterone and cortisol levels. Zinc is a great supplement for men's sex drive, it has helped me. A good copper-free multivitamin or multimineral would be a good idea. It doesn't have to be copper-free, but copper is a really problematic for me and others, and copper/zinc are antagonists so if your worried about estrogen stay away from copper.