It won't be an easy question to ask but it needs to be asked. Ask him right out if he is gay. Not only listen to his answer but watch his reactions.
Also, listen to your gut feelings---especially if your gut feelings have a good track record of being correct. If you do have good gut feelings, what are they telling you now?
All that said, some straight men just have very low sex drives. Sometimes, physical problems get in the way. Also, sometimes people have mental hang ups that keep them from engaging and enjoying sex.
If he says he isn't gay and you believe him, then ask him if he will: 1) get a complete physical check-up; and 2) go to marriage counseling with you.
Finally, just note that not all men can take v1agra. There are certain health issues that make v1agra use unacceptable.