You could have given him a yeast infection... Or he could have given you your yeast infection. Or, for that matter, your house, your water, or some majorly contaminated food the two of you indulged in together could have given you both yeast.
While GUT yeast isn't all that infectious, skin yeast, dandruff, etc. IS. And, most of the time, it couldn't care less what mammal to live on.