"No one is allergic to sulfur itself. Sulfur is not present as an isolated element in the body, but in combination with other elements and, most often, in complex molecules. The primary placement of sulfur in the human body is in the sulfur-containing amino acids: methionine, cysteine, homocysteine (and related cystine, homocystine), and taurine."
From my research, those who have difficulty taking sulfur nutrients or sulfur containing drugs are simply having trouble processing the mineral due to a kidney or liver weakness/deficiency.