I think what you said was pretty good. Especially if what they do offer that is considered "vegetarian" is covered in cheesy sauces of chemicals. Besides, you need variety. Even if they offer steamed broccoli, brown rice, apples, and a stale salad bar everyday, why should you have to exist off of such monotony? The others get variety (albeit at the expense of their health), whereas your health will be suffering because you cannot get the proper nutrients you need without variety. Like you said, you should not have to pay for things you CAN NOT eat (it's not that you don't like the food - you can't eat it). If you were actually allergic to animal products, for example, you couldn't eat them, and they probably wouldn't make you pay. You might not be "allergic" but you know you can eat the food, so in my eyes it's the same thing.
And you will have to pay even more to buy the food that you'll be eating daily, outside of the cafeteria. You'll be paying twice, not to mention the extra time and effort. If they're gonna make you pay, they'd better change dramatically and offer you very acceptable food.

If you happen to have any religious convictions regarding food, that might help too, not sure.