I think the terrain argument has something going for it - especially if you make it concrete and start talking about the immune system - where mind, body and spirit come together.
You are right about it being purely subjective though.
Confrontation needn't be stressful. And even the stress needn't be bad stress. It all depends on you and your personality. The real question is - is your job in line with your personality? If it isn't you will be stressed pretty much no matter what. If it is then any perceived "stress" that comes your way can be handled.
Of course real life isn't just about what is in line with your personality.