You definitely should keep going places. Don't let anyone's opinion of you stop you from enjoying your life. Should you subject yourself at every single moment? No, but this is still your life. You living less than does not make anyone think differently of you so you may as well have some fun.
Also, it is how you know if what you are doing is working. Keep working at it. It will bet better and going out will get easier and easier. Hopefully with fewer reactions each time. I hope you are getting out for the holidays.