Hi dragonheart,
I'm not sure how old this post is but I just came across it as I was searching for answers on how to deal with a similar situation that I am going through in my life at the moment.
Your story sounds so similar to mine, but I was actually still with my fiancé when he and my mother started their affair. I also have a child with him. And the rest of my family act similar to how you explain your family.
If you would be up for having a chat I would really appreciate that as I really have no idea of how to handle this situation and I have never come across anybody that has been through something so similar as it sounds you have.
You can email me if you would like too, if not I completely understand.
And I hope everything looks up for you. As nobody deserves to be made to feel this way.
Jess x