Re: Did my brother and mother abuse me?

 I agree with what Blue Rose has suggested.  More importantly, are there personal issues that you're experiencing that require some sort of "explanation?"  Often, we take random childhood memories and attempt to attach some serious significance to them to explain current adult issues, fears, phobias, etc.  

If you feel that there truly was a pattern of sexua| abuse, then it may be something that you might want to address with a therapist.  

For whatever reason, you're asking for permission or guidance to sever your relationships with your mother and your brother.  Only you know whether or not this would be helpful - the information that you've shared is scant and I imagine that there has to be much, much more behind your desire to cut these associations off so dramatically.  Mind, I'm not requesting more information than you're willing to provide, but only you know the depth of your experiences.

I wish the very best for you.