i agree that you need to find your own answers within. perhaps the spirit is attached to you because it can manipulate you somehow. it may not be evil -- just lonely.
i find that nagging feelings that seem to fly in the face of logic ought not to be ignored. it seems almost as tho the place you are is not safe or healthy for you and your family. you are faced with a choice happiness vs comfort or perhaps happiness vs. fear. again I agree that you need to assert yourself and move in your own self interest.
the enemy within knows our weaknesses like no other ...