I think you're on the right track. If you do go to visit, keep it short. As an adult, when I would visit my parents, there were many times I just wanted to cut it short due to my mother's behavior. At the time, however, they lived at the other end of the country so it would have been difficult and/or expensive to change plane reservations. In the end, I was glad that I didn't live closer to them. I tried that for a while (about two years) and having my mother that close by was not fun at all. For example, I told my parents where I was working but never gave them my work number. Yet my mother found out what it was and ended up calling me more than once. Most of the time it was to rant and complain. It's very uncomfortable to have to field calls like that in a work situation!
So...if you do visit your father, make sure you can easily cut your visit short if things get unpleasant. Will you have to fly to where he is? If so, changing your reservation won't be easy (probably) so arrange for a backup plan should things go sour. Get a rental car---that way, you can easily leave on your own and perhaps finish out your visit by staying in a hotel.