End it now, before you get hurt even more. He is still in love with his ex, sounds like he has lots of baggage. He also sounds very selfish in his answer to you about why he didn't answer your call. I have been involved with relationships like that, and always thought that I could "fix" the guy. My advice is run, don't walk, run. Believe in you gut instincts, and get out.