It could be a bacteria issue like dukeeeey said, but I doubt that since you've been taking a lot of Antibiotics that should have solved the problem easily if it was just a minor infection. You say there was no swelling but you were taking Antibiotics . Was there actually an infection? Was the bone graft rejected? Or just the implant? This is quite the confusing situation, especially since the other dental implant was a success.
I would personally go see another specialist for a second opinion on the matter. Be upfront with your dentist about this and they will more than happily refer you to someone they trust. Do any of these problems seem to fit your situation?