I agree that that you should keep note of your diet. Keep away from high glycemic foods (potatoes, pasta, junk food, corn), legumes, and processed foods. I know you're already eating a healthy diet, but certain things may affect you more than most.
Check out the paleo diet, it makes everything so simple and takes you back to your roots: vegetables, fruits, nuts, fish, and meat...that's basically it.