Have you soaked in vinegar like I suggested? That can stop almost any kind of itch.
You might be allergic to coconuts. Oil pulling is an ayruvedic technique...and the way they do it is to use unrefined sesame oil, not coconut oil. So you could try switching oils. Or just stop for a few days and see what happens.