My KP diminished by 90-95% after 3 weeks of oil pulling, using coconut oil for 20 minutes a day. Molly's husband, who had KP, also had success with oil pulling.
A KP forum has many who found success for the bumps, but not always with the associated redness when they started OP.
Stay with OP at least a month to see if it helps. If you don't see a difference in a month, you may want to change to a different type of oil. I use coconut, untoasted sesame and avocado, all with KP success.