I always brush my teeth after oil pulling. I feel like I have so many toxins in my mouth after oil pulling and brushing helps to remove them.

I suspect it can but for most of us, we are conditioned to believe we have to brush.

It is reported that rinsing with xylitol can replace toothbrushing. Perhaps after OPing, rinse well with water or mouthwash of choice and then follow up with a short xylitol rinse. See point #3 here:
http://www.healingteethnaturally.com/rinsing-mouth-brushing-teeth-with-xylito...