Personally, I've seldom had fillings done without pain and soreness for weeks after. A couple of times it was horrendous for over a month afterwards. Are you swishing gently or strongly? Try going as gently as possible. Just gently suck the oil through your teeth - the swishing action itself isn't what's effective but just that the oil circulates.
For any kind of mouth pain I've always had success swishing with Bentonite clay. It works great for that.