One of the criticisms of store bought probiotics is that when you stop taking them, within a few weeks they completely 100% disappear from the intestines, along with any benefits from them. There are studies that have proved this.
So any side effects from the probiotics bacteria themselves should also in theory disappear.