Someone smarter than me will explain to you in detail why you don't need to worry. But, what I will say is a big stone will break up before passing, although if you will go back and read past posts, you will read where quite a few people have passed big stones and said there was no pain. The biggest stone I remember passing was about the size of a grape. (I, too, had been diagnosed with gallstones)
I am sure you have already read about the kidney cleanses, so you might want to try one.