What are being passed are not real gallstones. They are formed by a reaction between the sterols in the olive oil and cholesterol, which form an insoluble complex that looks like gallstones.
The reason people feel better is that they are making their gallbladder work by getting some oil in their system. This causes the gallbladder to contract keeping the bile from becoming stagnant, which does increase the risk of real stone formation.