Kind of hard to guess over the net and with only layman's knowledge. Could it be that coffee makes your liver hurt but is not necessarily hurting the liver due to a possibly compromised liver condition? I would suggest starting with very small amounts, if any, and doing the other suggested things. Milk thistle and the other items can work wonders, especially in combination with juicing and a diet of mostly raw/lightly steamed veggies and some fruits.
You might also consider doing one of the gentler liver cleanses. Regardless of what the so-called stones may be, liver cleanses are generally beneficial for cleansing purposes and the gentler ones have little risk of adverse reactions such as lodged gallstones.