I'm really not familiar with Golden Milk, but the ingredients that you mention shouldn't be causing pain. Any other ingredients? Does it contain turmeric and raw honey? What kind of milk is being used? And where did you get it? Actually, the calendula should help the joints, but the only way to see if it is responsible for the pain is to stop drinking it for a few days and see what happens.
Turmeric should help with inflammation. And below is a link with suggestions in treating arthritis.