It may not dissolve in the water but it certainly absorbs into the skin. Anyone who has done a bath with even a few drops of green spirits of Turpentine can verify this. You can feel it penetrating your whole body and it does soften the skin, to much if overdone. You will smell it in your skin after you dry off.
When it oxidizes it becomes water soluble. Probably all Turpentine is mildly oxidized. If is is chlorinated water that could also oxidize it.