--- if you swim in a polluted area yes

Sea water is a non-issue. And honestly, I think many people go too far in their concerns over environmental mercury from natural sources. If we didn't get mercury from man-made sources and historically insane practices like Amalgams and putting it in vaccines, the impact of mercury on human life would be minimal.

Don't eat very large fish like tuna, swordfish, or mako shark. Shellfish can be problematic as well.

No, there is no danger in absorbing any significant amount of mercury by swimming in the ocean or in a lake, even though they have enough mercury to contaminate fish to dangerous levels. The problem with the fish is that the mercury level is bioaccumulated to high levels by organisms in the food chain. But eating fish from bays around the U.S. and other places can be very problematic regarding mercury.
Most predator fish(those most eaten) from the Gulf coast have dangerous levels of mercury: http://www.flcv.com/flhg.html similar for other areas