I agree with you that those things are toxic (if we widen the definition of "drug" to mean chemicals in plants, and man made substances such as pharmaceuticals). but I am extremely wary when an ordinary plant with known healing properties gets labelled as a "drug" by the authorities, stigmatised, and made illegal. Especially when you consider that the tobacco plant has killed a hundred million people http://www.foxnews.com/story/0,2933,329410,00.html.
Yet it's legal to smoke tobacco, but not cannabis, it just doesn't make sense.

I am inclined to think it was more to do with money:
http://www.drugs-forum.com/forum/showthread.php?t=71124&redir_from=71125
●1935 – Public awareness had dramatically increased as newspaper articles poured out blaming the downfalls of society on illegal drugs and ethnic minorities. In this year, Du Pont began to lobby the US Treasury Department to suppress hemp. Allegedly, several individuals such as Andrew Mellon held substantial shares in Gulf Oil and other petrochemical and pulp paper industries, and stood to loose billions of dollars if the commercial potential of hemp was fully realized.
●1936 – The Conference for the Suppression of the Illicit Traffic in Dangerous Drugs dealt with addressing the criminality of trafficking, and there soon appeared many international legislations and laws to reduce illegal activity. Many of them were not followed by individual countries because they wanted changes made to protect their own interests – but it was the USA that carried out most of the laws and this would impact the decades to come.
●1937 – The Marihuana Tax Act was passed, effectively controlling the social use of marijuana. It imposed a heavy tax on cannabis, including hemp, and destroyed the slowly fading hemp industry. New funding for the FBN was allocated, and Anslinger had accomplished his goal. A competition over public headlines had been present between Anslinger of the FBN and Hoover of the FBI, about who would eradicate more criminals for some years. Hoover was to win by a matter of publicity, as Nazi’s, Communists, and spies were seen as a greater thread than Mexican marijuana users – so Anslinger played up the thread that was now knows as the ‘Killer Weed’. William C. Woodward of the American Medical Association strongly opposed the Marihuana Tax Act, saying it help no scientific basis and was rushed with undue haste. It didn’t help that AMA was mostly Republican and the administration was Democratic, and thus the objections were overruled, and the bill was guided through Senate by the chairman who was a close friend of Du Pont.