ohfor07
The short answer is, generally speaking, no, canyenne type peppers and their seeds are not toxic. They will not harm you. Use them with a modest amount of common sense and they will serve you well.
The long answer is, without knowing the source of the book you found, the context of the specific cite, and if there is perhaps an agenda behind either, its difficult to answer directly to your findings in that book.
Be this as it may, here are some general purpose facts with regard to toxicity. #1, depending on amount consumed per a given amount of time, pretty much any substance is toxic, to include water and oxygen, both of which can be construed as toxic to the point of causing death if too much is taken too quickly. #2, with specific reference to tactics historically known to be used by medical authorities, "toxic" is interpreted merely to mean that a rediculously exorbitant amount of a substance was given to a test subject upon which the subject did a fairly predictable thing; had a serious reaction that may have induced death, near death or something in between. Best example I know of is the method U.S.F.D.A. used in the 50's for determining that Sasafrass is toxic. They isolated a specific compound from this herb, which is to say, they rendered a natural substance into an un-natural one, then injected increasing amounts of this isolate into a rat until the rat died whereupon they reported " sasafrass is toxic!"
With respect to the spectrum of cayenne peppers used for medicinal purposes "herbs", I would really like to know the vintage of the aforementioned book you read at the library, as well as the research, assuming there was research that yielded "cayenne is toxic". My hunch is that it is a fairly recent vintage, something less than 15 years old give or take. Main reason I am curious about this is that in the past few years there has been some sort of an effort being put forth by U.S. medical authorities to categorize cayenne type peppers as toxic.... even to the extent of labeling them "causes cancer". This I know from two widely separated avenues, one of them being an inside connection with somebody plugged into the upper echelons of the regulatory agencies governing medicinal policy in this country (CDC).