The Placebo Effect: And Another One Bites the Dust

The Placebo Effect: And Another One Bites the Dust

http://archderm.ama-assn.org/cgi/content/full/137/12/1639

Arch Dermatol. 2001;137:1639-1640.

Is the Placebo Powerless? An Analysis of Clinical Trials Comparing Placebo With No Treatment
Hrobjartsson A, Gotzsche PC
N Engl J Med.
2000;344:1594-1602

Placebo treatments have been reported to help patients with many diseases, but the quality of the evidence supporting this finding has not been rigorously evaluated.

The objective of Hrobjartsson and Gotzsche's study was to conduct a meta-analysis of clinical trials in which patients were randomly assigned to either treatment with placebo or no treatment. A placebo could be pharmacological (eg, a tablet), physical (eg, a manipulation), or psychological (eg, a conversation).

They systematically searched MEDLINE, EMBASE, PsychLIT, Biological Abstracts, and the Cochrane Controlled Trials Register for trials published before the end of 1998. They included only trials with concealed random assignment, masked outcome assessment, and dropout rates of less than 50%.

For each trial with binary outcomes (eg, stroke or no stroke), they calculated the relative risk of an unwanted outcome. The relative risk was defined as the ratio of the number of patients with an unwanted outcome to the total number of patients in the placebo group divided by the same ratio in the untreated group. A relative risk below 1.0 indicated a beneficial effect of placebo. For trials with continuous outcomes (eg, a change in diastolic blood pressure), they calculated the standardized mean difference, defined as the difference between the mean value for an unwanted outcome in the placebo group and the corresponding mean value in the untreated group divided by the pooled SD. A value of -1 signified that the mean in the placebo group was 1 SD below the mean in the untreated group, indicating a beneficial effect of placebo.

The pooled relative risk of an unwanted outcome for trials with binary outcomes and the pooled standardized mean difference for trials with continuous outcomes were calculated using random effects calculations. Hrobjartsson and Gotzsche performed preplanned sensitivity analyses to study the effects of the type of placebo used, the type of outcome, and the effect of sample size on their results.

They analyzed 114 randomized controlled trials that met their inclusion criteria, 32 with binary outcomes (involving 3795 patients, with a median of 51 patients per trial) and 82 with continuous outcomes (involving 4730 patients, with a median of 27 patients per trial). The trials investigated 40 different clinical conditions. The results indicated that compared with no treatment, placebo had no significant effect on binary outcomes, regardless of whether these outcomes were subjective or objective. The pooled relative risk of an unwanted outcome with placebo was 0.95 (95% confidence interval [CI], 0.88 to 1.02). For trials with continuous outcomes, placebo had a beneficial effect (pooled standardized mean difference, -0.28; 95% CI, -0.38 to -0.19).

The effect decreased with increasing sample size, indicating a possible bias related to the effects of small trials. The pooled standardized mean difference was significant for the trials with subjective outcomes (pooled standardized mean difference, -0.36; 95% CI,-0.47 to -0.25) but not for those with objective outcomes (pooled standardized mean difference, -0.12; 95% CI, -0.27 to 0.03). In 27 trials involving the treatment of pain, placebo had a beneficial effect, as indicated by a reduction in the intensity of pain of 6.5 mm on a 100-mm visual analog scale.

The authors concluded that they found little evidence in general that treatment with placebo had powerful clinical effects. Treatment with placebo had no significant pooled effects on objective or subjective binary outcomes or on continuous objective outcomes. It did have significant, pooled, small benefits in studies with continuous subjective outcomes and for amelioration of pain. Outside the setting of clinical trials, there is no justification for the use of placebos.

Editor's Comment

It is rather shocking that a belief so engrained in medical teaching turns out not to be supported by the evidence. The studies' results are so opposed to our expectations that they raise several questions. How good is the evidence? What is the evidence? Should we believe the results and conclusions? How should we use the results of this study?

How good is the evidence? Oxman et al1 outlined the critical appraisal of systematic reviews and meta-analyses. Hrobjartsson and Gotzsche's study fulfilled almost all the criteria of a valid and useful study. They addressed a focused (and important) clinical question. They established explicit and appropriate criteria to select articles. Their search was well designed, iterative, and thorough, making it unlikely that many relevant studies were missed. They appraised the quality of included studies. Both authors assessed each study. Although there was significant heterogeneity among the studies, it was adequately explored through preplanned subgroup analyses. Hrobjartsson and Gotzsche's study can be criticized for including studies that had dropout rates as high as 49%. Studies that do not adhere to intent-to-treat designs tend to produce a bias in favor of treatment. The effect that high dropout rate studies had on the difference between treatment with placebo and no treatment is unknown; it could have been addressed by Hrobjartsson and Gotzsche, but was not.

What, then, was the evidence? Compared with no treatment, treatment with placebo had no significant effect on binary outcomes, regardless of whether these outcomes were subjective or objective. The 95% CI around the pooled relative risk of 0.95 was narrow, because a large number of patients were included in the pooled result. It is therefore unlikely that a clinically significant "placebo effect" was missed. Among the trials with continuous outcomes, treatment with placebo had a beneficial effect in the trials with subjective outcomes but not in those with objective outcomes. Hrobjartsson and Gotzsche used the standard mean difference to measure continuous outcomes. The standard mean difference, also known as the effect size, converts results from varying scales to a single metric (ie, multiples or fractions of an SD).2 It thus allows studies using differing scales to be pooled. In 27 trials involving the treatment of pain, placebo had a beneficial effect, as indicated by a reduction in the intensity of pain of 6.5 mm on a 100-mm visual analog scale. The authors note that the pooled effect of placebo on pain corresponds to one third of the effect of nonsteroidal anti-inflammatory drugs, compared with placebo, in double-blind trials. Thus, in summary, in trials with binary outcomes and trials with continuous objective outcomes, the pooled results did not detect a clinically or statistically significant difference between treatment with placebo and no treatment. A small difference between treatment with placebo and no treatment was detected only in trials with continuous subjective outcomes. The biasing effects of small trials may have been responsible for the observed difference seen for continuous subjective outcomes.3-4

Should we believe the results and conclusions of Hrobjartsson and Gotzsche's study? Their study is apt to be the definitive study on the placebo effect. Many factors may contribute to the observed improvement in patients in the placebo groups in clinical trials.5 Hrobjartsson and Gotzsche's findings indicate that the contribution of the placebo effect, if any, is small. Other factors include regression to the mean, subject and observer bias, and nonspecific effects of interventions. Regression to the mean describes the phenomenon in which extreme states are most likely to be followed by states that are more average.6 For example, a patient with a fluctuating disease who is doing very poorly (and likely to enter a clinical trial) will on average regress to a more average (improved) state over time, whether the intervention is effective or not. Bias is an inherent problem in clinical trials, as patients and investigators have a vested interest in having the trial succeed.5 They are therefore receptive to evidence that appears to confirm that the medication (or placebo) is working.6 A well-recognized nonspecific effect of clinical trials is that patients may feel better as a result of the increased attention that they receive in trials.

How should we use the results of Hrobjartsson and Gotzsche's study? Quite simply, we should no longer interpret positive results among placebo-treated groups as indicating that "placebos work," except perhaps in trials involving pain control. This study reinforces the original rationale for the use of placebo in clinical trials (ie, as an attempt to control for participant and observer bias in the conduct and interpretation of clinical trials). Thus, placebo should be used to minimize bias in clinical trials, not as an effective therapy in and of itself.


AUTHOR INFORMATION


A cooperative effort of the Clinical Epidemiology Unit of the Instituto Dermopatico dell'Immacolata–Instituto di Ricovero e Cura a Carattere Scientifico (IDI-IRCCS) and the Archives of Dermatology


Michael Bigby, MD
Department of Dermatology
Harvard Medical School
Beth Israel Deaconess Medical Center
330 Brookline Ave
Boston, MA 02215
(e-mail: mbigby@caregroup.harvard.edu)


http://archderm.ama-assn.org/cgi/content/full/137/12/1639 Tweet