The value of acupuncture to treat chronic pain is a subject of fiery debate, fueled by the difficulty of rigorously determining whether the therapy works. To try to cut through the confusion, researchers in the United States and Europe have analyzed individual patient data from 29 trials, totaling almost 18,000 people with diverse types of chronic pain. The meta-analysis indicates that acupuncture offers a clear benefit over usual-care controls, and a much smaller, but still statistically significant, benefit compared to sham acupuncture controls. The results suggest that acupuncture may elicit some specific response, and confirm the notion that the bulk of its clinical advantage comes from placebo or other nonspecific effects. The study, from the Acupuncture Trialists’ Collaboration, was published online September 10 in Archives of Internal Medicine.
The authors conclude that their data “provide the most robust evidence to date that acupuncture is a reasonable referral option for patients with chronic pain.”
But far from settling the debate, the results raise fundamental questions: Do the differences reported translate into real improvements in pain or functioning for patients? And, if the bulk of pain relief from the treatment is due to placebo effects, should doctors recommend the therapy to their patients?
A massive meta-analysis
Although acupuncture has been used for thousands of years, researchers have had a hard time demonstrating its effectiveness using modern placebo-controlled, double-blind clinical trial designs. Carrying out a rigorous trial is difficult: Designing appropriate sham treatments for comparison is challenging. Keeping subjects and providers blind to the treatment status is impossible. In addition, variations in technique and sham procedures make trials hard to compare.
Recent Cochrane systematic reviews and other meta-analyses involving data on several thousand patients each have found acupuncture offers benefits over usual care, but have disagreed on whether true acupuncture has more effect than sham interventions—and whether any of the measured differences are large enough to be clinically relevant (Manheimer et al., 2010; Linde et al., 2009; Linde et al., 2009; Madsen et al., 2009).
Andrew Vickers, a statistician at Memorial Sloan-Kettering Cancer Center in New York City, US, and lead author on the new study, set out to make the most of the available trial data by pooling results from 29 randomized controlled trials of acupuncture for several types of chronic pain: back or neck pain, shoulder pain, osteoarthritis, and chronic headache (including migraine). He and his colleagues included only trials that met pre-specified quality criteria, including the use of appropriate control groups and sufficient blinding. For controls, trials were required to have had at least one group that received some form of sham acupuncture (such as inserting needles only superficially) or a usual care group (which varied substantially among trials, from unspecified care to specific medication advice).
However, rather than relying on summary information from trials, as most meta-analyses do, Vickers and his colleagues collaborated with trialists to obtain the raw data from individual patients (as outlined in Vickers and Maschino, 2009, and Vickers et al., 2010).
Compared to analysis of summary data, the patient-level approach “is helpful because a) it allows more powerful and precise statistical analyses to be conducted, and b) it allows more trials to be combined,” Vickers wrote in an e-mail to PRF. For example, some trials report results as mean changes in pain or function scores, while others report response rates (the percentage of patients who experience a given reduction in pain score). Those results cannot be combined in a typical meta-analysis, but with access to the original patient data, Vickers and colleagues were able to convert the different outcome measures to a common scale by expressing the changes with treatment in terms of standard deviation (SD) units.
A little better than sham, and a lot better than nothing
Pooling all the trial data revealed that, on average, patients who received acupuncture had less pain compared to the usual-care controls (0.42–0.57 SD decrease, depending on pain type). The benefit for acupuncture over sham controls was much smaller (0.15–0.37 SD), but still significant. Three trials from one research group showed unusually large effects of acupuncture; when the investigators excluded those, they report, the data for acupuncture versus sham were quite consistent among trials, and similar for different pain types (0.23 SD for back and neck pain, 0.16 for osteoarthritis, and 0.15 for chronic headache).
“For me, the most interesting thing was that, after eliminating a few outliers, the results in comparison to sham controls are so consistent,” said Klaus Linde of the Institute of General Practice, Technische Universität München, Germany, and an author on the study. “I think this analysis brings important new confirmation that we are now getting a clearer picture of what’s going on.”
“The evidence suggesting that acupuncture was more than sham was previously pretty equivocal. Our data clearly show highly significant differences between groups that are unlikely to be explained by bias,” Vickers e-mailed.
While the numbers show statistical significance, the real benefit for patients is harder to assess. Based on the reported effect sizes, the authors calculate that, for example, in a trial where patients entered with an average pain score of 60 out of 100, follow-up scores might be on average 43 in the no-acupuncture group, 35 in a sham group, or 30 in the acupuncture group. Or, if response is defined as a 50 percent reduction in pain, they calculate the response rate would be 50 percent in the acupuncture group, compared to 30 percent in the usual-care controls and 42.5 percent in the sham group.
The authors call the effect of acupuncture over usual care (~0.5 SD) “of clear clinical relevance.” But that is a problematic analysis, says Andrew Avins, Kaiser Permanente, Oakland, California, US, in a commentary that accompanies the study. “Determining a clinically relevant effect size is a contentious exercise, and the clinical relevance of an average 0.5 SD change is uncertain and likely varies with the measure used and the outcome being assessed,” he writes.
What’s a doctor to do?
For clinicians and patients, the question remains: Is acupuncture worth a try? The new analysis provides no ready answers. As Avins points out in his editorial, proponents of acupuncture can argue that the effects detected in the study prove that the therapy is worthwhile, whereas skeptics will point to the fact that most of the benefit can be conferred by sham treatment as proof that acupuncture is little more than placebo.
But even if much of acupuncture’s benefit does involve placebo effects, or nonspecific effects that are replicated by sham treatment, Avins argues, that does not mean that physicians should shy away from it. “It’s ideal to understand the mechanism of action, which carries the potential for developing more and better interventions. But the ultimate question is, Does this intervention work (or, more completely, do its benefits outweigh its risks and justify its cost)? At least in the case of acupuncture, Vickers et al. have provided some robust evidence that acupuncture seems to provide modest benefits over usual care,” he writes.
Edzard Ernst of the University of Exeter, UK, who was not involved in the current study, takes a more skeptical view. “If we treat our patients with compassion, empathy, time, and dedication using treatments that are effective beyond a placebo effect, they will benefit from both non-specific and specific therapeutic effects … merely relying on non-specific effects as the main element of a therapy is not in the best interest of our patients,” he writes (see Ernst’s full comment below).
Furthering the evidence
While the debates continue, Linde, Vickers, and colleagues continue their analysis. “What’s been published now is only a first step,” Linde said. The approximately 18,000 patients included in the current patient-level meta-analysis come from trials conducted through November 2008, and the researchers say they will continue to add new trial data. Having patient-level data will also allow the collaborators to investigate whether individual patient characteristics correlate with outcome. Vickers and Linde say the team is exploring the impact of variables such as patient age, as well as outcomes over time.
Comments
Edzard Ernst, University of Exeter
This meta-analysis is
This meta-analysis is technically well made, yet it has numerous weaknesses, too. For instance, it relies heavily on the German acupuncture trials which attracted much justified criticism when they were first published. Even if we disregard the limitations of this meta-analysis, its results can be interpreted in more than one way. The authors, who are all well-known proponents of acupuncture, seem convinced that they have provided the ultimate proof that acupuncture is more than a placebo. More critical experts are likely to arrive at the opposite conclusion.
In my view, this meta-analysis demonstrates that acupuncture is not a useful routine treatment for chronic pain. Compared to sham acupuncture, the effect is tiny and not clinically relevant. Moreover, it seems probable that the tiny effect is not real but due to residual bias, for example, caused by the fact that acupuncturists were not blinded.
Acupuncture enthusiasts are trying to tell us that such arguments are academic because the more meaningful comparison should be between acupuncture and no acupuncture, where the effect size is larger. In such a comparison, the placebo effect is counted in favor of acupuncture. In other words, any placebo therapy would be justified with such an approach simply because of its non-specific effects.
While this argument looks convincing and patient-centered at first glance, it turns out to be short-sighted and misguided on closer inspection. If we treat our patients with compassion, empathy, time, and dedication using treatments that are effective beyond a placebo effect, they will benefit from both non-specific and specific therapeutic effects. In other words, we need no acupuncture or any other placebo treatment to convey a placebo response, and merely relying on non-specific effects as the main element of a therapy is not in the best interest of our patients; in fact, it is arguably unethical and regressive.