If alternative medicine wants to be taken more seriously, the studies must be better designed and be put in the proper context.
UK’s The Telegraph reported last month that a study published in the journal Complimentary Therapies in Clinical Practice showed that reflexology was “as effective as pain killers.” It’s a bold claim.
However, this claim is backed up by nothing in the study. In fact, all the methodological flaws encourage a reflexive rejection of the study’s conclusions.
No Control, No Power
You don’t have to be a scientist to know what questions to ask about a study. Some of the most basic are “What was the sample size?” and “Was it double-blinded?” Even these basic questions can tell you a lot about what researchers find.
The reflexology study had a sample of 15 participants, most of them women, and each received both experimental conditions (we will come back to this point later on). If 15 sounds like a small number to you, that’s because it is. In fact, because the statistical analyses they were using looked at group averages, this small number gets broken down even further. With so few participants, this study does not have the power to comment on very much. In larger studies, vexing variations between individuals “cancel out” to hit on some average value. Whether this study hit on something interesting or not, we wouldn’t be able to tell—values are lost in the large variations between so few people.
And what of the TENS treatment that was supposed to act as a placebo? One systematic review concluded that there is “no benefit of TENS compared with placebo.” Another review found that “evidence for the efficacy of…is limited and inconsistent,” in regards to treating chronic back pain. The New England Journal of Medicine concluded that “treatment with TENS is no more effective than treatment with a placebo, and TENS adds no apparent benefit to that of exercise alone,” also referring to treating chronic back pain.
So, according to much larger studies, there is no reason to believe that TENS does much for pain. TENS could then effectively be a placebo, but the authors of the reflexology study . . .
- Small Study of Reflexology Finds Nothing, Headline Should Read (randi.org)
- Reflexology ‘as effective as pain killers’ (telegraph.co.uk)
- Use reflexology to complement drugs in pain treatment, survey findings suggest (independent.co.uk)
- Combining Conventional Drug Therapy With Reflexology can Help in Pain Treatment (medindia.net)