Written by Kyle Hill via randi.org
If alternative medicine wants to be taken more seriously, the studies must be better designed and be put in the proper context.
UK’s The Telegraph reported last month that a study published in the journal Complimentary Therapies in Clinical Practice showed that reflexology was “as effective as pain killers.” It’s a bold claim.
However, this claim is backed up by nothing in the study. In fact, all the methodological flaws encourage a reflexive rejection of the study’s conclusions.
No Control, No Power
Reflexology is based on the unsubstantiated belief that each part of each foot is a mirror site for a part of the body. (source: The Skeptic’s Dictionary)
You don’t have to be a scientist to know what questions to ask about a study. Some of the most basic are “What was the sample size?” and “Was it double-blinded?” Even these basic questions can tell you a lot about what researchers find.
The reflexology study had a sample of 15 participants, most of them women, and each received both experimental conditions (we will come back to this point later on). If 15 sounds like a small number to you, that’s because it is. In fact, because the statistical analyses they were using looked at group averages, this small number gets broken down even further. With so few participants, this study does not have the power to comment on very much. In larger studies, vexing variations between individuals “cancel out” to hit on some average value. Whether this study hit on something interesting or not, we wouldn’t be able to tell—values are lost in the large variations between so few people.
To control for possible placebo effects, the researchers used transcutaneous electrical nerve stimulation (TENS) as the “sugar pill” comparison to reflexology.
Transcutaneous electrical nerve stimulation (TENS) device placed on the wrist.
And what of the TENS treatment that was supposed to act as a placebo? One systematic review concluded that there is “no benefit of TENS compared with placebo.” Another review found that “evidence for the efficacy of…is limited and inconsistent,” in regards to treating chronic back pain. The New England Journal of Medicine concluded that “treatment with TENS is no more effective than treatment with a placebo, and TENS adds no apparent benefit to that of exercise alone,” also referring to treating chronic back pain.
So, according to much larger studies, there is no reason to believe that TENS does much for pain. TENS could then effectively be a placebo, but the authors of the reflexology study . . .
Related articles
- Small Study of Reflexology Finds Nothing, Headline Should Read (randi.org)
- Reflexology ‘as effective as pain killers’ (telegraph.co.uk)
- Use reflexology to complement drugs in pain treatment, survey findings suggest (independent.co.uk)
- Combining Conventional Drug Therapy With Reflexology can Help in Pain Treatment (medindia.net)
Testimonials and anecdotes are used to support claims in many fields. Advertisers often rely on testimonials to persuade consumers of the effectiveness or value of their products or services. Others use anecdotes to drive home the horror of some alleged activity or the danger of widely-used electronic devices like cell phones. In the mid-90s, there were many people, some in law enforcement, claiming that 
