If alternative medicine wants to be taken more seriously, the studies must be better designed and be put in the proper context.
UK’s The Telegraph reported last month that a study published in the journal Complimentary Therapies in Clinical Practice showed that reflexology was “as effective as pain killers.” It’s a bold claim.
However, this claim is backed up by nothing in the study. In fact, all the methodological flaws encourage a reflexive rejection of the study’s conclusions.
No Control, No Power
You don’t have to be a scientist to know what questions to ask about a study. Some of the most basic are “What was the sample size?” and “Was it double-blinded?” Even these basic questions can tell you a lot about what researchers find.
The reflexology study had a sample of 15 participants, most of them women, and each received both experimental conditions (we will come back to this point later on). If 15 sounds like a small number to you, that’s because it is. In fact, because the statistical analyses they were using looked at group averages, this small number gets broken down even further. With so few participants, this study does not have the power to comment on very much. In larger studies, vexing variations between individuals “cancel out” to hit on some average value. Whether this study hit on something interesting or not, we wouldn’t be able to tell—values are lost in the large variations between so few people.
And what of the TENS treatment that was supposed to act as a placebo? One systematic review concluded that there is “no benefit of TENS compared with placebo.” Another review found that “evidence for the efficacy of…is limited and inconsistent,” in regards to treating chronic back pain. The New England Journal of Medicine concluded that “treatment with TENS is no more effective than treatment with a placebo, and TENS adds no apparent benefit to that of exercise alone,” also referring to treating chronic back pain.
So, according to much larger studies, there is no reason to believe that TENS does much for pain. TENS could then effectively be a placebo, but the authors of the reflexology study . . .
- Small Study of Reflexology Finds Nothing, Headline Should Read (randi.org)
- Reflexology ‘as effective as pain killers’ (telegraph.co.uk)
- Use reflexology to complement drugs in pain treatment, survey findings suggest (independent.co.uk)
- Combining Conventional Drug Therapy With Reflexology can Help in Pain Treatment (medindia.net)
During a recent gathering with neighbours I found it hard to keep my cool when someone told me recent evidence had come out supporting reflexology’s credentials as a healing technique. Expressing just a touch of scepticism, ho ho, I got the irritated response that ‘science doesn’t know everything’. I’ve already treated that ‘criticism’ in my introductory ‘fountains of good stuff’ podcast, transcribed here, but I feel the need to go further in dealing with this odd line of attack, because it annoys the shit out of me.
‘Science doesn’t know everything’ is one of those semantically not-quite-right phrases that reminds me of the half-opaque lines of Ringo Starr (examples are ‘tomorrow never knows’ and ‘it’s been a hard day’s night’) that tickled Lennon and McCartney into basing songs around them. Science isn’t a sentient being as far as I’m aware – and if it is I hope it’s not a…
View original post 1,593 more words