Send letter to journal:
Re: Fatally Flawed Study Design in Herbal Medicine Review Article

Dear Editors,

A recent article, A systematic review of randomized clinical trials of individualized herbal medicine in any indication, published in October, 2007 in the Postgraduate Medical Journal, by R. Guo , P. H. Canter , E. Ernst, of the Peninsula Medical School, Universities of Exeter & Plymouth, Exeter, U.K., warns public health authorities that the effectiveness of individualized, herbal medicine has not been scientifically established. “The findings of this review are particularly pertinent because section 12(1) of the U.K.’s Medicines Act relating to regulation of unlicensed herbal remedies made up to meet the needs of individual patients is presently under review,” the authors note. But this thinly-veiled call for regulation – publicized in the national media in the U.K., in outlets including The Independent and The Guardian, inter alia, is fatally flawed.

The authors fail to take into account the dramatic differences that animate the diagnostic and prescriptive systems of Chinese medicine, Ayurveda, and European homeopathy. The herbs used by doctors in this complementary medical systems are not used in the same way, or for the same purposes. Chinese medicine is energy medicine – and seeks to balance the body and achieve harmony for the patient. But homeopathy is decidedly different, as it uses herbs to create a catalytic reaction in the patient, and thus stimulate his immune system. Ayurveda differs from both of these approaches.

Lumping the herbal treatments of these three, separate medical systems makes no scientific sense – and, may quite literally be comparing apples to oranges. This is a disservice to readers.

To be sure, the article is not totally without merit. The authors note that evidence of efficacy for some herbal medicines, has increased substantially during the past 20 years. The authors also note that the World Health Organization (WHO) has estimated that 80% of the population in developing countries relies primarily upon herbal medicine for basic health care.

But, the authors caution, most clinical trials of herbal medicine have focused on standardized extracts of single herbs or standardized formulae. “The individualized approach, in which patients receive tailored prescriptions comprising a mixture of herbs, is emphasized in most forms of practitioner based herbalism, including European medical herbalism, Chinese herbal medicine and Ayurvedic herbal medicine,” the authors note. “ Evidence from clinical studies of single herb extracts or standardized formulae cannot be generalized to individualized herbal medicine, and claims by practitioners that the latter has an evidence base requires confirmation. The non-standardized nature of individually prepared herbal prescriptions and the consequent increased potential for adverse events and negative interactions means that safety and effectiveness need to be firmly established before such practices can be endorsed.”

It’s an interesting statement, on the surface, but seems to be a bit of pettifoggery when examined closely. The drugs dispensed by allopathic physicians cannot withstand similar scrutiny – and it has been demonstrated, for example, individual patients react differently to the same, allopathic prescriptions. Some have side effects. Some do not. Some have adverse reactions. Some do not. Some are aided in their healing. Some are not. Is this fact cause for concern? Yes. It has given rise to the field of molecular medicine, wherein scientists are seeking to develop individualized therapies for individual patients. But it is far from the norm for conventional, Western medicine.

As the article’s authors show, they scanned Medline and other medical databases and picked studies of randomized, controlled clinical trials of different herbal treatments – and only three studies that they found featured individualized herbal medicine. This was of 1,345 references in the scientific literature.

The study claims that this methodology is rigorous and that the authors have set a new standard that all herbalists must now follow.

“This study sets a new benchmark for the tailored approach: not only must herbalists demonstrate that individualized treatment is superior to placebo, they must also show, for reasons of cost and safety, that it is superior to standardized treatment,” the authors write. “

We respectfully disagree with the authors of the study that their review has somehow set a “new benchmark” for herbalism. Far from it. In fact, we are concerned that the study design, as we outlined above, is so poorly conceived that it renders its findings and recommendations for public policy makers virtually useless. To lump Chinese medicine, Ayruvedic medicine and European homeopathy into one, undistinguishable category is, simply put, poor scholarship.

Education is needed to explain to allopathic medical practitioners and scientists whose work purports to inform allopaths and public health authorities of the differences between Chinese medicine, Ayurveda and homeopathy. The risk of failing to do so is great – regulators may be driven by hysteria, engendered by ill-conceived studies, into restricting the practice of complementary medicine, to the great disadvantage of patients.

FURTHER READING

1. R. Guo , P. H. Canter , E. Ernst, A systematic review of randomized clinical trials of individualized herbal medicine in any indication, Postgraduate Medical Journal (U.K.), October 2007. 2. Ernst E, Pittler M, Wider B, eds. The desktop guide to complementary and alternative medicine, an evidence based approach , 2nd ed. Mosby Elsevier, 2006.

Send letter to journal:
Re: Systematic review of individualised herbal medicine misunderstood

Dear Editors

We thank E. J. Koprowski for her/his comments on our recent systematic review of individualised herbal medicine which reveal that s/he has totally misunderstood the nature of our review in several respects.

Firstly, we were not at all interested in homeopathy as s/he seems to believe. We can only surmise that s/he is confusing homeopathy with Western medical herbalism which was included.

Secondly, our review is not a "thinly veiled call for regulation". In the UK, consultation has already started to begin the regulation of herbal medicines and herbal medicine practitioners. Part of this process, coordinated by the MHRA, is a working party consisting of herbal practioners, the Department of Health and other interested parties who acknowledge the serious risks associated with individualised herbal medicine. Our concern is that the the issue of efficacy will be left to the "professional" judgement of the practitioners. Given that there is presently no convincing evidence for the efficacy of the individualised approach, any safety risks are unacceptable. Regulation may therefore serve to give the practice an undeserved air of respectability. We would urge policy makers to consider whether a therapy with known safety risks and no evidence of efficacy should be allowed to be practised at all.

Thirdly, we would like to point out that rigorous clinical trials testing efficacy can be conducted whatever diagnostic system is used by the therapist. One outcome of our review is the demonstration that clinical trials of good methodological quality are entirely feasible in individualised herbal medicine. We therefore reject the notion that different schools of individualised herbal medicine cannot be included in a single review. We should also point out that if we were to separate them, then each school would be supported by even less rigorous evidence than that included in our review.

Fourthly, it is not our review which has set a new benchmark for clinical trials of individualised herbal medicine, it is the results of the clinical trial carried out by Bensoussan in Irritable Bowel Syndrome. That study showed that their was no significant difference between individualised and a standardised herb mixture and if anything the standardised mixture produced better results. Herbalists need to show not only that their treatments are better than placebo, but also that their individualised approach adds value. Otherwise, exposure to the increased risks and extra cost of the treatment are not justified.

References

R Guo, PH Canter and E Ernst. A systematic review of randomised clinical trials of individualised herbal medicine in any indication. Postgrad Med J 83: 633-637

Bensoussan A, Talley NJ, Hing M, Menzies R, Guo A, Ngu M. Treatment of irritable bowel syndrome with Chinese herbal medicine. JAMA 1998; 280: 1585-89

Send letter to journal:
Re: Oriental Medicine Doctors Licensed As Primary Care Providers in the U.S.

Thank you, Peter Canter, for your e-letter in the Postgraduate Medical Journal.

We find it somewhat shocking that you think herbal medicines are no better than a "placebo." This belies the facts.

In America, in states like Florida, acupuncturists are licensed as "acupuncture physicians" and are considered by medical regulatory authorities to be primary care practitioners, who can also dispense herbs, and provide other therapies. This is not the case now in the U.K.

One certainly hopes that U.K. regulators, however, make contact with American regulators in Florida and other U.S. states where there is actual knowledge of the efficacy of traditional oriental medicine when making their decisions on new policies.

Your claim that "we therefore reject the notion that different schools of individualised herbal medicine cannot be included in a single review," is as we say in Latin, ipse dixit. Just because you say it, does not make it so. Chinese medicine differs distinctly from European herbalism and Ayurveda.

Send letter to journal:
Re: A startling inadequacy in the links between the review evidence and the authors� conclusions.

Response to: R Guo, PH Canter, E Ernst. A systematic review of randomised clinical trials of individualised herbal medicine in any indication. Postgrad Med J 2007;83:633-637

From: Ally Broughton Bsc (Hons) MNIMH Director of Research for, and Member of, the National Institute of Medical Herbalists (NIMH) Contact: ally.broughton@blueyonder.co.uk

I am a trained herbal practitioner of 12 years and teach research methodology on BSc and MSc courses in herbal medicine.

A herbal practitioner undergoes a 3-4 year degree level accredited training course which includes clinical skills and differential diagnosis, materia medica (plant medicine), and clinical experience in accredited training clinics. Professional bodies are currently self regulating and exist to ensure professional and best clinical practice.

In teaching, I use critical appraisal tools developed by the Critical Appraisal Skills Programme (CASP)(i) , I used a tool to evaluate this review to minimise bias.

Study validity Only 3 small scale clinical studies are included for review, one of which is a pilot study. The review can have no statistical significance.

The studies have very different outcome measures and investigate entirely different conditions, making comparison difficult.

Selection criteria From the 1345 references identified, 3 studies were selected. There was no clear pre-determined strategy used to determine which studies were included.

The reviewers recognise this limitation saying they had ‘difficulties designing a search strategy to locate RCTs of individualised herbal medicine because of the large number of potential descriptors for such studies’. This is a strong limitation in the review methodology.

Statistical Analysis The results for each of the studies are assessed by statistical significance, however there is no clear indication given of p values or confidence intervals used by any of the studies.

Critical Analysis The analysis focus is on statistical data. Some potentially valid points are not reviewed. For example in the Bensoussan trial(ii) there was no significant difference between the individualised and standard treatments at 16 weeks were reported, however, on follow-up 14 weeks after treatment, only the individualised treatment group maintained improvement. The reviewers entirely dismissed this as statistically non significant. It is an important point considering that individualised, holistic treatment addresses underlying causes of illness which takes longer to resolve but often with longer lasting benefit compared to symptomatic treatments.

Discussion Analysis In the discussion analysis there is consistently inadequate links made between the review evidence and the conclusions made by the authors. Discussion on herbal practice and safety issues are not substantiated or referenced, in fact there are only 7 references for the entire review.

Conclusions There is a lack of scientific rigour in the review methodology. Based on the evidence presented in the review the conclusions made by the authors appear arbitrary and insufficiently based on the evidence presented.

Designing a randomised controlled trial to investigate individual treatment is highly complex and not without limitations. Within the field of complementary medicine current research is looking for new or adapted research paradigms appropriate to holistic medicine (iii,iv) .

(i)Oxman AD, Cook DJ, Guyatt GH, Users’ guides to medical literature. VI. How to use an overview. JAMA 1994; 272(17): 1367-1371

(ii) Bensoussan A et al. Treatment of irritable bowel syndrome with Chinese herbal medicine. JAMA 1998;280:1585-9

(iii)Boon H et al. Evaluating complex healthcare systems: A critique of four approaches. eCAM Advance Access 2006; doi:10.1093/ecam/nel079 (iv) Fonnebo V et al. Researching complementary and alternative treatments – the gatekeepers are not at home. BMC Medical Research Methodology 2007;7:7

Send letter to journal:
Re: The evidence applies to all individualised herbal medicine

In reply to Dr Koprowski's letter, we would like to point out that the absence of rigorous evidence supporting the efficacy of individualised herbal medicine shown by our systematic review remains fact - regardless of how the various types of individualised herbal medicine are categorised. If we had examined only individualised traditional oriental herbal medicine, we would have found even less evidence. If there is "actual knowledge of the efficacy of traditional oriental medicine" in U.S. states this has certainly not been translated into rigorous clinical trials with a positive result.