Anecdotal evidence

I started a big old debate when I criticised a supplement called Digestrol a while back, and some people strongly disagreed with my view of the product, saying that it had worked for them so I shouldn't criticise it.

Now, I'm quite happy to hear opposing opinions, and if any treatment works for you then that's grand. What I have been a bit dismayed about is the number of people who seem to believe wholeheartedly in the power of anecdotal evidence, and the flimsiest of anecdotal evidence at that.

Anecdotal evidence is just what it sounds like - evidence which is based on anecdotes, or word-of-mouth information. So, my friend Bob might tell me that a particular garage is fantastic at fixing cars, because they fixed his car last week, and now it works. That's all there is to it.

The argument is that if Bowlex works for you, or it works for a handful of other sufferers, then this is proof that Bowlex works at least some of the time, and therefore it is beyond criticism. Now, as the owner of an IBS website I hear masses of anecdotal evidence, and I therefore have proof that every kind of treatment from crystal dowsing to the wearing of magnetic bracelets works for IBS. People have told me so; it must be true.

A sister argument says that I cannot have an opinion on any IBS supplement that I have not tried myself, because if I haven't personally tried it there's no way for me to give a personal opinion (or no way for me to tell an "anecdote"). I must personally quoff every last pill and potion available, observe my own symptoms, and then report. There is no other righteous way.

We often use anecdotal evidence to make decisions. But if we rely on anecdotal evidence alone we're opening ourselves up to all kinds of problems, and this is especially true if we rely on anecdotal evidence when we're choosing a new IBS treatment. Here's why...

Bob tells you that the garage he went to is fantastic. His car was broken, and now it isn't. So that's great. This is proof that the garage have some kind of skill at fixing cars, and it is proof that on this occasion they managed to fix Bob's car.

But what about if you did a survey of the last 1,000 customers who used the garage? If you got 1,000 replies strongly agreeing with Bob, then that's all the evidence I would need as a consumer to tell me that that's a great garage.

You probably wouldn't get that though - you'd probably get some people who agreed with Bob, some who disagreed, and some in the middle. The number of people in each category would tell you how good the garage actually was. You might get 999 people who say the garage painted their cars bright pink and they're really not that happy about it, and one person (Bob) who was completely happy in every way.

So - the evidence of one person is not really evidence at all. And if you apply this to medicine, this means that if one person tells me that wearing sofa covers on their heads has cured their diarrhea, this doesn't in fact mean that sofa-cover-(on-the-head)-wearing is a legitimate IBS treatment.

And, in the case of IBS, and indeed other medical conditions, there are many other reasons why I shouldn't be convinced by the evidence of one person. First, IBS symptoms come and go, so we sometimes assume our new Bowlex pills are working when really we'd feel just the same without them.

Next, the placebo effect applies to all kinds of medical treatments, and we all know how powerful sugar pills and even fake surgeries can be - when we think we're going to feel better, we often do.

Next, IBS symptoms vary a great deal between sufferers. For example, I have IBS-C with a bit of diarrhea thrown in sometimes. I tried hypnotherapy twice, and it didn't do anything for me. Proof that hypno doesn't work for IBS? Or proof that hypno doesn't work for me? Conversely, I find a gluten-free diet very helpful. Is that proof that gluten-free diets work for IBS?

Now, I do have to say that there are times when anecdotal evidence is very useful. Indeed, my whole IBS Treatment site is based on the idea that anecdotal evidence can be helpful (although I try to post as many reviews of each product as possible, and I also include clinical trial data whenever it is available).

But it is important to recognise the limits of this kind of evidence. Anecdotes are great for getting new ideas for treatments and for comparing notes. Some forms of anecdotal evidence are backed up by science or commonsense which makes them stronger - for example, there are a lot of good reviews of calcium supplements for diarrhea on my site, and that seems logical as calcium is naturally constipating.

But do these reviews alone prove that calcium is a good treatment for IBS? Nope.

The gold standard of evidence is, of course, the good old randomised double-blind placebo-controlled trial, which was developed specifically to reduce or eliminate the influence of things like the placebo effect and the fact that the human body varies from day to day.

Without these trials, without this objective way to evaluate treatments, every time you went to the doctor he would just have to start at the top of a list of IBS treatments and work downwards until something randomly worked. (And OK, yes, that does feel like what some doctors do anyway - except that without the controlled trials, the list wouldn't read "Anti-spasmodics, fiber supplements, hypnotherapy", it would read "Aardvark juice, artichoke puree, bear fur souffle, dog biscuits".)

To be honest, I really don't see why anyone would be happy to rely on anecdotal evidence alone as proof of any treatment's efficacy. Perhaps this is especially true for me because I receive so many anecdotes.

If I were to rely on anecdotal evidence, and I subscribed to the view that there was no objective way to evaluate treatments, you just had to swallow them and see what happened, I would spend my entire time and all my life savings buying pill after pill after magnetic bracelet. And I bet I'd still feel crappy in the end.