Piling Up The Anecdotes

Piling Up the Anecdotes
 (chapter & of my book Believing Bullshit available UK here (US amazon.com here) [below is the original, uncorrected text]). New Scientist interview with me about the book here.

An anecdote involves the recounting of a short story or episode, supposedly true, and often testimonial in nature. There’s nothing wrong with anecdotes per se—they can usefully be used to spice up a dinner party conversation, provoke a discussion or illustrate a point. I’ve told a few in my time. However, alarm bells should start ringing whenever anecdotes are supposed to provide significant evidence in support of a claim, particularly a supernatural claim. Here are a few examples:

I know I’m psychic. For example, last week I was thinking about Aunt Sue, whom

I hadn’t talked to for ages, when the phone rang. And it was her.

Prayer clearly works. I prayed for Mark, John, Karen, and Rita and they all got better.

I have no doubt that ghosts are real. My mother saw one just last week. And she’s

a trustworthy woman not prone to making things up.

Anecdotal evidence is also a staple of snake-oil salesmen everywhere, who can usually produce a handful of supporting testimonials to the efficacy of their remedies: 

John ate three of my patented magic beans, and his cancer disappeared. Here’s his sworn testimony!

People are attracted to anecdotes. We especially love hearing tales of the extraordinary and supernatural. Many of us are easily swayed by anecdotal evidence for the existence of psychic powers, ghosts, or the efficacy of prayer or of some alternative medicine. Yet, as evidence, anecdotes are almost entirely worthless. Why? For a range of reasons. Here are a few examples.

Amazing Coincidence

First of all, note that amazing coincidences are inevitable. There are billions of people living on this planet, each experiencing thousands of events each day. Inevitably, some of them are going to experience some really remarkable coincidences.

Such coincidences will be thrown up by chance. The odds of flipping a coin and getting a run of ten heads by chance is very low if you only flip the coin ten times. But if billions of people do the same thing, it becomes very likely indeed that a run of ten heads will occur.

Such coincidences can easily generate the appearance of supernatural activity. For example, such coincidences can suggest that prayer can cure people of terminal diseases. Among people diagnosed with terminal cancer, a small percentage will spontaneously get better. Such rare occurrences are just a natural fact about cancer. Huge numbers of people are diagnosed with terminal cancer each year. And a significant proportion of them are prayed for. It’s likely, then, that a few of those diagnosed with terminal cancer and prayed for will recover. Is the existence of such people evidence that prayer works? Clearly not. These are people who would have gotten better anyway, prayed for or not. A handful of reports of such amazing recoveries is not good evidence of the efficacy of petitionary prayer.

What would be more impressive is if, say, after being prayed for, someone grew an amputated leg back. That’s something that really would run contrary to everything we know about how our bodies function. If, in response to prayer, God, really did heal people by supernatural means, and if his powers are unlimited, then he could just as easily grow someone a new leg as cure them of terminal cancer. However, well-documented cases of people growing legs back after being prayed for do not, so far as I am aware, exist. Interestingly, reports of “miraculous” medical recoveries tend largely to be restricted to the kinds of case in which such spontaneous remission is known to occur.

What about the phone-ringing episode? Just the other day, I was booked to play at a wedding in some fairly remote countryside about fifty miles from where I live. When I arrived, my brother walked out of the building to meet me. He was as amazed to see me as I was to see him. The venue was miles away from where either of us lived. But, by sheer chance, we ended up at the same place at the same time. A month or two ago, my wife took a train journey to a station in the North of England. When she stepped onto the platform, her father was standing there. Again, both were amazed. Again, this was a coincidence. The fact is, coincidences happen. Every now and again, people will run into each other in unexpected locations. Every now and then, the phone will ring and at the end of the line will be someone you were just thinking about.

Coincidence also accounts for at least some sightings of monsters. Consider the many thousands of people who look out over Loch Ness each year. All sorts of shapes are created in the water by floating logs, otters, wind patterns, the wakes of boats, and so on. Just by chance, a few will look a little monster-like. So, if a monster is what people are looking for, we should expect a few such reports of a monster, whether there’s a monster in the loch or not.

Those finding hidden codes in ancient texts also tend to rely heavily on coincidence. In the book The Bible Code, journalist Michael Drosnin claims to have discovered within the Bible a code revealing events that happened thousands of years after the text was written, events such as the assassination of John F. Kennedy at Dallas. Drosnin also claims no mere human could have encoded these hidden messages, and that he has therefore discovered mathematical proof that “we are not alone.”1

Has Drosnin really discovered such deliberately hidden predictive messages within the pages of The Bible? Critics note that Drosnin’s method of revealing his messages looks suspiciously as if it would throw them up by chance. Drosnin denies this. He claimed in Newsweek, “When my critics find a message about the assassination of a prime minister encrypted in Moby Dick, I’ll believe them.”2

One critic then proceeded to do just that. Mathematician Brendan MacKay subsequently used Drosnin’s method to find encrypted in Moby Dick “predictions” of the assassinations of Leon Trotsky, Indira Ghandi, Rev. Martin Luther King Jr., Rene Moawad, and Robert F. Kennedy. In fact it turns out that by using Drosnin’s method you can find such “messages” hidden in any large text. They’re thrown up by chance among the vast number of letter sequences that Drosnin’s method generates.

The important thing to remember about coincidences is this—what would be really odd is if they didn’t happen. If no one ever unexpectedly ran into a friend or relative, or if we never received phone calls from people we just happened to be thinking about—well that really would be pretty peculiar. The fact that amazing coincidences happen is, or should be, entirely unsurprising, and requires no supernatural explanation.

The Post Hoc Fallacy

People often assume that because one thing happens after another, that one is the cause of the other. But, actually, there needn’t be any causal link. To assume that, because B followed A, A caused B is to commit the fallacy known as post hoc ergo propter hoc (which means, “After this, therefore because of this”). Suppose my kettle boils immediately after a comet crashes into Mars. Did the comet cause my kettle to boil? No. That’s just a coincidence.

Similarly, the fact that someone diagnosed with terminal cancer recovers after prayer does not establish that prayer caused the recovery. To suppose otherwise is also to commit the post hoc fallacy.

Recovery after diagnosis of terminal cancer might be an amazing one-off coincidence. But what if we spot a pattern? What if, whenever A happens, B always, or very often, follows. Would that establish that A causes B? 

Suppose, for example, that some New Age medical treatment advertised like so: 90 percent of those suffering from unexplained lower back pain who took magic beans as treatment reported a significant improvement after just a few weeks!

Wow, that sounds impressive—90 percent! Surely we have here evidence that magic beans really do effectively treat lower back pain?

No, we don’t. Ninety percent of cases of unexplained lower back pain will have improved significantly after six weeks, even with no treatment at all. So the fact that 90 percent of those with unexplained lower back pain improve significantly after receiving magic beans, crystal healing, homeopathy, or a rubdown with pink blancmange is no evidence at all that any of these treatments have any sort of beneficial effect.

What we tend to overlook is the extent to which the supposed “effect” happens anyway, whether or not the alleged “cause” is present. One hundred percent of those people who drink water eventually die. That doesn’t establish that drinking water is the cause of their death.

Counting the Hits and Ignoring the Misses

Francis Bacon, a pivotal figure in the development of the modern scientific method, once said, “The general root of superstition is that men observe when things hit, and not when they miss; and commit to memory the one, and forget and pass over the other.” Anecdotes can appear to provide compelling evidence of psychic abilities and supernatural events, particularly when many are collected together in a book or article. Page upon page of anecdotes about the amazing insights of psychics can leave people thinking, “Well, there’s got to be something to it, surely!” But how is this evidence accumulated? Typically, as Bacon notes, people look for cases that seem to support the theory they believe in, and ignore those that don’t. This is called confirmation bias.

For example, someone who believes they are psychic will usually focus on the few “hits,” e.g., those times when they received a phone call from someone they just thought about. They forget about the many “misses”: all those times when they thought about someone but the person didn’t immediately ring. By collecting together several such “hits” and ignoring the innumerable “misses,” it’s not difficult to convince yourself that you have psychic powers, even if you don’t.

Similarly, someone who believes in the efficacy of prayer will typically ignore all those cases in which people diagnosed with terminal cancer were prayed for and didn’t recover—the overwhelming majority—and will focus exclusively on the handful of cases where there was a full recovery. By ignoring the “misses”—all those occasions on which sick people were prayed for but they experienced no recovery, and collect together only on the “hits,” the small proportion of occasions the person recovered, we can, again, easily convince ourselves that that we have amassed powerful evidence of the miraculous efficacy of prayer.

We can now see more clearly one of the main reasons why anecdotal evidence is such poor evidence. When we are simply presented with a large collection of anecdotes, we have no idea how idiosyncratic the cases are. If I casually take a sawn-off shotgun and pepper the side of a barn on which a small target is hanging, and a couple of shotgun pellets happen to fall inside the target, that’s not evidence of my great marksmanship. Someone that initially only looks at the two holes in the target might be impressed, but once they take a step back and see all the misses, it becomes obvious that there’s no evidence of marksmanship after all. The “hits” were highly atypical.

Not only do we tend to count the “hits” and forget about the “misses,” we also tend, when recounting anecdotes, to focus on those features that make the story sound dramatic and downplay details that make it less so. There’s often also an incentive to “sex up” anecdotes—sometimes even a financial incentive. Tabloid newspapers and TV production companies know that, as a rule, their audiences tend to be more interested in dramatic and extraordinary tales than in articles or programs that shed doubts on such stories. As a result, even while pretending to be “balanced,” TV programs on the paranormal are often little more than puffs for self-styled psychics. Doubts, if voiced at all, tend to be in the background. As a result of all this anecdote generating and peddling by the media, many people have become convinced there is abundant evidence that ghosts exist, that some people really are blessed with psychic powers, that some people have been abducted by aliens, and so on.

The Power of Suggestion and Our Tendency to “See” What Is Not There

Human beings are remarkably prone to “see” things that are not, in truth, there. Take, for example, the power of suggestion, nicely illustrated by Kenneth Arnold’s famous sighting of the very first flying saucer back in 1963. Arnold was flying his light plane near Mount Rainier in Washington when he saw a series of mysterious objects. On landing, he reported these unidentified flying objects. The news media picked up the story of Arnold’s flying saucers, and, soon after, very many other people were reporting the saucer-shaped objects in the sky. They have been reporting them ever since. The saucer-shaped spacecraft has become a staple of science fiction. But here’s the thing—Arnold did not report seeing flying saucers. What Arnold said he saw were boomerang-shaped craft that bobbed up and down, somewhat like a saucer would if skimmed across a lake. The reporter misheard, the story of “flying saucers” entered the public sphere, and other people started reporting saucers too. Why? Assuming most of them were sincere, and assuming it’s unlikely our alien visitors just happened to switch from using boomerang-shaped craft to saucer-shaped craft in 1963, it seems the saucer reports that followed were, and are, largely a product of the power of suggestion. People see something in the sky, and, because they expect it to be saucer shaped, that’s how it looks to them. Expectation strongly shapes perception.

Rigorous investigation of reports of unidentified flying objects has thrown up numerous examples of how our eyes can deceive us. In the autumn of 1967 there was a rash of reports of a UFO appearing nightly over the construction site of a nuclear plant. Sanitation workers reported it, then a guard. The police showed up. An officer confirmed, “It was about half the size of the moon, and it just hung there over the plant. Must have been there nearly two hours.” The strange object disappeared at sunrise. The next night the same thing happened. A county deputy sheriff described a “large lighted object.” An auxiliary police officer described “five objects—they appeared to be burning. An aircraft passed by while I was watching. They seemed to be 20 times the size of the plane.” A Wake County magistrate saw “a rectangular object, looked like it was on fire. . . . We figured it about the size of a football field. It was huge and very bright.” There was also a report from air traffic control of an unidentified blip on the radar scope.

When newspaper reporters arrived to investigate the mysterious object, it appeared again at 5 am. The reporters attempted to chase it in a car. They discovered that no matter how fast they drove they couldn’t get any closer. Finally, they stopped to take pictures of the mysterious object. The photographer looked through his long telephoto lens and said, “Yep . . . that’s the planet Venus alright.”3

Once the planet had been mistaken for a large hovering object by one person, well, that’s how everyone else saw it too, until, finally, someone finally looked at it through a magnifying lens and realized the truth. You might be surprised to discover that Venus is one of the biggest sources of UFO reports. Anyone who thinks that a group of honest, experienced, trained eye-witnesses—police officers, no less—can’t be seriously and repeatedly misled by the power of suggestion should think again. Also notice how coincidence threw into the mix of this story an apparent “independent” confirmation—that spurious radar blip.

It’s not just visual perception that’s affected by the power of suggestion. A auditory example, widely available on the internet, is provided by the song “Stairway to Heaven” by Led Zeppelin, one passage of which, when played backwards (easy to do on an old-style record player), is supposed to say:

Oh here’s to my sweet Satan.

The one whose little path would make me sad, whose power is Satan.

He will give those with him 666.

There was a little tool shed where he made us suffer, sad Satan.

Actually, listen to the song backwards without having seen the suggested lyrics (obviously, I’ve ruined this for you now) and people can’t make out much at all, except maybe one or two words, such as “Satan.” Play the reversed passage to people with these words in front of them, on the other hand, and they find it almost impossible not to hear the words.

How did the myth of the hidden message in “Stairway to Heaven” arise? Someone playing rock records backwards—either messing about, or actually looking for hidden messages—came across what sounds like the dramatic and noteworthy word “Satan” (thrown up by chance) in “Stairway to Heaven”, and then constructed lyrics suggested to them by the surrounding noises. Having produced the satanic lyrics, the more they listened, the more obvious it seemed to them the words were really there. The truth, of course, is that the satanic lyrics people “hear” are a product of the minds of listeners, not the mind of Led Zep’s lyricist Robert Plant.

Even setting aside the power of suggestion, various other factors can shape perception, including our obvious perceptual sensitivity to faces. Look up at passing cumulus clouds, or stare into the embers of a fire, and all sorts of things start to appear. By far the most common are faces. We are naturally attuned to them, and can easily “find” a face in most randomly generated patterns.

In 1976, the space probe Viking Orbiter 1 was busy photographing the Cydonia region of Mars. On July 25 it took a picture of what appeared to many to be an enormous alien face carved onto the planet’s surface. The Mars Face, as it become known, caused much speculation. One author, Richard Haugland, suggested in his book The Monuments of Mars: City On the Edge of Forever, that the reptilian-looking face was a vast monument created by some ancient Martian civilization, the Martian equivalent of the Great Pyramid of Giza. However, other photographs of the same region reveal that the Mars Face is just a hill that doesn’t look very face-like at all unless lit at a certain angle, when it happens by chance to take on a face-like appearance.

In fact, the Mars Face is a product of two factors: (1) Chance eventually threw up a rather face-like set of shadows among the hundreds of photographs of a planet’s surface. This face-like image was then further enhanced by (2) our tendency to “see” faces in such patterns anyway. These same two factors account for the many reports of mysterious faces appearing in things. If you have five minutes to spare, a quick trawl through the internet will reveal Mother Theresa’s face in a bun, Jesus’ face on the back of a bedroom door, and a demon’s face appearing in a cloud of smoke emerging from the Twin Towers.

The placebo effect provides another example of the power of suggestion. During the Second World War, anesthetist Henry Beecher, faced with a lack of morphine at a military field hospital, tried a rather desperate ploy. He injected a wounded soldier with inert saline solution, but told the soldier it was a powerful painkiller. Amazingly, the soldier relaxed and stopped exhibiting signs of significant pain or distress. When Beecher repeated the ploy on other soldiers, he got the same effect. We are remarkably prone to the power of suggestion when it comes to medical treatment. Tell people something will make them better—that it will relieve their pain, give their joints better mobility, reduce their acne, or whatever—and they’ll believe, and report in all sincerity, that it does. The placebo effect, as it’s known, can create the illusion that a treatment is medically effective when it is not. However, it can also contribute to the effectiveness of even bona fide medicines.

Beecher subsequently went on to publish a seminal paper, “The Powerful Placebo,”4 in which he argued for the importance of conducting double-blind, placebo-controlled clinical trials of treatments to establish their efficacy. If we want to know whether, say, homeopathic remedies have any effect other than placebo, we need two large groups into which individuals have been randomly assigned, one group receiving the homeopathic drug, the other the medically inert placebo. The trial should be double blind: the subjects should not know who is receiving the genuine treatment and who the inert alternative. The experimenters should also be blind to this information, in order to counter the “experimenter effect” (it is well established that experimenters can inadvertently influence the outcome of such trials if they know who is and isn’t receiving the genuine treatment). Unfortunately for homeopathy, such well-conducted trials have failed to provide any convincing evidence of the efficacy of homeopathic treatments for any particular ailment.

It is not just perception that can be led astray by the power of suggestion. The psychologist Jean Piaget once claimed his earliest memory was of nearly being kidnapped at the age of two while being walked in his pram by his nurse:

I can still see, most clearly, the following scene, in which I believed until I was about fifteen. I was sitting in my pram, which my nurse was pushing in the Champs Elysées, when a man tried to kidnap me. I was held in by the strap fastened round me while my nurse bravely tried to stand between me and the thief. She received various scratches, and I can still see vaguely those on her face. Then a crowd gathered, a policeman with a short cloak and a white baton came up, and the man took to his heels. I can still see the whole scene, and can even place it near the tube station.

Later, when Piaget was about fifteen, his family received a letter in which the nurse admitted the story was false:

She had made up the whole story, faking the scratches. I, therefore, must have heard, as a child, the account of this story, which my parents believed, and projected into the past in the form of a visual memory.5

Studies reveal that in somewhere between18 to 37 percent of subjects researchers can successfully “implant” false memories of events such as animal attacks, riding in a hot air balloon with one’s family, and witnessing a demonic possession.6

Other Mechanisms: Chinese Whispers, Fraud and Fakery, Etc.

Another factor that further undermines the credibility of much anecdotal evidence is what I call the Chinese whispers effect. When amazing tales are transmitted from one person to another, the retellings often involve some subtle or not so subtle editing. Those details that are dramatic tend to be remembered and exaggerated. Those that undermine the credibility of the anecdote tend to be airbrushed out. Even if each reteller reshapes the original story only slightly, it takes only a handful of retellings for the story to change significantly. So we can place even less credence in stories that reach us fourth, fifth, or sixth hand.

We should also remember that, when it comes to anecdotes about faith healing, spoon bending, mind reading, communication with the dead, and so on, many people have been revealed as frauds. In 1983, Christian healer Peter Popoff, who regularly “cured” people of serious illnesses during his revival meetings, was exposed as a cheat by magician James Randi. Popoff would wheel subjects on to the stage in wheelchairs, subjects who were then miraculously able to walk. It turned out these were people who could already walk that Popoff had simply brought on in wheelchairs. Popoff was also caught receiving information on audience members given to him by his wife via a radio earpiece.

The list of fakes and frauds is long, and includes the three Fox sisters, who helped generate huge mid-nineteenth century interest in communication with the dead. The sisters conduct séances in New York in which the dead would communicate by making rapping noises. The Foxes performed in public theaters, and their work attracted many notable people. Two of the sisters were later to admit “perpetrating the fraud of Spiritualism upon a too-confiding public.”7 Though they later retracted their confessions, Margaret had nevertheless demonstrated how she could produce the mysterious raps by cracking her toe joints at will.

Not all of the claims made about the Fox sisters’ séances were, however, a result of fraud. In some cases, the public were to add dramatic details of their own. Margaret was to say:

A great many people when they hear the rapping imagine at once that the spirits are touching them. It is a very common delusion. Some very wealthy people came to see me some years ago when I lived in Forty-second Street and I did some rappings for them. I made the spirit rap on the chair and one of the ladies cried out: “I feel the spirit tapping me on the shoulder.” Of course that was pure imagination.”8

 

Another form of fakery by psychics and mediums is the use of hot and/or cold reading. Hot reading involves research in advance. A psychic may prepare for a reading by researching their client on the internet. Some psychics will place stooges in the foyers of theaters in which they are performing to overhear the conversations of audience members, make notes, and pass information back to the psychic. Sometimes the person for whom a public reading is done will be known to the psychic, or someone close to the psychic, who may then pass on information. Sometimes stooges will join the audience, pretending to be ordinary members of the public.

Cold reading is more of an art form, and involves creating the illusion that the psychic knows things about their subject. Psychic readings typically begin like so:

PSYCHIC: I am getting someone whose name begins with “G.” George . . .

[pause] . . . Or Gerald.

CUSTOMER: Gerald! My uncle’s name was Gerald.

PSYCHIC: Yes, Gerald is here with me now. He is saying Hello!

CUSTOMER: That’s amazing!

PSYCHIC: He being quite shy, quite coy.

CUSTOMER: [No reaction]

PSYCHIC: Which is odd, because he was such an outgoing chap, wasn’t he?

CUSTOMER: Yes, that’s right. He loved the social club.

PSYCHIC: Ah, yes, he was just saying he missed his friends at there.

CUSTOMER. [Gets a little weepy] It’s really him!

PSYCHIC: I’m sensing he had some back trouble.

CUSTOMER: Yes he did! A slipped disc.

PSYCHIC: That’s right. He says that disc is all better now.

This customer may go away and tell her friends that the psychic knew she had a dead uncle called Gerald who was outgoing, missed his friends at the social club, and had a slipped disc. Her friends may well be amazed and think that perhaps there’s something to this psychic business after all.

However, our psychic, in reality, knew nothing. Let’s go through the reading again. The psychic tries a name. No reaction. Then another, and gets a hit. But she does not say whether Gerald is living or dead (it could be a message concerning a living person called “Gerald”). It’s the customer who supplies the information that she has a dead uncle of that name. The psychic then suggests Gerald is shy. No reaction, so the psychic switches to saying Gerald was outgoing, and gets another hit. The customer supplies the information that Gerald attended a social club. The psychic then suggests Gerald had back trouble. “So-and-so had back trouble” is what is known as a Barnum statement. It sounds pretty specific, but is actually true of most people. Almost everyone has back trouble at some point, so it’s not surprising the psychic gets another hit. Other examples of Barnum statements are: “You had an accident when you were a child involving water” and “You have been worrying about money recently.” Psychics will typically make lots of Barnum statements. But notice that even if Gerald’s back was always problem free, the psychic can switch tactics and say, “No, sorry, I misheard—Gerald is saying you have had some back trouble.” Even if that fails to score a hit, chances are the customer will quickly forget about it. As we have already noted, it’s the hits we remember—the misses are soon forgotten.

By using a combination of hot and cold reading, professional magicians can fake everything supposedly genuine psychics can do—often fooling audiences into believing they are genuinely psychic before revealing the truth. It is striking how closely the methods of the supposedly genuine psychics mirror the methods of such honest cheats.

However, it would be a mistake immediately to conclude that everyone who believes they are psychic—and who presents him or herself as a psychic—is a fraud. Several years ago, a friend and I played a simple mind-reading trick on another friend of ours. It’s a simple trick you can try yourself. One person holds up a playing card and then mentally “transmits” the color of the card to the another, who has to guess the color. To the amazement of onlookers, the guesser keeps getting it right. It looks like they are “mind reading,” but actually they are using a simple code: when the person holding the card says “OK,” the card is black, and when they say “Right,” it’s red. It’s a fairly obvious deception, and astonishing that people fall for it. But many do, especially if you set the trick up so that it seems to emerge as a bit spontaneous larking about. Throw in a few misses to give the scenario credibility, and the trick works better still.

What was interesting on this occasion was that, after impressing our victim with our psychic powers, we decided to test her to see if she, too, could read the mind of the card holder. She found, to her astonishment, that she could. She got more and more excited about her amazing psychic ability, until we finally had to disappoint her by revealing the truth—that she was merely subliminally picking up on the OK/Right code.

Just like my friend, some psychics lacking any genuine psychic ability may nevertheless sincerely believe they are psychic. They may be picking up on all sorts of entirely natural signals and clues without realizing they are doing so.

We may also unwittingly provide psychics with information through our body language. Consider the strange case of Clever Hans, a horse that could apparently perform mathematical calculations—tapping out the answers with his hoof. In 1907, the psychologist Oskar Pfungst conducted an investigation into the horse’s alleged mathematical abilities and discovered that the horse was not doing math, but picking up on the very subtle reactions of his human trainer. The trainer was, without realizing it, cueing the horse when to stop tapping. When the trainer did not know the answer, it turned out that neither did the horse. The Clever Hans effect, as it become known, illustrates how we can “leak” information without realizing.

“Tell Me a Story”

We have conducted a brief survey of some of the main ways anecdotal evidence for the existence of psychic powers, ghosts, alien abduction, monsters and so on can be generated. It indicates why such evidence is almost entirely worthless. You will remember in “But It Fits!” that we said evidence supports a hypothesis to the extent that the evidence is to be expected if the hypothesis is true, but not particularly expected otherwise. The evidence has to be, in a certain sense, “surprising.”

The problem with anecdotal evidence for the such extraordinary claims is that, knowing what we do about how such testimony tends to be generated, a great deal of it is to be expected anyway, whether the claims happen to be true or not. The existence of quantities of such anecdotal evidence is not, then, good evidence for the truth of such extraordinary claims.

Anecdotal evidence may be largely worthless as evidence, but it can be highly persuasive. Humans love a story, especially if it’s shocking, weird, or emotionally arresting. We enjoy comedies, tragedies, stories of wrongs righted, of revenge, of ghosts, aliens. One reason we find such stories appealing is that they tap into our tendency to feel empathy with others. We enjoy imaginatively putting ourselves in the subject’s position, imagining how it must have felt to exact that bloody revenge, see a ghost, or be abducted by aliens. The more emotional impact the story has, the more memorable it is.

As a consequence, a juicy story can psychologically trump a dry statistic, even when the statistic is rather more informative. The result of a double-blind clinical study of the efficacy of prayer is a dull set of figures easily forgotten, whereas a handful of emotionally arresting anecdotes about prayers answered may resonate with us for a long time.

The Amazingly Persuasive Power of Accumulated Anecdote

Pulling several anecdotes together can be particularly persuasive. A shower of anecdotes often explains why people become convinced that certain medical treatments are effective when they are not.

Bloodletting, popular from antiquity until the late nineteenth century, was used to treat almost every disease. In 1799, George Washington asked to be bled after he developed a throat infection. He died after large quantities of blood were removed. Benjamin Rush, a physician and one of signatories to the Declaration of Independence, was, like most of his contemporaries, entirely convinced that removing significant quantities of blood from patients helped cure many ailments. Rush, like other physicians, believed in the efficacy of bloodletting entirely on the basis of anecdotes about people being bled and then recovering. Here he cites two examples:

I bled a young man James Cameron, in the autumn of 1794, four times between the 20th and 30th days of a chronic fever, in consequence of a pain in the side, accompanied by a tense pulse, which suddenly came on after the 20th day of his disease. His blood was sizy. His pain and tense pulse were subdued by the bleeding and he recovered. I bled the late Dr. Prowl twelve times, in a fever which continued thirty days, in the autumn of the year 1800. I wish these cases to be attended by young practitioners.9

It was not until the Parisian doctor Pierre Louis conducted a controlled experiment in 1836—treating one group of patients with pneumonia with aggressive bloodletting, and another group with modest bloodletting—that the truth began to be revealed. The number of patients who died after aggressive blood letting turned out to be greater.

Or consider homeopathy. In their book Trick or Treatment, Simon Singh and Edzard Ernst (the latter is both a professor of medicine and trained homeopathic practitioner) conclude their assessment of the scientific evidence regarding homeopathy, that “it would be fair to say that there is a mountain of evidence to suggest that homeopathic remedies do not work.”10 So why do people think they work? Because of numerous anecdotes about the efficacy of homeopathy. These anecdotes are, in reality, a result of people just getting better anyway, the placebo effect, and other factors such as conventional medicines also having an effect, subjects not wanting to disappoint those interviewing them, and so on.

Many people also believe in the power of intercessionary prayer to help people through medical crises. In 2006, the American Heart Journal published the results of a $2.4 million experiment involving 1,802 heart-bypass patients, conducted under the leadership of Herbert Benson, a cardiologist who had previously suggested that “the evidence for the efficacy of intercessionary prayer is mounting” (so he was hardly biased against the claim that prayer works). The results were clear cut: prayer had no beneficial effect on the patients.11 Another large-scale trial of patients undergoing angioplasty or cardiac catheterization also found prayer had no effect.12 Unsurprisingly, such studies don’t convince all those who believe in the power of prayer. After these studies, Bob Barth, spiritual director of a Missouri prayer ministry involved in the Benson prayer experiment, said, “A person of faith would say that this study is interesting, but we’ve been praying a long time and we’ve seen prayer work, we know it works, and the research on prayer and spirituality is just getting started.”13

How did Bob Barth “know” prayers works? Apparently in the same way Benjamin Rush “knew” that bloodletting worked. On the basis of anecdotal evidence—that’s to say, various cases in which the treatment was “seen to work.”

Christian Science

It’s often said that “the plural of anecdote is not data.” And yet a pile of anecdotes can be made to look very much like solid “scientific” data. Attempts have even been made to build a science on the basis of anecdotal evidence. I will finish this chapter with a brief look at just such an attempt—Christian Science.

As Caroline Fraser, a former Christian Scientist herself, explains in her book God’s Perfect Child (on which much of this section is based) Christian Scientists believe that they have rediscovered Christianity by rediscovering the “exact method and means by which Jesus healed. Only Christian Scientists believe they can duplicate those healings systematically and repeatedly over a lifetime.”14

According to Mary Baker Eddy, the founder of the Christian Science movement, matter does not exist and disease is a product of the mind. A Christian Science practitioner’s training typically involves two weeks of religious instruction. Practitioners are not trained to diagnose illness, and in fact do not even believe in the reality of illness. The treatment carried out by trained “practitioners” of Christian Science is primarily prayer.

How do Christian Scientists know they have discovered methods that work? Because of the “scientific” evidence they have amassed over the years—tens of thousands of published testimonies of cases in which the methods of Christian Science have been applied and people have subsequently recovered. The “science” in Christian Science is meant literally. The suggestion is that the thousands of testimonies or anecdotes that the movement has accumulated over the years constitutes solid, statistical evidence that Christian Science works.

By 1989, around fifty-three thousand testimonies had been published in Christian Science periodicals. Early on, the cured were allowed to write up their own cases. More recently, Christian Science publications have prefaced such testimonials by saying:

The statements made in testimonies and articles with regard to healing have been verified in writing by those who can vouch for the integrity of the testifier or know of the healing. Three such written verifications or vouchers are require before testimony can be published.15

While this might strike some as being very “scientific,” the appearance of scientific rigor is misleading.

Notice, first of all, that the Christian Science movement ignores all cases of failure. It counts only its “hits” and ignores all its “misses.” Extraordinarily, no records of those who died after having received treatment are kept. In their “Empirical Analysis” conducted by Christian Scientists of over seven thousand treatments, the authors admitted that the study “does not provide comparative cure or mortality rates, nor does it consider cases in which healing prayer has not been effective.”16 This fact, all by itself, renders the evidence more or less worthless.

Worse still, many of the testimonials look very dubious indeed. In 1954, the academic R. W. England of the University of Pennsylvania published his analysis of a sample of five hundred letters published in the Christian Science Journal testifying to the power of Christian Science to heal. As Fraser notes, England found that the self-diagnoses of Christian Scientists were often unreliable:

The number of cancers, tumors, broken bones, and cases of pneumonia and acute appendicitis which were self-diagnosed by the writers seemed large. . . . It seems likely that most of the more dramatic cures are due simply to mistaken diagnoses. In scores of letters the writers describe how they broke their skulls, dislocated organs, awoke in the night with pneumonia, decided mysterious lumps were cancers, or found themselves in other ways serious victims of mortal mind. Their next move was to begin divine treatment, with or without a practitioner’s aid. Elated and gratified when the skulls mended, their organs returned to place, their pneumonia and cancers vanished, they wrote letters of testimony to the Journal.17

Moreover, most letters concerned fairly trivial—sometimes psychosomatic—conditions that tend to get better anyway:

Most conspicuous was an apparent ignorance of or indifference to the natural healing powers of the human body. Thus, a vast number of minor ailments, ranging from athlete’s foot to the common cold, were treated and cured by the application of Divine Truth. Furthermore, there is, among the 500 communicants, considerable attention given to types of disorders so insignificant as to be of practically no consequence so far as one’s daily life is concerned. Chapped hands, lone warts, a burned fingernail, hangnails, vague fleeting pains, a momentary dizziness were not infrequently the “healings” for which testimony was given.18

It should be fairly obvious by now why the “scientific evidence” for the efficacy of Christian Science is no such thing. It’s just a vast collection of anecdotes—tens of thousands of them—anecdotes of a sort that we might well expect to be produced by the kind of mechanisms described in this chapter, whether or not Christian Science actually works.

If Christian Science really worked, then that fact could be established by a controlled experiment, as the scientist Richard Feynman points out in his book, The Meaning of It All:

There is, in fact, an entire religion that’s respectable, so called, that’s called Christian Science, that’s based on the idea of faith healing. If it were true, it could be established, not by the anecdotes of a few people but by careful checks.19

The Christian Science movement has no interest in conducting such careful checks. They just stick with their accumulation of anecdotes, which they dress up as “science.”

Christian Science has cost lives. People can and have died as a result of their rejecting conventional medical services and plumping for the power of Christian Science instead. Hundreds, perhaps thousands, of children have received Christian Science treatment rather than conventional medicine. One of the most notorious cases is the 1979 incident involving twelve-year old Michael Shram. As Michael started showing increasingly serious gastric symptoms, his mother, a devout Christian Scientist, decided to rely on the services, not of a doctor, but a Christian Science practitioner. As a result, Michael died unnecessarily from a ruptured appendix. According to one account, four days after developing symptoms, Michael was vomiting violently and repeatedly. That night, he got up, washed his face, and brushed his teeth. He then returned to bed saying, “It’s all better Mommy,” and died. We don’t know how many children have died in this way, because Christian Science keeps no record of its failures.

Christian Science is undoubtedly an Intellectual Black Hole, and a potentially dangerous one at that. While other of the eight mechanisms described in this book also play a role in giving Christian Science a veneer of reasonableness and even scientific credibility, it is Piling Up the Anecdotes that does the bulk of the work.

Notes

1. Quoted in “Hidden Messages and the Bible Code” by David E. Thomas, which is published in Committee for the Scientific Investigation of Claims of the Paranormal, Bizarre Cases (Amherst, NY: 2000), p. 124.

2. Ibid., p. 127.

3. Philip J. Klass, UFOs: The Public Deceived (Amherst NY, Prometheus Books 1983), p 83.

4. H. K. Beecher, “The Powerful Placebo,” J Am Med Assoc 159, no. 17 (1955): 1602–6.

5. Jean Piaget, Plays, Dreams and Imitation in Childhood (New York: Norton, 1962), pp. 187–88.

6. See S. J. Lynn, T.  Lock, E. F. Loftus, E. Krackow, and S. O. Lilienfeld, “The Remembrance of Things Past: Problematic Memory Recovery Techniques in Psychotherapy.” In S. O. Lilienfeld, S. J. Lynn, and J. M. Lohr, eds., Science and Pseudoscience in Clinical Psychology (New York: Guilford, 2003). Also see: E. F. Loftus, “The Reality of Repressed Memories,” American Psychologist 48 (1993): 518–37; E. F. Loftus and K. Ketcham, The Myth of Repressed Memory: False Memories and Accusations of Sexual Abuse (New York: St. Martin’s Press, 1994).

7. Margaretta Fox Kane, quoted in R. B. Davenport, The Death­blow to Spiritualism (New York: Richardson, 2009), p. 76.

8. Harry Houdini, A Magician Among the Spirits (New York: Arno Press, 1972), p. 8.

9. From “Defence of Bloodletting,” http://www.archive.org/stream/medicalinquiries04rush/medicalinquiries04rush_djvu.txt. Accessed September 28, 2010.

10. Simon Singh and Edzard Ernst, Trick or Treatment (London: Bantam, 2009), p. 172.

11. H. Benson et al., “Study of the Therapeutic Effects of Intercessionary Prayer (STEP) in Cardiac Bypass Patients: A Multicenter Randomized Trial of Uncertainty and Certainty of Receiving Intercessionary Prayer,” American Heart Journal 151 (2006): 934–42.

12. M. W. Krucoff et al., “Music, Imagery, Touch, and Prayer as Adjuncts to Interventional Cardiac Care: The Monitoring and Actualization of Noetic Trainings (MANTRA) II Randomized Study,” Lancet 366 (2005): 211–17.

13. Quoted in Richard Dawkins, The God Delusion (London: Black Swan, 2006), p. 90.

14. Caroline Fraser, God’s Perfect Child (New York: Metropolitan Books, 1999), p. 417.

15. Quoted in ibid., p. 421.

16. Quoted in ibid., p. 425.

17. R. W. England, “Some Aspects of Christian Science as Reflected in Letters of Testimony,” American Journal of Sociology 59 (1954): 542. Quoted in Fraser, God’s Perfect Child, pp. 432–33.

18. England, “Some Aspects of Christian Science,” p. 451, quoted in Fraser, God’s Perfect Child, p. 432

19. Richard Feynman, The Meaning of It All (London: Penguin, 2007), p. 93.

---