Turning a blind eye to blinding

Blinding, sometimes called masking, is the process of keeping key persons involved in the conduct of a clinical trial unaware of group assignment. In a typical randomized controlled trial, neither patients nor therapists know who is getting the active treatment and who is getting a placebo pill. The idea behind this is simple: knowledge of who’s getting what can influence the results. If patients know, for example, that they are getting a promising treatment they might be more optimistic about their current health status (a placebo-effect) or they might report symptoms according to what they think will please the investigators (response bias). Blinding has gradually become standard practice since the 1950s, even though its principles were well understood before that.

Mesmerized

To understand the importance of blinding, one has to go back to France at the end of the 18^th century. In the last years of the Ancien Regime, the court of King Louis XVI was enchanted by what was called ‘animal magnetism’. The German physician Franz Anton Mesmer, had proclaimed that this invisible natural force present in all living things, formed the key to sickness and health. By the use of magnets and their proclaimed healing powers, Mesmer and his followers believed they could influence the mysterious fluid running through the body and cure patients. King Louis XVI, however, had his doubts about the proclaimed wonders of this new treatment. He ordered a commission of wise men to investigate if it had any merit. The Royal Commission assembled included famous scientists such as chemist Antoine Lavoisier and the American polymath Benjamin Franklin.

The royal commission

The commission did several experiments and quickly understood that the effects of mesmerism were caused by suggestion and imagination. The decisive experiment was to blindfold patients in whom mesmerism seemed to work. When these patients were blindfolded and told they were being mesmerized, they reported all sorts of effects even though nothing was being done to them. When someone did mesmerize them without their knowledge, they reported no effects. The conclusion of the commission was insightful and remains relevant to this day:

“Let us take the standpoint of a commoner, for that reason ignorant, struck by disease and desiring to get well, brought with great show before a large assembly composed in part of physicians, where a new treatment is administered which the patient is persuaded will produce amazing results. Let us add that the patient’s cooperation is paid for and that he believes that it pleases us more when he says he feels effects, and we will have a natural explanation for these effects; at the least, we will have legitimate reasons to doubt that the real cause of these effects is magnetism.”

An unfair competition

What the commissioners didn’t do is randomize patients to mesmerism and a (passive) control condition to see which group reports the largest improvement on symptom questionnaires. If mesmerism was compared to relaxation therapy or a waiting list control condition it’s quite likely that it would appear to be effective. As the commissioners explained, mesmerism involved a great show in which patients were persuaded by physicians that it will produce amazing results. It was a successful recipe to create placebo effects and response bias rather than a method to improve patients’ health.

Although it is still frequently used today, randomizing patients and organizing a competition to see which intervention causes the largest improvements in reported health, is not a fair test. It’s a competition that can easily be won by interventions that are more aggressive in misleading patients or in instructing them to be more optimistic about their health. What is needed is a method that accounts for bias caused by the expectations of trial participants, something the royal commission had understood more than two centuries ago.

BRANDO: the empirical evidence

Today, we have empirical evidence that a lack of blinding leads to bias and overestimation of treatment effects. Researchers study this by looking at meta-analyses. These provide an overview of randomized trials that studied the same treatment for the same patient group. So if a meta-analysis contains both trials that used and didn’t use blinding,  one can get a rough estimate of the effect of blinding on the results. Several research teams have tried this and, luckily, they have agreed to pool their results into one big database for a project called BRANDO (Bias in Randomised and Observational studies). The results show that unblinded trials overestimate treatment effects but mostly on subjective outcomes. For objective outcomes such as mortality, the effect is much smaller to negligible. According to the authors, the data give a clear message about how trials should be conducted and interpreted:

“Our results suggest that, as far as possible, clinical and policy decisions should not be based on trials in which blinding is not feasible and outcome measures are subjectively assessed. Therefore, trials in which blinding is not feasible should focus as far as possible on objectively measured outcomes, and should aim to blind outcome assessors.”​

This conclusion is supported by a 2014 review by the Agency for Healthcare Research and Quality (AHRQ) and an interesting study that used an alternative approach to measure the influence of blinding. Hrobjartsson and colleagues looked at studies that had both blinded and unblinded groups within the same trial. These studies were pretty rare; the authors only found 12 and most were about acupuncture. Nonetheless, the conclusion was similar to that of the BRANDO project. The average difference in effect size for patient-reported outcomes was 0.56, meaning that in trials with a moderate effect size, a lack of blinding can cause an exaggeration of the estimated effect by more than 100%.

A gratifying error

For reasons that remain obscure to me, open-label trials with only subjective outcomes are still popular and well-respected. The unfair competition is still allowed to proclaim its winners. Franklin, Lavoisier and their fellow commissioners wouldn’t be surprised. They understood mesmerism was but an old trick in a different disguise. One that is perhaps too tempting not to be reused. In their report they wrote:

“Magnetism therefore is only an old error. This theory is being presented today with a more impressive apparatus, necessary in a more enlightened century; but it is not for that reason less false. Man seizes, abandons, takes up again the error that gratifies him. There are errors which will be eternally dear to humanity.”