M11 · REAL-WORLD EVIDENCE
Can you sit in the middle?
Last lesson ended on a slider that felt like a forced choice. Slide toward control and you get a randomised trial: clean causal proof, but an idealised world that doesn't resemble routine care. Slide toward realism and you get real-world evidence: real patients and real outcomes, but no randomisation, so confounding floods back. Protection or realism — pick one.
But that framing hides a possibility. What if you didn't slide all the way to either end? What if you kept the one thing that makes a trial trustworthy — randomisation — but ran it in conditions much closer to real life? Broad patients instead of idealised ones. Ordinary clinics instead of expert centres. Outcomes that matter to patients instead of short-term surrogates. You'd be randomising inside the real world.
That study exists. It's called a pragmatic trial, and it's the middle of the slider made real — not a compromise that gives up rigour, but a deliberate attempt to keep the randomisation and trade away only the artificial conditions.
Explanatory vs pragmatic: same randomisation, different question.
Trials come in two flavours, and the names are worth learning because they organise this whole space.
An explanatory trial is the classic RCT you already know. It asks: can this intervention work under ideal conditions? It carefully controls everything — hand-picked patients, strict protocol, close monitoring, often a placebo comparator — to isolate the pure biological effect. It's built to measure efficacy.
A pragmatic trial asks a different question: does this intervention work when we actually deploy it in normal practice? It loosens the controls — broad patients, flexible delivery, routine care, a real-world comparator — to measure what happens in the messy real world. It's built to measure effectiveness.
Here's the point that everything hinges on: both of them randomise. Both are RCTs. Both hand out treatment by the toss of a coin, so both defeat confounding and both support a clean causal claim. The difference between explanatory and pragmatic is not whether they randomise — it's the conditions they randomise under, and therefore the question they answer. Explanatory controls the world to ask "can it work?"; pragmatic embraces the world to ask "does it work?" Same shield, aimed at a different target.
Pragmatic ≠ real-world evidence.
This is the single most important distinction in the lesson, and the one most often got wrong: a pragmatic trial is not real-world evidence.
They look similar from a distance — both live in realistic, routine-practice conditions, both care about effectiveness, both enrol the kind of patients you'd actually treat. But there's a wall between them, and the wall is randomisation.
- A pragmatic trial randomises. However realistic its conditions, it still assigns treatment by chance — so the treated and untreated groups are comparable, and confounding is defeated. It earns a causal claim.
- Real-world evidence does not randomise. It observes what happened to patients who were treated for reasons — and those reasons contaminate the comparison. It's exposed to confounding, and can only claim association.
So a pragmatic trial is the best of the arrangement from last lesson: real-world-like conditions, but with the causal shield of randomisation still up. It captures effectiveness and keeps the protection. That's why it sits genuinely in the middle of the slider — realistic like RWE, causally sound like an RCT — and why calling it "real-world evidence" throws away the very thing that makes it special. The randomisation is not a technicality; it's the whole difference between "the drug caused this" and "this happened near the drug."
Pragmatism is a spectrum of dimensions.
There's a second subtlety, and it dismantles the idea that a trial is simply "pragmatic" or "not." Pragmatism isn't a switch — it's a set of dials, and a trial can be turned up on some and left down on others.
Think of the separate choices that make a trial more explanatory or more pragmatic:
- Who's included — narrow, idealised patients (explanatory) or the broad, messy real population (pragmatic)?
- How the intervention is delivered — a rigid fixed protocol (explanatory) or flexibly, as a clinician actually would (pragmatic)?
- Where — expert centres with intensive monitoring (explanatory) or ordinary clinics (pragmatic)?
- Against what — placebo (explanatory) or usual care, the real alternative (pragmatic)?
- What's measured — a short-term surrogate endpoint under close supervision (explanatory) or a patient-relevant outcome tracked routinely (pragmatic)?
Each of these is an independent dial. A trial might enrol a broad population (pragmatic) but measure a surrogate endpoint under tight monitoring (explanatory) — a hybrid. So "how real is this evidence?" is never a yes/no; it's a profile across several dimensions. (There's even a formal tool, PRECIS-2, that scores a trial on nine such dials.) The upshot: a single pragmatic-sounding feature — "it used usual care!" — doesn't make a trial pragmatic overall, any more than one strict feature makes it explanatory.
Tune the trial.
A pragmatic trial isn't one thing — it's a set of independent choices. Set each dial toward the idealised (explanatory) end or the real-world (pragmatic) end, and watch where the trial lands overall, what it ends up measuring, and what it costs you.
Eligibility
Intervention
Setting
Comparator
Outcome
Overall profile
This trial is: EXPLANATORY · measures EFFICACY — can it work? · precision high · smaller sample needed
Notice two things. First, there's no single switch labelled "pragmatic" — the trial's character is the aggregate of independent choices, and you can build genuine hybrids (broad patients, but a tightly-measured surrogate). Second, as you push toward realism, the readout warns you about cost: a more pragmatic trial measures something more useful (real effectiveness) but with more noise, so it needs a bigger sample to see a smaller, diluted effect. Every dial you turn toward the real world buys relevance and spends precision — even though the randomisation stays firmly on the whole time.
Now you.
Each line describes one feature of a trial. Tag each as explanatory (idealised, controlled) or pragmatic (real-world, routine).
1. "Enrols only patients aged 40–65 with no other illnesses."
2. "Compares the drug against usual care, not placebo."
3. "Measures hospitalisations and deaths over 5 years."
4. "Delivers the drug under a strict fixed protocol with adherence monitoring."
5. "Recruits from ordinary clinics across the whole eligible population."
6. "Uses a surrogate endpoint measured every two weeks under close supervision."
Realism isn't free (even with randomisation).
It's tempting to conclude the pragmatic trial is simply better — realistic and randomised, the best of both worlds at no cost. It isn't. Loosening the controls lets reality in, and reality is noisy.
When you broaden the patients, some respond less. When you deliver the drug flexibly, adherence slips. When you use routine clinics, care varies from site to site. All of this adds noise and often shrinks the measured effect — not because the drug got worse, but because you're now measuring it through the friction of real practice. The consequences are concrete:
- You need a bigger, longer, costlier trial to detect a smaller, diluted effect through all that noise.
- The effect is harder to interpret. In an explanatory trial, "the intervention" is a clean molecule. In a pragmatic one, "the intervention" is really the whole implementation package — the drug plus how it was delivered, adhered to, and supported in practice. If the result is disappointing, was it the drug, or the delivery? Harder to say.
- Blinding is often impossible. You can't blind a flexible, real-world intervention the way you blind a placebo pill — which reopens some of the biases the explanatory design controlled.
So the pragmatic trial doesn't escape the trade-off from last lesson — it relocates it. You keep randomisation's protection against confounding, but you pay in precision, cost, and interpretability. There's still no free lunch; there's just a different bill.
Why HTA loves them (and why they're rare).
For health technology assessment, the pragmatic trial is close to an ideal witness — and understanding why also explains why you don't see more of them.
Why HTA prizes them: a decision-maker wants to know the real effect a technology will have when funded and rolled out — that's effectiveness — and they want it causally credible, not confounded. A pragmatic trial delivers exactly that unusual combination: a real-world effectiveness estimate carrying the causal authority of randomisation. It answers "what will we actually get, and can we believe it caused the difference?" in one study. Nothing else does both.
Why they're nonetheless rare:
- They're expensive and slow. Large, broad populations followed for years to patient-relevant endpoints cost far more than a tight efficacy trial.
- They're hard to run cleanly. Flexible interventions resist blinding and standardisation; real clinics are harder to coordinate than expert centres.
- The incentives point elsewhere. Regulators (EMA, FDA) mainly demand efficacy for approval, so that's what manufacturers build first. A pragmatic effectiveness trial is an extra, later, voluntary expense — often only run when a payer or HTA body specifically asks for it.
The result is a familiar tension: the study design HTA would most like to see is the one the system is least set up to produce. Which is exactly why HTA so often falls back on the other way of chasing real-world effectiveness — observational real-world evidence, with no randomisation at all. And that raises the hardest question in the module: when you can't randomise, how do you make a causal claim anyway?
Which is the pragmatic trial?
A submission includes two studies of the same drug. Study 1: a large database analysis of 80,000 treated patients versus 80,000 untreated, showing better survival on the drug. Study 2: a 3,000-patient trial that randomised patients to the drug or usual care, in ordinary clinics, tracking a patient-relevant outcome. Which is the pragmatic trial, and why does the distinction matter?
Why this matters for HTA
Pragmatic trials occupy a prized but awkward place in the evidence a submission can offer, and reading them well takes a few habits:
- Check that "real-world" evidence is actually randomised. The word "real-world" gets attached to both pragmatic trials and observational RWE, and they are worlds apart on causal strength. The first question is always: was treatment randomised? If yes, it's a pragmatic trial with a causal claim; if no, it's RWE exposed to confounding, whatever its size or realism.
- Read pragmatism dial by dial, not as a label. A study described as "pragmatic" may be pragmatic on only one or two dimensions. Look at each — patients, comparator, setting, outcome, delivery — and judge how realistic the evidence genuinely is. A usual-care comparator bolted onto an otherwise idealised trial doesn't make its result real-world.
- Weigh the effectiveness estimate against its noise. A pragmatic trial's effect is more relevant but noisier and often smaller than an explanatory one's. Don't mistake a diluted real-world effect for a failed drug, or demand explanatory-trial precision from a design that deliberately traded it away for realism.
A pragmatic trial keeps the one thing that makes a trial believable — the coin toss — and spends everything else to buy realism. It's the closest medicine comes to answering "does it actually work, out here?" without letting go of "and can we prove the drug is why."
Pragmatic trials, in one breath.
- A pragmatic trial sits in the middle of last lesson's slider: it keeps randomisation (so it defeats confounding and earns a causal claim) but runs in realistic conditions to measure effectiveness — "does it work in practice?" — rather than efficacy.
- It differs from an explanatory trial not by randomisation (both randomise) but by conditions and question; and it differs from real-world evidence by the one thing that matters most — a pragmatic trial randomises, RWE does not. Pragmatic ≠ RWE.
- Pragmatism is a spectrum of independent dimensions (eligibility, delivery, setting, comparator, outcome), not a label — a trial can be pragmatic on some dials and explanatory on others.
- Realism still isn't free: loosening control adds noise, so pragmatic trials need bigger samples, yield diluted, harder-to-interpret effects, and often can't blind — and they're expensive enough, and so misaligned with regulators' demand for efficacy, that they stay rare.
The explanatory trial asks whether a drug can work in a world we build for it. The pragmatic trial keeps the coin toss but tears down the walls, and asks whether it works in the world we actually have.
Pragmatic trials are the ideal answer to "real-world effectiveness, causally proven" — but they're a luxury: costly, slow, and often simply not done. Far more often, all HTA has is observational data — patients who were treated for reasons, with no coin toss anywhere. So the module's hardest question is unavoidable: when nothing was randomised, how can you make a causal claim at all? That's causal inference, next.