What it means for a trial to call itself pragmatic
JAMA revisits a 60-year-old term as a snack-based vaccine trial shows what “pragmatic” should mean in practice
By The Weekend · · 5 min read
A trial can be rigorous and still be useless to the doctor who has to act on it tomorrow morning. That gap — between proving a treatment works somewhere and knowing it will work in your clinic, with your patients, under your constraints — is what the phrase "pragmatic clinical trial" was invented to close, nearly 60 years ago. A new JAMA analysis argues the term has drifted so far from that original purpose that it now gets slapped on studies that are pragmatic in name only. A trial published alongside it, testing a nutritional supplement's effect on measles vaccination rates, shows what the real thing looks like.
Sixty years old, and the definition still isn't settled
The distinction was drawn in 1967 by two biostatisticians, Daniel Schwartz and Joseph Lellouch, who separated "explanatory" trials — tightly controlled experiments asking whether a drug can work under ideal conditions — from "pragmatic" trials, which ask whether it does work under ordinary ones. An explanatory trial recruits a narrow, compliant population, standardizes every variable it can, and measures a biological outcome. A pragmatic trial recruits the messy population a clinic actually sees, lets real-world conditions vary, and measures something that matters to patients: did they get better, did they show up, did they survive.
The JAMA piece's complaint is that "pragmatic" has become a label applied after the fact to studies that only partly meet that bar — large sample size, maybe, or a simplified protocol, but not the full commitment to real-world conditions and patient-relevant outcomes that the term was built to describe. The result is a literature where readers can't tell, from the label alone, whether a trial's findings will transfer to their own setting.
773 villages and a monthly snack bar is what pragmatic actually looks like
The figure worth sitting with comes from a cluster-randomized trial published in Nature Medicine, testing whether a small quantity lipid-based nutritional supplement — a fortified paste handed out monthly to children aged 6 to 23 months — would raise measles vaccination coverage. It did. Children in supplemented clusters were vaccinated at meaningfully higher rates than children in clusters that got standard care alone, measured across thousands of children in a real public health delivery system, not a research clinic built for the study.
That's the headline mechanism, and it's worth restating plainly: a food handout, not a vaccination campaign, moved vaccination numbers. The supplement's job was nutrition. Its effect on immunization was a byproduct of getting families into regular contact with the health system that delivers both.
Why the number is bigger than a lab result would ever let it be
A standard efficacy trial for this supplement would have measured nutritional biomarkers — weight gain, micronutrient levels, growth curves — in a population chosen for compliance and followed under close supervision. That trial, done well, tells you the supplement works biologically. It does not tell you whether, handed out at scale through an ordinary clinic system with ordinary staffing and ordinary no-shows, it changes anything a health ministry actually tracks.
The Nature Medicine trial is structured to answer the second question. It's cluster-randomized — whole villages or clinic catchments assigned to one arm or the other, not individual children — which matters because the intervention changes contact patterns at the community level, not just the nutritional status of one child. It ran inside the existing delivery infrastructure. And it measured vaccination coverage, an outcome a health minister can act on directly, rather than a surrogate biomarker a minister would have to translate into policy. That design is precisely the definition Schwartz and Lellouch proposed in 1967, and precisely what the JAMA piece says most self-described pragmatic trials fail to achieve.
The honest objection: pragmatic trials trade precision for relevance, and sometimes that trade loses
The strongest case against over-prizing pragmatic design is that it can obscure mechanism. If vaccination coverage rose in supplemented clusters, is that because the supplement itself prompted more clinic visits, because the monthly distribution schedule created a habit of attendance, or because clinic staff used the supplement visit as a prompt to check vaccination records? A cluster trial measuring a downstream outcome often can't separate those pathways, and without knowing which one did the work, a health system can't be sure the effect will hold if it changes the distribution schedule, the supplement, or the staffing model.
That's not a reason to discount the finding — thousands of children across hundreds of clusters is a real, costed, operational result — but it is the honest limit of what a pragmatic design buys you. You learn that the combination works. You don't automatically learn why, and without the why, scaling the program to a different country's clinic system is itself an experiment, not a guarantee.
Who pays for the supplement, and who pays for not knowing why it worked
The practical next step is where the distinction the JAMA piece is defending earns its keep. A health ministry reading only the biological efficacy data on this kind of supplement would fund it as a nutrition program and expect a nutrition outcome. A ministry reading the pragmatic trial has a stronger case for funding it, partly, as a vaccination program — which changes the budget line it competes against and the metric it gets judged on.
But scaling on a pragmatic result that doesn't isolate mechanism means the next clinic system to try this is effectively re-running the trial with its own staffing, its own distribution schedule, its own baseline vaccination rates — any of which could shrink or erase the effect. The JAMA argument, in the end, isn't academic housekeeping about what words mean. It's a warning that a trial labeled "pragmatic" without meeting the original bar gives a policymaker false confidence that a result will travel. The measles-and-nutrition trial is a demonstration of the label used correctly — real population, real delivery system, real outcome — and even it leaves open exactly how much of the credit belongs to the snack, and how much to the habit of showing up that the snack created.