Replication audits have become psychology's most visible accounting exercise. They name the original authors, compare effect sizes, and report how often a published result survives a second attempt. What they almost never report is who paid for the first attempt. The grant line sits outside the audit, and that omission shapes where the costs of being checked actually land.

The Audit That Misses the Grant

A replication study typically begins with a published paper and a claim. The team reads the method, contacts the original authors for materials, registers a protocol, and collects fresh data. Every step in that chain touches the researcher who wrote the paper. None of it touches the funder who financed it.

Funding acknowledgements appear in the original article, usually as a sentence in a footnote. Replication projects rarely transcribe that sentence into their coding sheets. The result is a literature of corrections aimed at individuals, with institutional money left off the page.

The asymmetry matters because grants and careers operate on different clocks. A grant runs for three or five years and closes with a report on outputs. A researcher's reputation runs for a working lifetime, and a failed replication attaches to a name rather than a portfolio. Accountability flows to the more legible target.

What Replication Projects Actually Measure

Large multi-study efforts in psychology and behavioural economics have converged on a few standard outputs: the proportion of findings that replicate, the ratio of replication effect sizes to original ones, and the role of sample size and statistical power in predicting success. Those are useful numbers, and they have shifted how journals and departments talk about evidence.

The protocols behind these projects are often registered in advance, which is a genuine improvement over post-hoc commentary. Registration constrains the analysis to what was planned and lets readers see the decisions that were made before data arrived. It also makes clear what was never planned, including any coding of funder identity.

Effect sizes in reruns tend to come out smaller than the originals, a pattern reported often enough to be treated as a general finding rather than a surprise. Sample sizes are compared after the fact, sometimes with the original study's power calculated retrospectively. Funding source is not among the coded variables in most published audit datasets, so the question of whether grant structure predicts replicability stays open.

The coding sheets themselves are revealing. A typical audit dataset carries columns for study identifier, original effect size, replication effect size, original sample size, replication sample size, and a handful of design descriptors such as whether the replication was close or conceptual. Funder name, award number, and programme type are absent. Adding them would take an afternoon of transcription for a small batch of studies, which is a useful way to see how cheap the missing variable would be.

The Cost of Being Audited

For an original author, a failed replication is a professional event. It arrives in a journal, gets cited, and follows the researcher into review panels and hiring committees. Some respond by cooperating with replication teams, sharing materials and code; others decline, which is itself a rational response to an asymmetric risk.

Replication labor is largely unpaid or grant-funded at the margins. The teams doing the reruns are often early-career researchers building a publication record in a genre that some journals still treat as second-tier. A null result can be slow to place, and the delay costs the replicators time they could spend on their own studies.

Funders sit outside this loop. A grant that supported a finding which later failed to replicate closes on schedule, with the same final report it would have filed otherwise. The infrastructure that produced the check, meanwhile, is assembled project by project from small awards and institutional goodwill. This site has argued in a related piece that behavioural nudges raise retirement savings less than structural defaults, a finding that similarly redirects attention from individual choice to system design.

Why Funders Stay Off the Ledger

Grant reporting is built around outputs: papers, trainees, datasets, conference talks. Replicability has no field in the standard form. A programme officer can report that an award produced four articles and two doctoral graduates; reporting that one of those articles survived an independent rerun is not part of the template.

Portfolio metrics reward novelty because novelty is what review panels are asked to score. A panel judging a proposal on originality has little reason to weigh whether the applicant's earlier work held up. The incentive runs toward the new claim, and the old claim quietly becomes someone else's problem.

Audit design follows data access. Replication teams can obtain published papers and, with cooperation, original materials. They cannot easily obtain grant files, reviewer comments, or the internal correspondence that would let them link an award to a specific claim. What is measurable gets measured, and the funder's role stays in the unmeasurable column.

There is a defensible version of the funder's position. Awards are made on the quality of a proposal and the track record of an applicant, and a programme officer has no standing to police what happens to a claim years after the money is spent. Retrospective scoring of a portfolio by replication outcome would penalise funders for testing decisions made by others, and it could give agencies a reason to steer money away from the kind of novel, risky work that is most likely to be tested at all. That risk is worth naming before anyone builds the ledger.

A Cheaper Audit at the Grant Line

Linking awards to replication outcomes would not require new experiments. It would require coding. A funding agency could attach a persistent identifier to each award, require that resulting papers carry it, and then join that identifier to the public results of replication projects. The join is cheap once the identifiers exist, and some agencies already assign award numbers that could serve the purpose with modest standardisation.

Publishing grant-level replication rates would be the harder step, because it exposes portfolios rather than people. An agency that reported a low survival rate for a programme would be inviting questions about its review process. The objection is real: a funder cannot control whether a grantee's finding is later tested, and penalising a portfolio for being audited could discourage cooperation with replicators.

A middle path would report the join without ranking the portfolios. An agency could publish, for each programme, the share of funded claims that have been put to an independent test, alongside the share that survived. The first number is a measure of exposure and the second is a measure of durability, and separating them lets a reader see whether a low survival rate reflects weak work or simply a programme whose claims attract scrutiny.

Cheaper interventions exist alongside the ledger. Funding the publication of null results removes a structural bias at the source. Rewarding pre-registered designs at the proposal stage shifts effort toward protocols that can be checked. A related piece on this site notes that fluid dynamics simulations yield different vortex streets below grid resolution, a reminder that method choices made early determine what any later check can detect.

What to Ask Before Trusting a Rerun

Read the replication paper's methods before its conclusion. The protocol should be registered, with a date that precedes data collection, and the registration should be cited. If the rerun was not pre-registered, treat the result as a useful signal rather than a verdict.

Compare sample sizes directly. A replication with a fraction of the original's participants can fail for reasons that have nothing to do with the original claim being wrong. The reverse also holds: a very large rerun can detect a tiny effect that has no practical consequence.

• Check the original paper's funding acknowledgement and ask whether any replication project has coded it.
• Look for a grant-level reporting scheme at the funder, and note its absence if there is none.
• Compare the original and replication sample sizes side by side before reading the verdict.
• Ask whether the replication team had access to the original materials or reconstructed them from the text.
• Treat a single rerun as one data point about one claim under one set of conditions.

None of this makes replication audits less valuable. It locates their limit. A rerun tells you whether a claim held in a second laboratory; it does not tell you whether the money that produced the claim was well spent. The second question needs its own ledger, and the field has not built one. Until it does, the audit will keep naming the researcher and skipping the grant.