A Real PMID Is Not Enough: What Biomedical Claim-Checking Agents Need From Their Harness

Ask a closed-book language model to back up a biomedical claim with citations, and it may hand you PubMed IDs. In one audit, 335 citations were checked: 327 pointed to real papers. A cross-family LLM judge rated only 3 of those 327 existing citations as relevant to their claims. Citations from a search agent were rated relevant 67% of the time. That is a result for the setup tested, not proof that search agents are generally trustworthy citers.¹

For a life-sciences team, the gap between “327 real” and “3 relevant” is the whole story. A valid identifier tells you a record exists; it does not establish that the paper supports the proposition attached to it. The study draws attention to the harness around the model: how it queries, retrieves, and links evidence to claims.¹

Why a valid PubMed ID can still be a bad citation

In this audit, the closed-book models’ citations mostly passed an existence check and rarely passed the relevance check. The search agent’s citations, by contrast, were rated relevant 67% of the time.¹

That split turns citation validation into two separate jobs:

• Does the identifier resolve to a real record? This can be checked by resolving the identifier.
• Does that paper support the specific claim it is attached to? This requires judging the relationship between a paper and a proposition, not just looking up the record.

The study’s finding is that checking whether a PubMed ID resolves is not enough to safeguard a scientific agent.¹

Analysis: This reframes “hallucinated citations” for literature-search systems. A citation need not be fabricated to be misleading: it can be real and retrievable while still being irrelevant to the claim it is cited for. Evaluations of biomedical claim-verification agents should measure evidence relevance directly; link validity is not a reliable proxy for it.¹

What the study actually compared

The researchers ran a controlled ablation on SciFact, using the live PubMed API as the only retrieval tool. They held the model backbone and output format fixed, varied the harness across four designs, and tested each with two efficient frontier backbones.¹

The four designs:

• Closed-book: answer without retrieval.
• One-shot retrieval: use the claim itself, verbatim, as the PubMed query.
• ReAct-style retrieval: let the agent write its own PubMed queries.
• Oracle evidence: give the agent the gold abstracts tied to the claim.

This is a narrow test of SciFact claim verification using PubMed as the sole tool.¹

The oracle condition helps distinguish reasoning over evidence from finding evidence: it gives the agent gold abstracts instead of requiring it to retrieve them. Treat it as a diagnostic ceiling, not a design to ship.¹

The component that mattered most: query formulation

One-shot retrieval—querying PubMed with the claim text itself—found the gold paper for only 6% of claims. Let the model formulate its own query instead, and gold-paper recall tripled, with verdict accuracy reaching 62%, the best result among the realistic, non-oracle harnesses. The other realistic designs landed in the 49–59% range.¹

That makes agent-written queries the standout component in this experiment. It does not establish that the same approach wins for other databases, claim types, or models; the test covered one benchmark and two backbones.¹

Analysis: A claim is not necessarily a good search query. Papers may describe findings using different terminology or framing. The 6% recall figure cautions against simply pasting claim text into a retrieval call and treating that as a sufficient search strategy.¹ If you are weighing other levers on retrieval quality, our piece on calibration and quantized text embeddings covers a different part of the retrieval stack.

The oracle condition quantifies the difference when gold abstracts are supplied. With those abstracts, the same models reached 79–86% accuracy, compared with 62% for the best realistic harness. The source describes the 17–24-point gap as significant and identifies retrieval, rather than reasoning, as the main bottleneck.¹

The practical finding is that performance improved when the models were given the relevant evidence, while the realistic harnesses had to retrieve it. Evaluate a literature-search agent on retrieval and citation support, not only on its final verdict.¹

What the study didn’t establish

The accuracy differences among the three realistic harnesses were not statistically significant at the reported sample size.¹ ReAct-style retrieval produced the best verdict accuracy, but the evidence does not establish that its ranking over the other realistic harnesses is settled.¹

The agents used fewer than half of their available search budget.¹ The available summary does not explain why. Possible explanations include that the agents did not need more searches, did not know when to continue, or were constrained by the setup; the reported result does not distinguish among them.¹

The oracle numbers are not a production forecast. Gold abstracts remove the retrieval task by design; a real system still has to find and select relevant papers. The 79–86% range describes performance with gold abstracts, not a deployed agent’s expected accuracy.¹

The available summary does not provide details such as the claim count, the identities of the two backbones, exact scoring rules, search-budget caps, or uncertainty intervals. It also does not report tests of multi-paper claims, conflicting evidence, or recent literature beyond the SciFact setup.¹

How to evaluate a biomedical claim-checking agent

Score retrieval before you score the verdict. A useful evaluation reports:

• Whether the gold paper was retrieved.
• Whether each cited paper is relevant to its claim.
• Whether the final verdict is accurate.
• Whether the agent abstains when evidence is weak rather than guessing.
• How much search budget it spends, and whether spending more improves relevance.

These checks address failure modes reported in the study: low gold-paper recall from claim-as-query retrieval, low relevance among closed-book citations, and use of less than half the available search budget.¹

Where possible, run the same four-condition comparison: closed-book, claim-as-query, agent-generated queries, and an oracle-evidence diagnostic.¹ Holding the model and output format fixed while varying the harness helps distinguish a reasoning problem from a retrieval problem.

Look at the search trace, not just the final answer. Does the agent rewrite its query after a weak first pass? Does it stop early? Would another search have changed what it cited? The study reports underused search budgets but does not explain why; trace review can help investigate that question in your own system.¹

Audit relevance directly rather than relying on identifier validity. The study used a cross-family LLM judge to assess relevance.¹ Before using an automated judge as a release gate, compare its ratings with expert human review; the available summary does not report that comparison.¹

Test beyond the reported SciFact and PubMed setup before relying on the pattern in production. Multi-paper claims, conflicting evidence, recent literature, terminology differences, and sources beyond PubMed are possible next tests, not results established by this study.¹

What to build first in a biomedical AI harness

Build a harness that writes and revises its own queries rather than assuming the claim text is a usable search string. In this study, the ReAct-style design produced the strongest realistic result, though its advantage over the other realistic harnesses was not statistically significant.¹

Make the evidence trail inspectable. Store retrieved abstracts, tie each citation to the specific claim it supports, and keep the search log. The authors report releasing code, prompts, and cached tool calls, which can help others reproduce the evaluation pattern.¹ For a broader look at how retrieval choices shape agent design, see our write-up on what SPEM changes for LLM agent architecture.

Treat abstention as a safety behavior, not a verification signal. Single-shot retrieval turned some overconfident errors into abstentions without improving accuracy.¹ Fewer confident wrong answers may be useful, but that is not the same as finding better evidence.

Analysis: Before adopting a claim-verification system, ask for the corpus, claim set, model backbones, harness variants, relevance-judging method, search-budget limits, and uncertainty reporting. Without those details, it is difficult to tell whether a benchmark result applies to your workflow. The available summary of this study does not supply all of them.¹

What’s still open

The reported test covers one benchmark, SciFact; one database, the live PubMed API; two model backbones; and four harness designs.¹ That supports a narrow conclusion: retrieval and harness design deserve first-class evaluation in biomedical claim verification. It does not establish that a particular harness generalizes across domains, databases, models, or claim types.

Two measurement gaps remain in the available summary. First, it reports a cross-family LLM judge’s relevance ratings but not a comparison with expert human reviewers.¹ Second, the headline accuracy and recall results are difficult to fully assess without details such as claim count, exact scoring rules, and uncertainty intervals.¹

The takeaway for anyone buying or building one of these agents isn’t “use this harness design.” It’s “make the agent show that it found the right paper before you trust what it says about it.” A real PMID alone does not establish that a paper supports the claim; check whether your evaluation scores relevance or stops at whether the link resolves.¹

For related reading on AI systems that work with research literature, see LitPivot and AI critique of research ideas.