Evidence-grounded benchmarking of retrieval-augmented generationarchitectures for scientific claim verification

Abstract

Relevance. Retrieval-augmented generation is usually evaluated with proxy metrics that ignore evidence grounding, and new architectures are rarely compared with strong baselines under equal conditions. The purpose of the article is to determine, under one protocol, whether agentic, graph-based, and corrective architectures improve evidence retrieval and verdict accuracy over lexical, dense, hybrid, and reranking baselines in scientific claim verification on abstracts, and at what cost. Tasks: formalise the protocol as an algorithm; implement eight pipelines on one sentence-level index of a public benchmark of biomedical abstracts with annotated evidence; evaluate them on a fixed subset with repeated runs and on the development split; quantify cost. Methods: shared embedding model, generation model, and output contract; ranking, evidence, and per-class verdict metrics with bootstrap intervals and paired tests corrected for multiple comparisons. Scientific novelty: an evidence-grounded protocol comparing seven pipeline families on one index and generator with statistical tests and cost. Practical significance: released code, identifiers, and prompts make the protocol reusable. Results: corrective and agentic pipelines ranked evidence best and cited annotated rationales most completely, but no difference from the dense baseline remained significant after correction; the lexical baseline was significantly worse; hybrid fusion and reranking did not help; verdict accuracy varied little with the retriever and was dominated by the generator's decision bias; the agentic pipeline needed two and a half times the latency and three times the calls of the baseline. Conclusions: gains are small, confined to sentence-level evidence ranking, and cost two to three times more calls.