Grading the Narrators: An Isnad-Rijal Framework for Claim-Level Provenance in Multi-Agent Knowledge Systems
Abstract
Modern multi-agent knowledge systems increasingly accumulate knowledge through chains of autonomous transformations rather than direct retrieval. Existing provenance work records what happened - execution traces, tool calls, evidence links - and source-reliability estimation is long established (truth discovery, reputation systems). What is missing is an operational framework that attaches graded, per-domain transmitter reliability to claim-level transmission chains, with completeness semantics, transformation-typed aggregation, decoupled content criticism, and serve/review/quarantine routing. Classical Islamic hadith science confronted a structurally similar problem: deciding whether knowledge transmitted through chains of human narrators should be accepted. Over centuries it developed a rigorous methodology - isnad (a complete transmission chain attached to every claim), rijal (systematic grading of each narrator's integrity and precision), weakest-link chain evaluation, corroboration through independent chains, and matn criticism (content evaluated independently of chain quality). This paper transfers that methodology to AI system design. We contribute a formal mapping from hadith-science concepts to multi-agent pipelines, a relational schema implementing claim chains and a graded narrator registry, a decision matrix combining chain grade with content criticism, and an evaluation on 20,000 claims from real physics textbooks. The evaluation validates weakest-link quarantine and independent-chain corroboration; reports a partial failure of the grade-recovery loop, which missed the highest-fault narrator; and reports two analyses as inconclusive, including a matched-coverage comparison the framework could not reach with the reference content critic. The paper is explicit throughout about which claims the evidence does and does not yet support.
Community
Author here. Multi-agent pipelines fail silently: an unreliable intermediate link degrades the output while the final answer stays fluent and confident. Existing work largely verifies the agent's identity and permissions, not the truth and corroboration of the claim itself. ISNAD adapts classical isnād–rijāl chain-of-transmission verification into claim-level provenance - graded transmitter chains, weakest-link bounding, cross-chain corroboration scoring, and content critique separated from provenance. Code and eval are open, and the paper states plainly which mechanisms are validated and which are not yet.
Read both the paper and the reference implementation, and this is the rare framework transfer that holds up at code level. The mappings that convinced me it is structural rather than decorative: riwāya bi-l-maʿnā held to a stricter standard mapping onto generative versus destructive transformation typing, and madār, the pivot narrator, mapping directly onto correlated model families, a problem the multi-agent literature still has no clean name for.
Two design decisions deserve wider adoption independent of the framework itself: UNVERIFIABLE (tawaqquf) as a first-class critic verdict routed conservatively, and the ordinal-first grading that refuses false numeric precision. And the honest-negative reporting (the missed 18%-fault narrator, the 4.8% coverage ceiling, declining to cite the template-critic eval) is what makes the validated claims credible. More papers should read like their own framework graded them.
Disclosure of interest: I am building an LLM-maintained knowledge base with authority-gated admission and promotion, and the decision matrix plus per-domain narrator grading map closely onto problems I have been solving with cruder tools. I will leave a few technical findings on the repo.
One question: §8.4’s matched-coverage comparison failed because the reference critic returns UNVERIFIABLE on most real prose. The repo already ships an LLM-backed critic and an NLI critic. Have you run the §8 gated pipeline with either? Whether the coverage ceiling lifts, and what served-error looks like when it does, seems like the single most informative next experiment.
Get this paper in your agent:
hf papers read 2607.24117 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper