Verifying Legal Claims with GANDR

GANDR is a two-agent architecture designed to fix the tendency of legal language models to hallucinate citations while producing otherwise plausible text. Where previous systems evaluate the validity of an answer as a single block of text, GANDR mandates a strict protocol-anchored commit. The system forces the Drafter agent to adhere to a rigid output format where every generated claim must be immediately followed by a pointer to a specific retrieved passage. This pointer is not just a citation at the end of a paragraph; it is a structural token that maps the claim to an index of the retrieved text. This moves the focus from semantic similarity to structural verification, effectively creating a hard constraint where the model cannot proceed to the next sentence without tying the current assertion to a specific source segment.

Traditional grounded-generation pipelines suffer from the aggregate score problem. A model can provide a correct legal conclusion based on a document that does not actually contain the information, or it can hallucinate a citation that looks correct but points to nothing. Because most evaluation metrics check the final output against a ground-truth summary, a model that 'hallucinates correctly' often earns a high score despite failing the primary requirement of legal work: verifiability. GANDR replaces this by decoupling the drafting and the verification steps, ensuring the system cannot rely on probabilistic guesswork for factual accuracy.

The Mechanism of Agent Interaction

The Drafter agent produces answers using a specific legal-reasoning format that requires every factual assertion to be linked via an anchor to the retrieved text. The Critic agent then acts as an internal auditor. It does not simply grade the answer; it verifies the link between the claim and the retrieved passage using an entailment classifier. If the retrieval is absent or the logic between the claim and the passage is tenuous, the system labels the claim as unsupported and returns it to the Drafter. This mechanism forces the model to treat its context window as a hard constraint rather than a suggestion.

For an example of this workflow, consider a query regarding civil liability in a specific jurisdiction. A baseline system might pull the correct statute but misapply the conditions. In the GANDR pipeline, the Drafter must produce the claim [Liability_Condition_Ref] alongside its explanation. If the Drafter attempts to assert a condition not found in that exact subsection of the statute, the Critic identifies that the claim does not entail the source text linked by that specific pointer and rejects the output. The system prevents hallucinations by refusing to sign off on any output that fails this mapping.

On the 185-item legal benchmark, the 11.3-point increase in strict accuracy is not driven by rewriting or prompt engineering tricks. According to the research, this performance gain is localized to the structural enforcement of the protocol-anchored commit; when this specific rule is reverted within the study, performance drops by 22.7 points.

Limits and Unknowns

The Critic uses four labels to evaluate claims: supported, not-supported, neutral, and irrelevant. These labels are currently advisory rather than absolute directives, which explains why their finer-grained categorization shows only weak agreement with human annotators. For a developer building on this, the primary value lies in the system's ability to act as a binary detector for supported versus unsupported claims, where it maintains a strong F1 score of 0.84. Expanding this to production environments with ambiguous or conflicting case law remains a significant challenge, as the current entailment classifier struggles with the nuance required for high-level legal interpretation where a single statute can be read in multiple valid ways. Whether the system can evolve from these advisory labels to a more authoritative role in complex litigation analysis is the next logical barrier.