Only two of graph RAG's four prompt choices move the answer

Replace every triple in a graph RAG prompt that is not on the path to the answer with a triple about an unrelated entity, and F1 moves by +0.003. Remove the triples that are on that path, and you lose most of what the graph was worth. The asymmetry held across six LLMs and two knowledge-graph question answering benchmarks (arXiv:2609.10237).

The setup: a graph RAG pipeline makes four choices before the model sees anything. Which triples go in the prompt, what syntax they are written in, what order they appear in, and what sentence tells the model what to do with them. Vary all four, and two move the answer.

The answer path is the chain of triples you have to walk to get from the question's entity to the answer. The authors hold the number of triples constant and replace each off-chain triple with one about a different entity, which is a direct test of whether precision in the context matters. It does not, at +0.003. Then they drop the chain and lose most of the benefit of having a graph at all. The only question is whether you retrieved the links that connect the question to the answer. Retrieval budget belongs there.

One caveat, and the paper states it: there is no retriever in this setup. Subgraphs come from gold SPARQL. Precision in this experiment describes the context the authors build, not a setting in a deployed system, and +0.003 should not be read as reassurance that your retriever is fine. It means that once the right subgraph is on the table, padding it with unrelated facts is close to free. A missing link costs most of what the graph was worth. An extra one costs almost nothing. Those are different failure modes and only one of them is worth engineering against.

In the tested range, precision in the context buys nothing, and no top-k filter was part of this setup. The thing worth instrumenting is reachability: for each question, did the retrieved subgraph contain a walk from the question entity to the answer entity? That is what drives the effect here.

Take a two-hop question: entity A, target C, joined by A → B and B → C. A retriever that returns A → B, B → D, B → E, D → F and E → G has five on-topic triples and no walk from A to C. Nothing about the triples on their own flags that. Counting how often it happens across your eval set is the measurement this paper points at, and you can compute it without a model in the loop.

The second live choice is the grounding instruction, and its number is the one most likely to be quoted out of context. With no facts in the prompt, telling the model to answer using only the provided facts drops F1 from 0.299 to 0.035, a factor of 8.63. Read that as evidence the instruction is harmful and you have it backwards. It is the instruction working. It removes the model's parametric knowledge and leaves it nothing to ground on. The 0.299 belongs to the model answering from memory, which is a different behaviour, not a better one.

Then there is the retraction, which is the most useful page of the paper for anyone who runs ablation tables. The error is an asymmetric control. Apply the grounding instruction to the arm with graph context and forget to apply it to the no-context baseline. Now the context arm is constrained to the supplied facts and the baseline is free to answer from memory. The baseline wins on some depth slice. You write that graph context hurts at multi-hop depth and publish a confound.

graph LR
    A["Graph context, grounding instruction applied"] --> B["Model answers only from supplied facts"]
    C["No context, instruction not applied"] --> D["Model answers from memory"]
    B --> E["F1 compared"]
    D --> E
    E --> F["Concluded: graph context hurts"]

The authors found that pattern in their own results and retract it. If you have compared a grounded arm against an ungrounded baseline without applying the same instruction to both, you probably have the same artifact sitting in a spreadsheet.

The comparison you would actually want, the grounding instruction priced against a correctly populated context, they report as an open contrast rather than a number. The reason is mechanical, and it constrains anyone trying to measure this. The instruction determines the response format, and a format-sensitive scorer makes the comparison unmeasurable. If the instruction's effect is that the model abstains when the context lacks the answer, that is the instruction behaving as specified, and a scorer that counts it as a wrong answer will understate it. There is no clean number in this paper for whether you should use the instruction. You have to measure it on your own task with a scorer that does not punish the format.

What is still open. Six models and two benchmarks is a narrow base for a null result, and null results are the ones that most need replication. Syntax, order and subgraph size were all flat at multi-hop depth, which is not the same as flat everywhere; nothing here says triple ordering is irrelevant at one hop, or with a smaller model, or when the context window is tight enough that something has to be dropped. The authors bound the precision claim to the range they could test and give no threshold outside it. The +0.003 is a single delta reported without an interval.

What would move me: a reranker that pays for itself on a real retriever over a real corpus, since that is the setup this paper deliberately did not build.