ConvMem Turns Long-Context Reading Into a Convolution

ConvMem treats the language model as a convolution kernel and reads a long document as a tree instead of a chain. The paper, on arXiv as 2609.10441, reports better long-context reasoning with no training at all: no fine-tune, no RL run, nothing to fit. The previous generation of this idea needed a training pipeline to work.

ConvMem replaces the sequential memory agent. MemAgent and its relatives extend the effective context by reading a document in segments and folding each one into a fixed-size memory that carries forward: read chunk one, update memory, read chunk two conditioned on that memory, update again. A document split into 200 segments is 200 model calls that cannot overlap, because call k reads the memory written by call k-1. Latency grows linearly with document length and you cannot parallelize your way out of it, since each step depends on the last. Errors compound the same way: anything corrupted in the memory is inherited by every later step, and by the end you cannot tell which summarization level the mistake came from.

The model is the kernel

The kernel in ConvMem is a call to the LLM with your query in the prompt, not a weight matrix. Applied to one text segment, that call returns a summary conditioned on what you asked. Applied to every segment at once, it produces one summary per segment, all in parallel.

Those segment summaries are grouped and summarized again, and again, until one representation remains. Depth is logarithmic in the number of segments rather than linear: with pairwise merges, 256 segments is eight levels, not 256 sequential steps. That is the whole trick, and it is why the paper describes the reasoning path as shortening from a chain into a tree.

Three named components do the work. The abstract lists them and their purposes and stops there, which leaves more to reconstruction than I would like, so I have flagged where I am inferring.

• Configurable strides. Named as one of the components behind "robust evidence capture and propagation," with no arithmetic given. Take the term in the convolution sense and a stride is the distance the kernel moves between applications, and which reading you take decides the coverage. If a stride only widens the merge window, every leaf segment still gets its own kernel call and nothing is lost at the leaf. If it steps over segments, those segments are never summarized, contribute nothing to any merge, and are gone before the final call, with one level of carry as the only chance of pulling them back. The abstract does not say which, and the answer decides whether the coverage failure mode below is a weakness of the design or part of it.
• Skip connections. Named, with CNNs as the stated inspiration and evidence capture and propagation as the stated purpose. In a CNN a skip connection carries a layer's input past the layer and adds it to the output. The equivalent here, and this is my inference rather than the paper's text, would be a merge call that still has access to the inputs its children were built from, so a detail a child dropped is recoverable. Without any carry, the third level up is a summary of a summary of a summary, and whatever the first kernel judged irrelevant to the query is gone permanently. How far back the carry reaches is not in the abstract.
• Multi-kernel convolution. Complex queries get decomposed into what the abstract calls "disentangled semantic channels," each running over the same document. What happens to the channel outputs afterwards is not described. My reading is that they have to be reconciled by a final call conditioned on the original query, otherwise the decomposition produces several partial answers and no answer. Splitting the query is a reasonable bet whatever the recombination looks like: ask one summarizer to track two unrelated questions at once and I would expect it to drop one of them, which is the argument for giving each its own channel.

The call-count row is the one that gets misread. 511 calls against 256 is roughly double, and each merge call carries at least its children's output as input, so the token bill grows faster than the call count alone suggests. The win is depth, not volume: eight merge levels that can overlap against 256 steps that cannot. More tokens, less latency.

The last row is the one people will get wrong. ConvMem does not give the model a longer window. Each call still sees a segment, or a handful of them. What changes is the schedule of calls, not the capacity of any individual one. It is also not retrieval, as far as the described design goes: no index, no embedding, no vector store, nothing searched. The paper does not discuss retrieval at all, so that is inference from the description rather than a claim in it. Either way the bet differs from RAG's. RAG wagers that the relevant chunks can be found cheaply and the rest ignored. This wagers that a query-conditioned summarizer decides relevance well enough on the first pass.

What you would actually get

Nothing here needs weights or a training loop, so the whole thing is orchestration: split, fan out, merge, answer. If you already pay per token, you can build the shape of it against any API.

# schematic, not the paper's API
leaves = [kernel(chunk, query) for chunk in document]  # parallel
while len(leaves) > 1:
    pairs = group(leaves, stride)
    leaves = [merge(kernel(pair, query), pair) for pair in pairs]
answer = kernel(leaves[0], query)

You pay for it in tokens rather than in latency. Under the no-skip reading of strides, every segment gets at least one call, and every merge level adds one call per group, so the bill scales with document length in a way a single long-context call does not. The abstract reports no wall-clock or cost figures, and that is the number I would want before believing the parallelism claim is worth much in production.

The failure mode also moves. A sequential chain hides its errors at the bottom; a tree localizes them. If a merge drops the sentence that mattered, the loss is confined to that branch, and a skip connection gives the level above a second chance at it. What a tree cannot do is recover a fact the first kernel threw away because it looked irrelevant to the query. So the method should be strong exactly where relevance is query-relative, and weak where you need exhaustiveness: find every mention of a name, prove no such clause exists, check that two sections do not contradict each other. Those are coverage problems, and query-conditioned summarization is the wrong tool for coverage.

Where the second hop comes from

RULER-HotpotQA and RULER-2WikiMultiHopQA are chained questions. Take "which city hosts the team that drafted player X." No single chunk contains the answer. Something first has to resolve X to a team, and only then does the chunk naming that team's city become scoreable against the query.

In the tree as described, every kernel call is conditioned on the original query, and nothing carries hop one's answer sideways into a leaf being scored for hop two. A chunk that names the team but never mentions X reads as off-topic against the original question and gets summarized down or dropped at the leaf, before any call knows the team mattered. Nothing re-enters the tree until the final merge, if it re-enters at all.

The only named component that could supply the second hop is the multi-kernel decomposition: run "which team drafted X" as its own channel, and its output is hop one. But the abstract stops at naming the channels and gives no recombination step, so where that answer re-enters the tree is not something I can read off the source. Either ConvMem earns those numbers through the decomposition, or through a mechanism the abstract does not describe.

That makes the friendly-territory framing backwards as written. A parallel tree scored against one static query is the opposite of friendly to a chained hop. Multi-hop is friendly only to the extent a decomposition carries hop one's answer forward, and that is the first figure I would ask for.

What is not settled

Everything quantitative. The abstract reports that ConvMem outperforms training-free baselines on the two benchmarks, but the numbers are not in it, so I cannot tell you whether the margin is two points or twenty. On the trained-model side the claim is narrower than a head-to-head: the paper says ConvMem avoids the risk of overfitting to parametric priors that is often observed in RL-trained models on out-of-distribution tasks. That is a statement about a failure mode, not a measured win against an RL-trained memory policy on its own distribution, and a measured win is what would settle the architectural argument.

I would also want exact recall at depth, documents where the evidence is spread across hundreds of individually unremarkable facts, and a token-cost comparison against sending the whole thing to a long-context model. The abstract names its components and their purposes rather than specifying them: strides, skip connections and multi-kernel convolution are all listed, and "robust evidence capture and propagation" is as much mechanism as it gives for the first two. Treat the shape of the pipeline as the paper's claim, and the merge prompts and the terms of the skip carry as mine.

If parallel calls can beat one long serial call, long context stops being a memory-capacity problem and becomes a scheduling problem. The token arithmetic says the calls are not free; the latency argument says they may still be worth it.