Five rungs to recursive self-improvement, with the top one empty

A preprint on arXiv, The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement, proposes five stages of autonomy for recursive self-improvement and a diagnostic the authors call the Headroom-Closed Index. It closes by identifying "key challenges to achieving genuine RSI," which reads as an admission that nothing has reached the top. Its own abstract describes the empirical support as "preliminary," and the framing has arrived ahead of the demonstration.

It is still worth reading. Naming the rungs makes the next round of claims checkable, and it gives you a vocabulary for asking where a given system actually sits.

The ladder, and why only the top rung is hard

The abstract's definition is worth quoting exactly, because the second clause is where the difficulty lives:

Recursive self-improvement (RSI) enables AI systems to turn experience and feedback into persistent changes that improve both their capabilities and the process of future improvement.

Improving capabilities is ordinary training. Improving "the process of future improvement" is a different object. Below that final rung the abstract lists four kinds of autonomy, in order: improvement-execution, improvement-strategy, experience-acquisition, environment-adaptation. It does not define them, so the names carry the load. Treat the table as my reading of those names, not the authors' definitions.

The bottom row is the paper. The first four rungs all describe a system getting better at the task while a process you designed decides what "better" means, when to stop, and what to do with the result. At rung five that process is the thing being edited. Which changes what you would have to evaluate. A single weight update is not the object any more; the rule that generates them is.

graph LR
    E["Experience"] --> P["Improvement process"]
    P --> C["Capability"]
    C --> E
    P -->|"rung five"| P

The self-loop is the part with no shipped example in the abstract. The abstract does say the work draws on "diverse industry practices," so the lower arrows have something behind them. It does not say which practices, or that any of them close the loop back onto the improvement process itself.

An index of remaining headroom

The abstract says the Headroom-Closed Index is used to "reveal the problems of existing LLMs," which is a diagnostic job rather than a scoreboard job. Beyond that it gives no definition, no formula and no numbers. Reading the name: a capability benchmark asks whether a system can do a thing. A headroom index would ask how much of the distance between what it does now and what it could do without a structural change has been closed by the change you just made.

Benchmark scores are weak evidence of RSI for exactly that reason. A model can gain ten points on a public benchmark through better data and a longer training run, and the gain tells you nothing about whether it improved its own improvement process. A number aimed at closure of headroom would at least be pointed at the right question. Whether the HCI is that number is not something I can tell from the abstract, and neither can anyone else who only reads the abstract.

The rungs, made concrete

Take a coding agent whose job is to make a failing test suite pass. Here is the same agent at two rungs. This is my construction, not a case from the paper.

Rung one. The agent is told the suite is red and to make it green. It proposes a patch, runs the tests, reads the failures, proposes another patch, repeats. It decides how to carry out each update: which file to touch, how big the diff, whether to retry or revert. It does not decide the objective, the corpus, or the stopping condition. The harness does. The loop is real, the autonomy is real, and every boundary around it was drawn by a person.

Rung two. Same agent, plus it picks the method. It may decide the failure is a missing dependency rather than a logic bug and switch from patching code to adding a package. It may decide the failing test is the wrong test. More autonomy, still inside a frame someone else built: the objective is still green tests, and someone still wrote the suite.

At rung five the agent would edit the thing that produces its updates. Not the patch, not the strategy, but the procedure that generates strategies. To evaluate that you cannot just run the suite before and after. You would have to hold the outer frame fixed and ask whether the inner procedure got better at producing good procedures. That is the question the HCI is apparently meant to answer, and the abstract does not show how it would.

A blunter way to see the gap: if the agent's improvement procedure is a page of prompt text and it rewrites that page after a week, you have a rung-one system with a text editor. Rung five asks whether the rewrite made the rewriting better.

Three scenarios, and what actually separates them

The abstract names scientific discovery, embodied intelligence and software engineering as scenarios with distinct requirements and distinct development speeds. That is the whole of the claim. It does not rank the three, does not say which gets there first, and gives no timeline.

My own read, and it is mine rather than the paper's: the difference is mostly in what a candidate costs to check. A software change can be validated by running a test suite in seconds. A scientific claim needs replication, which is slower than the search that produced the candidate. A physical action has to be taken in the world, and the world does not hand out a million attempts. If that reading is right, the cost of verification sets the speed of the loop, and the three scenarios differ mainly in that number.

If you build agents, you are on rung one or two

Know which rung your system is on. If an agent rewrites its own prompts from failure traces, that is rung one. The rewriting rule is yours, the stopping condition is yours, the definition of failure is yours. Draw that boundary explicitly, because it is where the system plateaus.

Rung five is the one to be sceptical about. The phrase costs nothing to claim and sounds like the end of a curve. When it shows up on a product page, ask which rung, and what specifically changed about the improvement process rather than the output.

The taxonomy is the deliverable. The contribution here is vocabulary and a diagnostic proposal, not a capability. That is a smaller thing than the title suggests, and a more durable one.

What is still open

Nearly everything, and the abstract is candid about it. It closes by identifying "key challenges to achieving genuine RSI," which is a way of saying genuine RSI has not been achieved. The evidence base is industry practice plus preliminary empirical work; that is not a controlled demonstration. The abstract gives no definition for the Headroom-Closed Index, so nobody reading only the abstract can compute it.

The four lower rungs may not be as separable as a ladder implies. A system that chooses its own training data has already taken a bite out of the objective, and at some point the rungs blur. Whether that happens before or after rung five is unresolved, and it matters, because a blurry ladder cannot tell you which rung a claim belongs to.

Watch for the first HCI value attached to a named model with the computation described. That is the point at which this stops being a roadmap and becomes something you can check.