Recursive Self-Improvement — the RSI Ladder and the Verification Problem

Synopsis

Two primary papers and several secondary pieces on recursive self-improvement. The first sets out a ladder of what self-improvement can mean, from a model tuning its own prompts up to a model rewriting its own training process. The second reports a system that detected its own development metric had stopped tracking the target it was meant to proxy for, and rewrote its search policy to deliberately lower that proxy.

The design details are the interesting part. The system runs against an immutable reference substrate, with memory-free homogeneous workers and a meta-agent bounded by a constitution it cannot modify. The obvious alternative (specialised agents, each carrying state) did not scale, because compounding from mid-states compounds unobserved variance along with everything else.

Where I land

This one really excites me, more than anything else I have collected. I wonder if I can figure out how to contribute to this endeavour somehow.

The reason it lands for me is the constraint set. The controls that made the system work (immutable substrate, memory-free workers, an unmodifiable constitution) are the same controls a security framework would prescribe for an agent you do not fully trust. Those were arrived at independently and for performance reasons. The safety control and the scaling control turned out to be the same control, which is the most interesting thing I have read this month, and it is consistent with something I already believed: value (and harm) lives in the system around the model, not in the model.

The self-correction result is the part I keep turning over. A system noticing that its own metric had drifted from the thing the metric was for is the antidote to the failure mode I keep finding everywhere else (institutions that stop re-examining their assumptions), performed by a machine.

Recursive Self-Improvement — the RSI Ladder and the Verification Problem

Synopsis

Where I land

Connections

Links