Open vs. Closed Recursion in Self-Improving Systems

· Zi Wang · 4 min read

Last week, I had a 3+ hour chat w/ some GDM friends about recursion, specifically, open vs. close loops.

Loops are well known in math, physics, computer science ... all have them, and anyone who has looked at a fern has seen a shape that contains itself. What's new is that the loop now writes the next version of the loop. AI builds AI (https://www.anthropic.com/institute/recursive-self-improvement)

A closed recursion eats its own output. It has a goal, it optimizes toward the goal, and the only thing it hears is itself. Nothing enters from outside. Run it long enough and you get the family resemblance problem—every generation looks more like the last, and the defects compound. Incestuous is the honest word.

An open recursion takes perturbation. Outside signal enters at every turn, so the loop never closes cleanly. It tightens, slackens, shifts frequency, drifts off its own center and comes back different. It isn't stable. That's the point. Stability is what a dead loop looks like.

Most AI systems are closed: a goal, a scoring function, and a system talking to itself about how it's doing. We call that self-improvement. But a loop that can't be interrupted by the world isn't improving—it's just getting more confident.

Which leaves the question I couldn't answer. How much outside noise does a recursion need before it is an open loop?

/Notes on Recursion: 6-level taxonomy of recursion in AI systems (outputs, memory/state, evaluation/rewards, policy/tools, abstraction/ontology, world/self-modification). The hierarchy is intuitive and the escalating scope (from DPO to unknown unknowns).

  • [forcing_question] Level 1 (output recursion / DPO): DPO uses preference pairs to shift a policy — the output doesn't feed back into training in real-time, it's an offline comparison. Is the claim that outputs ARE used recursively right now, or that they COULD be? If the latter, what's the mechanism you have in mind?

  • [forcing_question] Level 2 (memory/state): 'recursive memory that feeds on self' — is this episodic memory (storing past outputs and retrieving them), weight update (learning from prior activations), or something else? The distinction matters: current transformers don't have the latter at inference time.

  • [forcing_question] Level 3 (evaluation/rewards): You say there are 'positive attributes' of recursive evaluation — what are the failure modes? RLHF reward hacking, Goodhart's Law, and reward model collapse are well-documented. Acknowledging them would sharpen the claim rather than weaken it.

  • [forcing_question] Level 4 (policy/harness/tool-calling): Why is this 'even better' than recursive evaluation? Is the argument that tool use grounds the recursion in external reality, breaking the closed-loop problem of pure self-reference? Spell out the mechanism.

  • [forcing_question] Level 6 (world/self-modification): 'Unknown unknowns' and 'self-reinforce the mechanism itself' — is this describing meta-learning (learning how to learn), constitutional AI self-critique, or something more radical like runtime weight modification? These are very different claims with very different evidence bases.

  • [annotation] The escalating 'even bigger / even better' framing is doing rhetorical work without argumentative support. Each level needs a concrete example and a reason why depth = improvement, not just scope.

  • [annotation] Level 5 (abstraction/symbolic ontology/policy discovery) is the haziest. It's listed between tool-calling and world-modification but its relationship to both is unclear. What does recursion at the abstraction level mean in practice — concept drift correction, ontology updating, something else?

  • [assumption] Assuming the intended audience is technically literate (ML practitioners or researchers) — if this is for a general audience, every level needs a one-sentence concrete example before any claim will land.

  • [assumption] Assuming 'recursion' here means the output/state of a process becomes input to the same process — not the computer-science sense of a function calling itself. Worth making this definition explicit, since the CS meaning is the first thing readers will reach for.

  • [fix] Anchor each level with: (a) a real system that exhibits it today, (b) what the recursive loop actually is, and (c) the failure mode if the loop runs unchecked. This converts the taxonomy from a list of vibes into a falsifiable framework.

  • [fix] The ordering implies a value hierarchy (deeper = better). Either defend that hierarchy explicitly — why is self-modification of the mechanism superior to, say, recursive evaluation? — or reframe as orthogonal dimensions rather than a stack.