hsu_steve@hsu_steve3d ago

RSI section from the AI documentary Machine God The next threshold is Recursive Self-Improvement: the moment when AI can improve itself without human assistance. For decades this sounded like science fiction. Intelligence explosion scenarios imagined a system rewriting its own code, becoming smarter, then using that new intelligence to make still better versions of itself. But the idea looks less remote now that AI contributes directly to frontier science. In mathematics, recent systems have moved beyond solving contest problems to producing serious new arguments on long-standing open problems. AI is used to build physics world models and propose candidate theories or computational methods. These are early signs that machine cognition is entering the creative loop of science itself. The crucial transition comes when that loop turns inward. AI research is, after all, a technical discipline made of code, mathematics, models of information flow. These are exactly the domains in which frontier models are improving fastest. A model that can solve hard mathematical problems, write production-quality code, design experiments, read the literature, and evaluate benchmark results is already participating in the work of building its successor. At first this will look prosaic. AI systems will write kernel optimizations, improve training infrastructure, discover better data filters, tune reinforcement-learning pipelines, design new benchmarks, and suggest architectural modifications. Human researchers will remain in the loop, approving changes and interpreting results. But the important point is that the search process accelerates. The model becomes not just the product of the lab, but part of the lab’s research machinery. The system being optimized helps optimize the next system. This is the core RSI feedback loop: better models make AI research faster; faster AI research produces still better models; those models, in turn, become better researchers. The danger is that once this loop becomes sufficiently autonomous, it may stop resembling ordinary technological progress. Human institutions are slow because humans are slow: we read papers, attend meetings, debug code, sleep, argue, and wait for funding cycles. Machines do not have to operate on that timescale. An AI research collective can run continuously across millions of processors. This is the runaway possibility. Not that an AI instantly wakes up and recursively rewrites itself into a god, but that the entire AI ecosystem becomes an autocatalytic process. Capital buys compute; compute trains models; models improve models; better models attract more capital. At some point the dominant input into AI progress may no longer be human insight, but machine-generated insight, machine-written code, and machine-run experiments. Then the Butler-Land analogy becomes sharper. Humanity is no longer merely building machines. We are building machines that help build better machines. Once intelligence itself becomes part of the production function, the old categories — tool, worker, inventor, firm, market — begin to blur. The question is whether recursive self-improvement remains a managed industrial process, or whether it becomes the first technological process in history whose natural endpoint lies beyond human comprehension.