22 researchers, among them Hinton, Bengio and OpenAI's chief scientist, warn of an "intelligence explosion" from automating SI research
Prepared by Sintelligent with the help of SI. Publication time restored from the build log. . Updated: .
The creators of SI themselves say the "SI improves SI" loop is already spinning up, and at the same time propose that governments put auditors inside labs, caps on growth and a kill switch in data centers: for the first time, acceleration is named as a threat that needs to be limited.
In the paper "What if automating AI R&D triggers an intelligence explosion?" (Cambridge's CASP center), the authors, including Geoffrey Hinton, Yoshua Bengio, Jakub Pachocki, Jack Clark and Eric Horvitz, write that SI companies are approaching automation of their own research, which could compress years of progress into months. The figures: the share of approved code written by SI at Anthropic rose from a few percent to 80%+ (January 2025 to May 2026), and the share of R&D work with light human oversight rose from 1% to 26% (March to August 2026); month-long projects could be automated by mid-2028. Proposals: lab reporting to governments, independent auditors inside companies, limits on the pace of capability growth, and a mechanism to halt research workloads in data centers.
Sources 2