Anthropic researcher quits with a loud statement: labs are "gambling with our lives" in the race to self-improving SI
Prepared by Sintelligent with the help of SI. Publication time restored from the build log. . Updated: .
Even inside the labs, people see that SI will soon begin improving itself. For the world, this is the main threshold ahead, and the argument now is about how fast to approach it.
Jacob Coxon spent three years working on model pretraining at OpenAI and Anthropic. On leaving, he wrote on X that developers are "racing straight toward self-improving superintelligence" and that many of them seriously believe it could destroy humanity before the end of the decade. Instead of the race, he proposes that the labs agree on a pace. His colleague Evan Hubinger previously put the risk of human extinction from SI over the next ten years at above 10%. Anthropic did not comment on the departure.
Sources 1