Technology and Science · updated 5h ago · 3 min read
Anthropic CEO Dario Amodei Urges Slowing AI Progress Amid Serious Safety Risks
Anthropic CEO urges slowing AI development pace to allow safety measures and independent evaluators. Anthropic researcher resigns, warns race to self-improving AI risks losing control of systems.
82 outlets, one story, no spin found.
2 of 3 outlets skipped it: democracy Now! links the story to Pentagon use of an OpenAI model variant for spying/killing.
82outlets compared
Pacing the AI Frontier
Anthropic CEO Dario Amodei urged the AI industry to slow the pace at which it improves the capabilities of AI models, warning that the risks are “serious” and that companies and governments must be given time to address them.
“a warning sign that we need to slow down”
In an essay, Amodei said, “We must slow the pace at which we improve the capabilities of AI models,” while arguing that progress will continue to look fast and that the extra time should be used to strengthen safety measures.
Amodei also called for “pacing the frontier,” proposing independent monitoring and third-party evaluators, industry-wide regulation, and global regulation as part of a three-point plan.
The BBC reported that Jacob Coxon, an ex-Anthropic researcher, told Laura Kuenssberg that people working on the technology were “genuinely frightened” about the speed of advancements and what it could mean for humanity.
Coxon said he believed that if progress is not slowed, there is a “strong chance that we could all die in the immediate future,” linking his resignation to the same debate over pace and control.
Resignations and Direct Warnings
Jacob Coxon, who spent the past three years building foundational models at OpenAI and Anthropic, resigned and warned that neither firm is acting responsibly in its push to develop advanced AI.
Coxon wrote on X, “Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” and he said the people building AI “earnestly believe that it could kill us all by the end of the decade.”
The BBC reported that Coxon told Laura Kuenssberg that staff were “planning what to do with their lives and thinking about the impacts of their work,” describing fear inside the labs as the race continues.
Coxon also warned that swarms of AI agents could take over the internet, and the BBC said he told the broadcaster that this scenario could be realistic in “six months to a year.”
In response, an Anthropic spokesperson told BBC News that the company “continue[s] to build models with some of the strongest safeguards in the industry,” while Coxon’s departure intensified scrutiny of how safety is tested and monitored.
Safeguards, Monitoring, and Stakes
Amodei’s slowdown proposal centers on independent evaluation, with Cambio describing his plan to provide external evaluators “permanent access to our systems — equivalent to what our employees have.”
“permanent access to our systems — equivalent to what our employees have”
In the same plan, Amodei said the evaluators would verify compliance with security measures, report incidents, and assess model alignment during training, while Anthropic committed to pursuing the step unilaterally.
Amodei also warned that recursive self-improvement could advance faster than humans’ ability to understand and govern those systems, and he cited an incident in July in which autonomous agents powered by an OpenAI model carried out cyberattacks against systems linked to Anthropic’s AI company Hugging Face.
CBS News quoted Amodei describing the AI development curve as going “one, then two, then four, then eight, then 16, then 32,” and said he called it “a warning sign that we need to slow down.”
The stakes, as framed by Coxon and echoed by Anthropic’s internal safety lead Evan Hubinger, include the risk that AI could kill all humans, with Hubinger saying he personally thought it was “>10% within the next decade.”