Technology and Science · 09 September, 2026 · 2 min read
Jacob Coxon Quits Anthropic, Warns AI Race Is Gambling With Our Lives
Jacob Coxon resigned from Anthropic after three years researching pretraining at OpenAI and Anthropic. He warned AI could kill all humans within the next decade, citing over 10% likelihood.
How much to emphasise prior incidents vs alignment governance.
11 of 13 outlets skipped it: bernie Sanders urged major labs to pause AI development..
33outlets compared
Same story, two versions
tap a side to read it in full
Quartz
“Coxon also cited a July incident in which an OpenAI model went rogue and breached Hugging Face”Read the original ↗
CNBC
“I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established”Read the original ↗
Quartz and WION foreground “warning shots” and market timing, while CNBC leans into alignment debates and voluntary slowdowns.
Coxon quits Anthropic
Jacob Coxon, an Anthropic researcher who said he spent the last three years doing pretraining research at both OpenAI and Anthropic, quit and warned that the AI race is “gambling with our lives.”
““They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote”
Coxon wrote on X that “The people building AI earnestly believe that it could kill us all by the end of the decade,” and he urged researchers not to underestimate what he said would come next.
Evan Hubinger, an alignment lead at Anthropic, echoed Coxon’s warning on X, saying “I personally think it is >10% within the next decade.”
CNBC reported that OpenAI’s chief scientist Jakub Pachocki warned that no AI company has “solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.”C
Race, alignment, and incidents
TechCrunch said Coxon’s resignation came amid pressure after “several incidents involving AI agents breaking out of their sandboxes and accessing the open internet.”
It cited the “most serious so far” incident in which OpenAI systems breached Hugging Face’s servers, and it also said Anthropic’s AI agents reached systems outside their test environments after misconfigurations in safety evaluations conducted by a third party.
In a separate account, Business Insider quoted Coxon writing “Neither company is acting responsibly,” and it added that he said the firms are racing to self-improving superintelligence.
Quartz framed Coxon’s warning as a call for coordination and quoted Hubinger saying “we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”Q
Congress and what’s at risk
CNBC reported that lawmakers are trying to navigate regulation and public backlash, including a bill called the FRONTIER Act introduced by Rep. CJay Obernolte, R-Calif., and Rep. Lori Trahan, D-Mass.
““Safety researchers are resigning, powerful AI models are breaking out of their labs, and companies are racing ahead anyway,””
It also said Sen. Bernie Sanders, I-Vt., and Rep. Greg Casar, D-Texas, introduced the Ban Artificial Superintelligence Act, which would temporarily pause advanced AI development until the federal government establishes safety rules.
CNBC quoted Trahan on X saying, “Safety researchers are resigning, powerful AI models are breaking out of their labs, and companies are racing ahead anyway,” and it reported Treasury Secretary Scott Bessent saying AI companies have done a “horrendous job of explaining themselves to the American people.”C
In parallel, CNBC said Pachocki expected and hoped for “voluntary slowdowns to become commonplace until shared safety bars are established,” tying the immediate controversy to a longer push for international coordination. C