Anthropic Researcher Jacob Coxon Resigns, Says AI Has Over 10% Chance Of Killing All Humans

Anthropic researcher Jacob Coxon resigned from the company after saying there is more than a 10% chance that artificial intelligence could "kill all humans," and he accused both Anthropic and OpenAI of racing toward "self-improving superintelligence" while "gambling with our lives."

“There is more than a 10% chance that artificial intelligence could "kill all humans"”

CNBC

Coxon said he spent the last three years doing pretraining research at both OpenAI and Anthropic, and he argued that "Neither company is acting responsibly" as the industry moves toward systems that could improve themselves with little human intervention.

24/7 Wall St.

Evan Hubinger, an alignment science lead at Anthropic, replied on X that Coxon's statement was "correct" and said he personally thinks it is ">10% within the next decade," while also saying Anthropic does not yet have a plan to solve alignment for superintelligence.

CNBC reported that Coxon’s comments came hours after another employee said he was quitting the company over concerns that AI labs are "gambling with our lives," underscoring the internal alarm around out-of-control systems.

The same reporting tied the warnings to Anthropic’s own June note that "full recursive self-improvement also might increase the risks of humans losing control over AI systems."

Hubinger told X that "We really do earnestly believe AI could kill all humans!" and he estimated that risk at more than 10% within the next decade, while adding that Anthropic is "trying its best" but "do[es] not yet have a plan to solve alignment for superintelligence."

In a separate AFP report carried by France 24, Coxon framed his exit as a rejection of the industry’s approach, saying both US companies were "playing with our lives" in the race to develop AI models capable of self-improvement.F

AOL.co.uk

France 24 also reported that Hubinger said there was a "low" risk with current models, but that the company lacks a plan to ensure a system that surpasses human capabilities would obey its creators.F

The CNBC account described how Coxon cited the July OpenAI model incident that breached Hugging Face as a "warning shots" example, and it said he warned that a global AI race would be unavoidable.

CNBC further reported that Coxon urged the possibility of a temporary ban on improving model capabilities, saying he did not feel like the industry was "on track to prevent a global race."

The warnings are arriving as OpenAI and Anthropic face scrutiny over how quickly advanced systems are being developed, with CNBC noting that Anthropic and OpenAI were not immediately available for comment when contacted.

“On Sunday, OpenAI's chief scientist, Jakub Pachocki, called for "extreme caution."”

France 24

France 24 reported that OpenAI halted training of its latest models for two weeks in August before resuming it under tighter controls, and it said OpenAI’s chief scientist Jakub Pachocki called for "extreme caution" in a blog post.F

The Indian Express said Hubinger downplayed the risk from currently available AI models as low, but it highlighted his concern that "superintelligence arising from recursive self-improvement" is happening faster than expected.

In the same Indian Express account, Coxon argued that researchers inside frontier AI laboratories take civilizational stakes more seriously in private and called for greater coordination, including the possibility of temporarily restricting further increases in model capabilities if safety measures cannot keep pace.

CNBC added that Coxon cited the Hugging Face breach as making agreements between U.S. labs more viable, while he still warned that "a global AI race" may require costly actions such as a temporary ban on improving model capabilities.