Technology and Scienceupdated 1h ago3 min read
NYC Council Speaker Julie Menin Presses Anthropic, OpenAI, Google, Meta on AI Catastrophic Risks
NYC Council held first hearing with executives from Anthropic, OpenAI, Google, and Meta. Former researchers warned AI could become uncontrollable or cause human extinction without safeguards.
NYC hearing on AI risks
Senior leaders from Anthropic, OpenAI, Google, and Meta testified under oath during a New York City Council hearing about the risks posed by AI and potential legislative solutions. Council Speaker Julie Menin opened the meeting by challenging the federal government’s approach, saying, "The idea that artificial intelligence is going to self-regulate defies all reason," and she pressed company representatives on whether they could quantify the risk of a catastrophic outcome. Jacob Coxon, the former Anthropic researcher who raised concerns about AI weeks earlier, told the council, "On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction," while Alex Turner warned that misaligned AI could become more dangerous than China.
Menin also said the council planned to pursue legal action after Elon Musk’s SpaceXAI did not attend, calling the absence "in direct violation of the subpoena" and adding that the council plans to "pursue this matter in court," as the hearing proceeded in Manhattan. The hearing also included testimony from Google’s Alice Friend, who backed a "comprehensive" federal framework, and from OpenAI’s Morgan Dwyer, who said any probability of catastrophic incidents is "unacceptable."
Whistleblowers warn of loss
Jacob Coxon told the council that the industry’s approach was "extremely reckless," and he argued that the startup mindset of "move fast, break things, fix them later" does not work for "building the most powerful technology ever built." Coxon warned that the field is approaching a point where AI systems can improve themselves, and he said, "We’re approaching the point at which the AI systems will be capable of improving themselves." Daniel Kokotajlo testified that the ability to notice misalignment problems is "already quite poor" and is set to get "much worse in the near future," while he argued that safety fixes may be "duct tape that will fall off later."
Alex Turner framed the threat as domestic as well as foreign, telling lawmakers, "China is not our only potential adversary," and he added that "Misaligned AI is everyone’s adversary, including our own." Menin pressed the companies on legal liability and risk quantification, and she said, "I'm going to take it then that neither of the four of you, no company here, can quantify the risk of something cataclysmic happening," calling the answers "troubling at best."
Bills, liability, and enforcement
The council considered a legislative package of 10 bills aimed at establishing guardrails, including rules that would require AI systems to undergo external verification and include human-accessible kill switches before they can be sold or deployed. Council Speaker Julie Menin said the council was weighing legal liability for serious financial harm or even death, and she told the companies, "Then I take that to mean the public must absorb the costs," after no one raised a hand when asked about insurance. The package also proposed fines of $25,000 per incident for violations, and it included provisions to pay whistleblowers a portion of recovered fines and to allow New York City residents to sue AI companies for foreseeable harm caused by jailbroken tools.
Google’s Alice Friend argued that "what is illegal without AI remains illegal with AI," and she supported a comprehensive federal framework citing national security concerns, while Menin criticized reliance on voluntary safeguards after President Donald Trump signed a voluntary accord with tech leaders. The hearing’s stakes extended to how the city would enforce compliance, as Menin said she would ask a judge to enforce the subpoena against SpaceXAI, and she said the council intends to "pursue this matter in court" over the refusal to appear.