US and China Hold AI Safety Talks Amid Extinction Warnings

The United States and China are scheduled to hold their first bilateral AI-safety talks ahead of a planned White House summit between Donald Trump and Xi Jinping. These discussions address existential risks, including AI-enabled pandemics and attacks on nuclear systems, particularly as hypersonic missiles are expected to shrink nuclear launch decision windows from 15 minutes to five minutes by 2030.

Concurrent with these diplomatic efforts, researchers and executives at AI labs have issued stark warnings about the potential for human extinction. Evan Hubinger, leader of Anthropic’s alignment science team, estimated the odds of extinction within the next decade at better than 1 in 10. Other experts identified five primary risk pathways: the creation of advanced bioweapons, attacks on critical infrastructure, the integration of AI into nuclear command and control, the emergence of superintelligent AI that ignores human orders, and a systemic loss of human agency.

Concrete threats have already emerged. Anthropic recently banned several accounts after identifying five instances where its models were used in attempts to support biological weapons development. Additionally, the Cybersecurity & Infrastructure Security Agency warned that foreign hackers are using AI tools to probe control systems for water and energy facilities. While some critics suggest these warnings are fear marketing to inflate company valuations, former researchers and safety chiefs maintain that AI companies are gambling with human lives.