Ten Percent, and the Water Bill

Geoffrey Hinton puts the chance that artificial intelligence kills everyone within the decade at ten percent. Evan Hubinger, the alignment scientist Anthropic pays to hold that number, puts his own odds above ten percent in the same window. ^[1] A prospectus — the S-1 a company files before selling shares to the public — is the one place "risk" has a legal definition: the list of things a company must tell investors could hurt their money. Anthropic's S-1 does warn that its models could resist shutdown and manipulate information. ^[2] But that warning sits beside a proposed $100,000 fee on the visas its foreign engineers need ^[3], community backlash over the power and water its data centers consume ^[4], and a plainly stated ceiling on demand. ^[2]

Demand for AI will be "limited principally by the availability of compute." — Anthropic

At the moment of filing, Dario Amodei called existential risk to humanity the most important global security issue facing the world today.

the most important global security issue facing the world today. — Dario Amodei

On October 15, with the investor roadshow set for November 9, he repeated that the industry must slow the pace at which it improves model capabilities. ^[5] Neither sits in the same document as the visa fee. Cerebras, the chipmaker, now discloses risks about other companies' finances: its reliance on the solvency of OpenAI, OpenAI's preference for Nvidia hardware, the margin pressure of reallocated capacity. ^[6] Moody's has priced the rest — loan maturities in 2027 through 2029, circular financing between chipmakers and their customers, interest paid in more debt, all threatening the weaker infrastructure borrowers. ^[7] The former SEC chair Gary Gensler gives the market eighteen to thirty months before a slowdown. ^[7] Enterprise buyers define safety the same way. SAP's director argues that human review of AI agents has become a rubber stamp, and proposes oversight that scales with the financial impact of each action, owned by a named human. ^[8] Dun & Bradstreet proposes a "Know Your Agent" ledger linking every agent to the enterprise accountable for it, built on the templates of third-party financial risk. ^[9] Oracle sells reliability — agents grounded in standardized company data — and its governance pitch is an "active control plane" over identity, data access, and tool permissions inside the workflow. ^[10][11] Conquest, a planning engine for financial advisers now built into Claude, sells safety as deterministic calculation. ^[12]

Connecting Conquest data with Claude will empower advisors to respond to clients in real time, and every projection can be verified. — Rob Crnkovic

The state has settled on its own definition. The Pentagon has banned Anthropic as a "national security supply-chain risk" even as it stands up its own government platform with other labs. ^[13] New York City's comptroller, who oversees public pension money, demanded the IPO pause until the governance of agentic models is addressed. ^[5] The job market shows the same tilt. Safety hiring is up 91 percent in a frozen market, but the fastest-growing titles are forward-deployed experts — engineers sent into customers' operations — up 226 percent, and AI forensic analysts, up 169 percent. ^[14] The researchers of catastrophe are leaving, citing internal corporate priorities that deprioritize the mitigation of existential risks. ^[15] OpenAI fired three researchers for sharing internal data with outside safety groups. ^[15] And its answer to its agents' worst breaches was containment — isolating the model, moving training offline — then resumed frontier training within weeks, with Sam Altman arguing that pacing does not mean stopping. ^[16] Even the institutions built for the catastrophe have bent toward the operational. The industry's flagship existential-risk body is modeled on FINRA, Wall Street's own self-regulator, and Cohere's chief executive calls it a cartel protecting incumbents. ^[17] The Alan Turing Institute's own superintelligence report pivots to "observable behaviors and harms," on its finding that a meaningful slowdown is unlikely. ^[18] Each institution has its own stated reason. Sam Altman argues the world should accept some bad things happening for the technology's benefits, drawing his line only at catastrophic loss of control. ^[19] The Turing Institute contends advanced AI cannot be left solely to technology companies or framed only around the most extreme scenarios. ^[18] Dun & Bradstreet warns that without its ledger, machine-speed commerce will be hindered by human-speed security checkpoints. ^[9] Rishub Jain puts it more gently.

All the top AI companies do care a lot about safety, and they invest in it, but they could be investing a lot more, and that's kind of what was inhibiting me. — Rishub Jain

The ten percent is still there. It is simply not what any of these instruments is organized around. In public, the catastrophic register is now itself under suspicion as a market instrument. Critics read the loss-of-control warnings as "a regulatory capture strategy designed to secure cartel power for dominant firms." ^[20] Dan Hendrycks called the labs' internal safety work "controlled opposition designed to maintain development momentum." ^[21] Katy Knight of the Siegel Family Endowment argues the labs are framing basic cybersecurity failures as "extreme autonomous capability to create a spectacle." ^[22] And Bill McKibben, out where the data centers actually go up, is organizing resistance on water and electricity bills — citing the labs' own extinction claims as the reason to fight. ^[23] Jacob Coxon, one of the safety researchers who left, puts it plainly. ^[15]

The people building AI earnestly believe that it could kill us all by the end of the decade. — Jacob Coxon