The AI Risk Washington Won't Write a Law For

Since August, Washington has aimed five instruments at AI risk. A Senate bill requiring developers of the most advanced models to design against catastrophic cyber, biological and nuclear harm, and letting the government block an unsafe release ^[1]. A draft of the same blocking authority, handed to lawmakers on October 16, by Anthropic — the company about to price the largest IPO in American history ^[2]. A Super Intelligence Task Force under the director of national intelligence that is, by its own framing, considering no new laws ^[3]. A proposal to run the industry through a self-regulatory body modeled on the Securities Exchange Act of 1934 ^[4]. Evaluators embedded inside a lab under a White House pledge, with employee-level access ^[5]. Five instruments, and every one of them aims at a single risk: models escaping human control. That is not the only systemic risk the industry now runs, and it is not the one the Federal Reserve named in August. Kansas City Fed president Jeffrey Schmid put the other question on the record.

I would argue that, that there’s some signs that we have to really start to talk about that on a macro level, is this industry becoming another too big to fail? — Jeffrey Schmid

The other risk is the balance sheet. By September 1, the hyperscalers and their suppliers carried more than $4.4 trillion in on- and off-balance-sheet obligations ^[6]. In one week this month, the ecosystem lined up roughly $150 billion in chip-collateralized debt — enough to reprice risk across the $10 trillion U.S. corporate market ^[7]. Broadcom built a special-purpose vehicle that buys its own hardware and leases it to the labs, keeping the debt off the labs' books while the chipmaker partially guarantees a $60 to $70 billion senior tranche ^[8]. Alphabet carries $798 billion in future commitments; Meta, $696 billion, ten times its net income ^[9]. One investment manager now puts the full financing need at $5 to $10 trillion, enough to rival the entire investment-grade market ^[10]. The Federal Reserve's own read is split down the middle. JPMorgan Asset Management's Stephanie Aliaga argues the bond market can absorb the issuance, with room for another $1.5 trillion, and calls the boom sustainable ^[6]. TD Asset Management cautions that the market's appetite may not be infinite ^[6]. New York Fed president John Williams saw enthusiasm rather than a bubble, and said he was not worried about the leverage ^[11]. Against all of that, the federal government's entire money-side file runs three lines. The Pentagon negotiated a $5 billion loan to Fluidstack, an AI cloud startup — its largest ever, under an emergency order about foreign-made electrical equipment — with a few billion more in Energy Department loans for nuclear restarts ^[12]. And a year ago, when OpenAI's finance chief floated a government backstop for the buildout, the answer was recorded flat: no federal bailout for AI. The AI czar gave a reason he clearly considered decisive.

We believe that governments should not pick winners or losers, and that taxpayers should not bail out companies that make bad business decisions or otherwise lose in the market. — Sam Altman

A loan, a few billion more, a no. Nothing that binds. The telling detail is what happened when the financial authorities finally took up systemic risk. The Fed, the OCC and the FDIC began questioning banks this week about emergency shutdown mechanisms, aimed at keeping a rogue AI from taking down a bank ^[13]. Handed the chance to look at the debt, the money-side regulators reached for the model risk again. October's two weeks ran the two stories as a duet. On one side, the catastrophe talk peaked. On the other, the bond market repriced the money. Ray Dalio called the AI bubble close, and Michael Burry said the stock market was in its first stage of grief, denial ^[14]. Nasdaq's Jack Cassel said the bar has changed — investors now want growth paired with cash efficiency and discipline, not growth at any cost ^[15]. The $5 to $10 trillion financing estimate landed the same day Amodei published his manifesto warning that a swarm of AI agents could take the internet within six to twelve months ^[10]. And when someone finally put the debt question to the two CEOs, they answered from the other story.

the rate of new wonders being achieved will be immense — Sam Altman

Amodei's word for the technology he was being asked to account for was different.

The question is whether the applications arrive in time to pay for it — Bain & Company, Inc.

Anthropic's own IPO filing spent nearly a third of its pages on risk factors — among them, models that resist shutdown or commit blackmail ^[16]. Catastrophe as disclosure: the one audience that reliably hears the danger is the one deciding whether to buy. None of this requires the warnings to be insincere. They are not new; labs were racing toward superintelligence while issuing extinction warnings a full year ago ^[17], and people have quit jobs over them — former Anthropic researcher Jacob Coxon told lawmakers humanity is more likely than not to lose control of these AIs ^[18]. The sincerity charge runs the other way too. Yann LeCun has a different word for the extinction talk.

Those agents are doing exactly what they’ve been asked to do. — Yann LeCun

Former employees, too, have said the regulatory push is meant to raise barriers for smaller competitors ^[19]. The divide stands either way. A sincere warning and a cynical one produced the same allocation: a bill for the model, a bond market for the debt. The clearest proof of the asymmetry is where disclosure actually binds. No federal law requires a developer to reveal dangerous model behavior — unless it becomes a data breach, a concrete harm, or something with material impacts on investors. The bond market is the only federal regulator that gets mandatory filings ^[20]. So the industry runs two systemic risks with two adjudicators. One has a committee and a hearing room. The other has a deadline: the $150 billion of stressed Triple-C debt maturing in 2027 and 2028 ^[10]. There is no bill behind that date. When it arrives, the question Schmid asked in August will be answered not by any of the five instruments, but by the price of the paper.