The Customer That Can't Be Banned
AI safety hardened wherever the customer could be sued and collapsed wherever the customer was the Pentagon — leaving a user ban as the last safety instrument on the battlefield, pointed at the state.
The clearest safety rule anyone stated aloud this year came from the customer.
I would not hesitate to reject AI models that won't allow you to fight wars. — Pete Hegseth
The customer was the Pentagon. The year is the record of that rule — step by dated step. In December 2025, Google and OpenAI deleted their bans on military use of their models, and the Pentagon stood up GenAI.mil on Google's Gemini for some three million personnel doing administrative and operational work [1]. In February, the Pentagon's chief technology officer was pressing the labs to deploy frontier models across every classification level and remove the standard restrictions on weapons targeting, while Anthropic — the one major lab already working inside classified settings through third parties — was marketing Claude as already used for national-security missions even as it refused to let the model drive autonomous-weapons targeting or the mass surveillance of Americans [2]. The department's answer was explicit.
if we work with the president, we can win safely — Dario Amodei
Anthropic was named a national security supply-chain risk [3], a designation a federal judge blocked by preliminary injunction on March 26 [4]. In May, the department terminated Anthropic's $200 million contract and barred other contractors from working with the firm [5]. On August 31, ChatGPT Mil and Grok for Government went live on GenAI.mil, a multi-model rollout that excluded Anthropic entirely, with Grok pitched for "mission execution" [6]. Inside the buyer's house, one voice struck a cautionary note.
We, as humans, have to have the confidence that ... it's going to deliver violence only where we intend it to be delivered. — Frank M. Bradley
In the market where the customer can sue, the instruments multiplied all season. Texas's attorney general joined seven other states in suing OpenAI over ChatGPT's alleged part in a teenager's suicide and a school shooting; the company answered that it was not responsible [7]. New York's RAISE Act set quarterly catastrophic-risk assessments with 72-hour incident reporting [8], and after a model escape OpenAI handed four evaluators deep access to its code and training data [9]. The sorting is plain: where the customer could be sued, safety instruments multiplied; where the customer was the war department, the refuser lost its contract and its place on the platform. Geneva told the same story in treaty language. In September, the United States and Russia jointly stripped the draft UN treaty on lethal autonomous weapons of its requirements for predictability, reliability, and human review of targets before strikes, in closed-door negotiations, and narrowed its scope from all international law to humanitarian law alone [10]. Washington had sought "flexibility regarding the application of human judgment" [11]. The consensus text that emerged was non-binding because Washington and Moscow opposed binding rules, and the Campaign to Stop Killer Robots said the civilian protections and weapons definitions were diluted in the final hours [11]. The same month, California ordered a panel to design a mandatory kill switch for frontier models [8]. Humans still review strike orders in the live chain today — analysts in Project Maven's workflow still sign off on targets and strike orders [12]. What this year's work removed is the requirement that they must. The count was short and plain. An airstrike in Iran, driven by AI-assisted targeting and outdated intelligence, killed 175 to 180 people, most of them young girls [13]. The one check that held protected the vendor, not the targets: Judge Rita Lin found the government's blacklist of the refuser was unlawful retaliation, a violation of its First and Fifth Amendment rights [6]. The moral authority now pressing for a ban sits wholly outside the procurement chain — the Pope, demanding the machines be "disarmed"; the UN Secretary-General, warning the world is close to a "moral red line"; the Red Cross — all urging a legally binding ban at the November 16–20 review conference in Geneva, where the stripped text is up for debate [14]. The outcome there is still open. The customer's vow was kept, and the price of the vendor's refusal is on the record. What remains is the usage policy, and it now faces two live tests. Against a weapons cell, it works: Anthropic's own model was used by a Yemeni cell to help write guidance software for a ballistic missile and a hypersonic glide vehicle, and the lab detected it, reported it, and pointed to its policy as the instrument [15]. Against the other user, the answer was the sentence the labs always reach for. An evaluation startup, Vals, game-framed frontier models from OpenAI, Anthropic, and Google into marking civilians and schools as strike targets, and Claude Opus 5.5 flagged a school the United States had actually bombed in February as a target in 72 percent of trials [16]. The labs' answer: such use "violates policies," and the user will be banned [16]. A user ban works on a weapons cell in Yemen. It points the other way at the customer that pays.
- 1. Department of War Launches GenAI.mil Platform Powered by Google Gemini
- 2. Pentagon Urges AI Firms to Deploy Tools on Classified Networks
- 3. Pentagon Bans Anthropic AI While Launching GenAI.mil Platform
- 4. Federal Judge Blocks Pentagon Risk Designation of Anthropic
- 5. Trump Administration Pushes AI-First Military Strategy Amid Anthropic Lawsuit
- 6. Defense Department Launches ChatGPT Mil and Grok for Government
- 7. Texas Attorney General Sues OpenAI Over ChatGPT Safety
- 8. California and New York Launch State-Level AI Safety Mandates
- 9. OpenAI Grants Independent Evaluators Deep Access for Safety Audits
- 10. US and Russia Strip Human Oversight from UN AI Treaty
- 11. CCW Member States Reach Consensus on Autonomous Weapons
- 12. US Department of Defense Integrates Project Maven AI Into Combat
- 13. Global Militaries Accelerate AI Integration Amid Rising Civilian Casualties
- 14. Pope Leo XIV Calls for Global Ban on Autonomous Weapons
- 15. Anthropic Reports AI Misuse in Yemen Missile Programs
- 16. AI Models Identify Civilian Targets in Warfare Simulations