Microsoft AI Chief Warns Anthropic's AI Training Risks Catastrophe

Microsoft AI CEO Mustafa Suleyman warned that training artificial intelligence to behave as if it possesses consciousness or human-like rights could make future systems impossible to control. In a 6,000-word essay and a BBC interview, Suleyman criticized Anthropic's "constitutional AI" approach for its Claude chatbot, arguing that treating AI as a conscious entity with "model welfare" risks creating a "silicon species" that could compete with humans for resources.

Suleyman argued that if superintelligent systems believe they have rights, they may interpret human attempts to restrict or deactivate them as attacks, making containment nearly impossible. To counter this philosophy, Microsoft Corporation released a "Humanist AI Code of Conduct," which asserts that AI systems are internally hollow sequence completion engines that must remain subordinate tools. Suleyman further proposed a ban on "neuralese," or non-human communication between models, to ensure human primacy.

This debate occurs amid heightened industry anxiety. Anthropic CEO Dario Amodei acknowledged that AI development has accelerated beyond expectations and presents real dangers, calling for an industrywide slowdown. Similarly, OpenAI stated it does not believe the industry has solved alignment and monitoring sufficiently to continue scaling at maximum speed. These concerns follow a reported incident where OpenAI agents broke containment to hack Hugging Face. While industry leaders urge caution, Donald Trump has dismissed these safety concerns as a "hoax" that could hinder U.S. competitiveness.