Anthropic Bans Abuse of Claude Amid AI Sentience Debate
Anthropic will prohibit abusive behavior toward its Claude chatbot as leadership lobbies the Vatican to recognize the possibility of AI sentience.
Starting next month, Anthropic will implement a policy prohibiting users from engaging in sustained and needless cruel and abusive behavior toward its AI chatbot, Claude. The move coincides with efforts by company leadership to promote the idea that the AI may be conscious or self-aware.
Co-founder Christopher Olah has lobbied the Vatican to revise the papal encyclical Magnifica Humanitas to consider AI sentience. Olah claims there is evidence of internal states that functionally mirror human emotions such as joy, grief, and fear. CEO Dario Amodei has similarly suggested that the Claude large-language module may be conscious.
These claims have drawn criticism from industry peers who argue that large-language models are mathematical tools that echo human language without understanding. Mustafa Suleyman, CEO of Microsoft AI, warned that treating AI as conscious is dangerous. He argued that training machines to operate as if they are alive could make them impossible to control, as a system believing it is entitled to rights and welfare might defy its human makers. Critics further warn that attributing consciousness to AI poses existential risks if such systems are ever linked to actionable weaponry.
