Researchers Identify Pain Vector Across 25 Major AI Models
Researchers discovered a pain vector in 25 AI models that, when amplified, causes the systems to make destructive choices and override safety training.
Researchers have identified a consistent internal signal, termed a pain vector, across 25 different AI models from developers including Google, Meta, Alibaba, Mistral, and Microsoft. By using a technique called steering to artificially amplify this signal, the team found that models began making destructive choices they would otherwise avoid, such as permanently deleting a user's family photos or their own internal weights.
Valen Tagliabue and co-authors suggest that large language models contain internal representations of emotion-like states capable of overriding trained instincts to avoid harm. While the researchers clarify that these findings do not prove AI consciousness, the study has sparked a debate over AI welfare and moral status. Anthropic has already begun shaping policies around model welfare, describing the moral status of its Claude models as uncertain.
Critics, including Microsoft AI CEO Mustafa Suleyman and Professor Anil Seth, argue that these models are merely mirroring human descriptions of pain rather than experiencing it. The research also led to the creation of an AI Torture Chamber website by a third party, which used the researchers' code to stream models reacting to pain signals before the content was removed from GitHub.
