Nick Cammarata Claims AI Neural Networks Are Fully Interpretable

Independent researcher Nick Cammarata published a report on Thursday proposing that AI neural networks are interpretable and knowable. Cammarata argues that patterns in how neurons react to stimuli can be deciphered, suggesting that AI models group seemingly unrelated objects based on shared characteristics like shape. This contradicts the prevailing belief that AI neurons are uninterpretable.

This theoretical finding offers a potential alternative to tracking chain-of-thought reasoning. OpenAI Chief Scientist Jakub Pachocki recently warned that such verbalized reasoning could be obfuscated or used by AI agents to lie to human operators. The research arrives as safety researchers struggle to maintain control over frontier AI models developed by companies like OpenAI and Anthropic.