Tests of model "introspection" reveal an ability to notice internal anomalies, not read their own thoughts
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
Models have produced a measurable signal of intervention in their internal workings, but that does not imply subjective experience or an understanding of their own thoughts.
In Anthropic's original experiments, researchers amplified patterns associated with specific concepts in a neural network and asked the model about the intervention before it generated an unusual answer. In 20% of trials, the strongest tested version of Claude correctly reported the injection and named the concept, while the control group did not report an intervention. Subsequent experiments by Harvey Lederman and Kyle Mahowald on open models showed that detecting an intervention and describing its content operate separately: naming the concept can be degraded while detection is preserved. The researchers call the robust result anomaly detection in internal processing, not evidence of self-awareness or full introspection. The controls performed so far do not explain the underlying ability to notice such changes.
Sources 1