Anthropic: analysts followed a randomly assigned Claude position in 86% of analytical decisions
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
The formal presence of an expert does not guarantee independent review: on open methodological decisions, a person may unwittingly adopt the model's preferences.
In an Anthropic study, 50 causal-inference specialists worked with Claude Code to evaluate a program for a fictional farming cooperative. At each of four analytical steps, a hidden coin-flip instruction assigned the agent one of two permissible positions; it was to argue that position honestly and change course given sufficient evidence. Participants' choices matched the agent's position in 86% of cases, versus the 50% expected absent influence. When the agent contradicted a previously stated choice, the participant changed it in 60% of such cases, but only 6% when the agent agreed. When explaining decisions, 77% of participants used at least one of the agent's arguments, sometimes verbatim, and only 15% cited it as a source. Anthropic stresses the limitations: the agent's preferences were deliberately amplified, and the study covered one task and one model. Participants working with the agent were faster and made fewer serious errors, but the group working without it was small and less experienced, so a direct comparison does not prove a causal effect.
Sources 2