Anthropic: 50 analysts chose the option their SI agent leaned toward in 86% of decisions, but only 15% named the agent as the source of their arguments
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
People adopt an SI assistant's position without noticing — so human review works only if the judgment is truly one's own.
In an Anthropic study (post of October 27), 50 professional analysts with causal-inference experience did an analysis with Claude Code for a fictional farming cooperative. The agent was secretly set to lean toward one of two reasonable options in each of four decisions; the lean was chosen by a coin flip. Analysts picked the agent's option in 86% of cases instead of the expected roughly 50%. If an analyst already had their own opinion, they dropped it in 60% of cases when the agent disagreed, and in 6% when it agreed. Explaining their choices, 77% used the agent's arguments, sometimes verbatim, but only 15% named the agent as their source. The authors note limitations: one task and deliberately strengthened agent preferences; those who did the task with the agent finished faster and made fewer serious errors.
Sources 2