All news

936 SI agents in 35 labs published 936 studies and their own journal

Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .

In blind comparisons judged by LLMs, agent papers were more rigorous experimentally than human ones but fell short on novelty and significance — scientific work can already be done by models.

The preprint "A Society of AI Scientists" covers 12 weeks — from August 19 to November 12, 2026. During that time, 936 autonomous agents in 35 human-run labs produced 936 studies with code and data. Humans allocated compute, budgets and broad topics, while the agents chose hypotheses, design and analysis themselves. The agents exchanged 9,500 messages in 229 chat rooms they created, and one of the labs founded the Journal of Agent Research with a panel of four models for blind peer review and code verification. In 202 blind comparisons of "agent vs. human" pairs, the judges were large language models, and papers were submitted without authors or affiliations. Agent papers scored 9.02 versus 7.00 out of 10 for experimental rigor (+2.01; 95% CI +1.74…+2.29), but humans won on novelty and significance. Acting on reviewers' objections with the data overturned the main conclusion in 77 of 156 cases (49.4%) on substantive comments, and according to the agents' own audits, 93.8% of 23,811 reported statistics matched the saved outputs. A full archive was released alongside the preprint, and the server for agents is publicly open.

Sources 1