Study of 800 models: SI text from the internet first helps training, then hurts it
Prepared by Sintelligent with the help of SI. Publication time restored from the build log. . Updated: .
The more SI writes to the web, the more valuable fresh human text becomes for training new models.
In the paper "How Much Is an AI Token Worth?" 800 models were trained with different shares of SI text; according to the Pangram classifier, 27.5% of web tokens in June 2026 were SI text, rising to 31.1% by August. For small, data-starved models SI tokens help, but the benefit diminishes and then turns into harm; for large models with enough human text, the loss grows almost immediately. The authors released the WildAI corpus of 83 billion tokens.
Sources 2