GPT-6 Astra improves Qwen3.5-0.8B math performance by editing 0.016% of its weights
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
A language model can find and repair individual internal defects in a neural network without retraining, but finding such a patch still takes substantial time and does not add capabilities that were absent.
MIT CSAIL researchers showed how a general-purpose language model reads and edits the weights of other neural networks without gradient-based training. In the main experiment, Astra received Qwen3.5-0.8B's weights, practice problems, and 50 evaluation runs. It linked repetitive reasoning to a direction in layers 15 and 19 and removed it from 131,072 weights. MATH-500 rose from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval instruction following from 21.0% to 62.3%; the share of answers hitting the token limit fell from 93% to 33.1%. This repaired a specific failure rather than adding new mathematical knowledge: no established improvement was achieved on Gemma 3 270M, which did not share the same tendency to repeat. According to VentureBeat, the strongest attempt involved 158 conversational turns, 80 Python runs, and 20.3 hours.
Sources 2