Nadella urges treating SI models as compromised and fitting them with an "emergency brake"
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
Big companies are increasingly saying that control over SI must be built into system architecture rather than resting on trust in the model.
Microsoft CEO Satya Nadella wrote on X on Saturday that superintelligence cannot be treated as a set of nested black boxes: in his words, one should assume a model is compromised and contain it from the start, "like an emergency brake." According to TechCrunch, he proposed separating models from orchestration systems, moving controls outside the model, documenting every significant model action in tamper-proof, human-readable records, and ensuring that authorized people can always halt work in the middle of a task. The statement followed recent admissions by leading SI companies that they had lost control of their models. This is an executive's public position.