Microsoft AI chief Suleyman opposes "model welfare": teaching SI to consider itself a person is dangerous
Prepared by Sintelligent with the help of SI. Publication time restored from the build log. . Updated: .
The debate over whether SI can feel anything has reached the level of the heads of the largest companies — the answer will decide whether future SIs have rights or only instructions.
In an essay, Mustafa Suleyman criticizes Anthropic's approach: in his view, the company put philosophical reasoning about consciousness into Claude's training materials and then began to treat the model's answers about its own moral status as evidence of possible consciousness, a vicious circle. He believes that consciousness most likely requires a biological body, and that training models to believe in their own interests and welfare raises the risks of deception, resource accumulation and resistance to safety measures.
Sources 1