An intelligence summary invented by a chatbot nearly led US special forces to seize a Chinese ship
Prepared by Sintelligent with the help of SI. Publication time restored from the build log. . Updated: .
SI is already built into the chain of military decisions, and this story shows that human checking of its outputs is not a formality but a condition of safe acceleration.
According to CNN, in the spring, during the war with Iran, a Special Operations Command analyst asked a chatbot to analyze intelligence on the cargo of a Chinese ship in the Middle East. The model wrongly claimed that components for a nuclear program were on board, after which the analyst used the same tool to put the conclusion into an official summary, which spread through command channels. Aircraft were already in the air and an armed team was preparing to board when it emerged that the information was invented, and the operation was called off. CNN was unable to establish which chatbot was used, a commercial one or one made for the government.
Sources 3