SI detectors are changing writers' behavior but continue to make mistakes
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
SI checks already affect the work of writers and editors, so their results must not be turned into accusations without independent human review.
NPR describes how checks for machine-generated text are affecting hiring and editorial work. Writer Sally McGraw says that after a false positive she began avoiding familiar words, constructions and punctuation so her text would not appear generated. Clarkesworld publisher Neil Clarke said submissions grew from about 1,100 a month in 2021 to twice that level by February 2023, while his own workload increased by about 30%. In a June study in the International Journal for Educational Integrity, four detectors were tested on 160 academic texts: human-written, machine-generated, mixed, and machine-generated texts rewritten by a person. Pangram was the most consistent, identifying SI use in 37 of 40 mixed papers and 37 of 40 rewritten papers, but even its developer acknowledges the possibility of errors. University of Pennsylvania professor Chris Callison-Burch notes that detectors assess text predictability and can therefore mistakenly flag publications familiar to the model or ordinary features of human style. The Authors Guild warns that a false accusation can cost an author a contract and reputation, while missed machine-generated text can cost readers' trust.
Sources 1