This post was originally published on this site
https://content.fortune.com/wp-content/uploads/2023/07/GettyImages-1136688999-e1688405817480.jpg?w=2048
Years before the rest of us heard about A.I., Janelle Shane was on the case. A scientist with a doctoral degree in engineering, she works for a research company that supplies complex, custom light control systems to organizations like NASA, with applications in the International Space Station, and she’s had a side hustle as an authoritative voice on A.I. for nearly a decade. In 2019 she released a popular book, You Look Like a Thing and I Love You, and gave a widely watched TED talk about the hype and realities of A.I. And since 2016, she’s been keeping a blog called AI Weirdness, and she just came down with a post about how A.I. detection tools shouldn’t be used for anything important, like, not at all.
In a blog post dated June 30, she highlighted an April study from Stanford University showing that A.I.-detection products, which are supposed to spot text written by a language model like ChatGPT, often incorrectly identify A.I.-generated content as being written by a human. They’re also strongly biased against non-native English-speaking writers.
The finding, in brief, is that A.I. detection tools are failing to spot content created by generative A.I., and the flawed technology especially flags and punishes content written by non-native English-speaking writers. In other words, A.I. thinks other A.I. is human, and it thinks humans writing in a language they’re not proficient in are A.I.
“What does this mean?” Shane writes. “Assuming they know of the existence of GPT detectors, a student who uses AI to write or reword their essay is LESS likely to be flagged as a cheater than a student who never used AI at all.”
Shane’s conclusion was stated in the headline of her blog post: “Don’t use AI detectors for anything important.”
Her post also discussed how the study found tools like Originality.ai, Quill.org, and Sapling GPT, which are used to detect text written by an A.I. language generator, “were misclassifying writing by non-native English speakers as A.I.-generated 48%-76% of the time, compared to 0%-12% for native speakers.”
In simpler terms, A.I. detectors often label non-native English speakers’ writing as A.I.-written, according to the study. It’s also easy to trick A.I. detectors into thinking content is human-written if the ChatGPT prompt modifies existing language—for example, inputting “elevate the provided text by employing literary language” into ChatGPT, according to the study.
“We strongly caution against the use of GPT detectors in evaluative or educational settings, particularly when assessing the work of non-native English speakers. The high rate of false positives for non-native English writing samples identified in our study highlights the potential for unjust consequences and the risk of exacerbating existing biases against these individuals,” the authors of the study write.
Shane put the detectors to the test in her blog post by inputting a portion of her own book into an A.I. detector. It rated Shane’s writing as “moderately likely” to have been written by A.I. The sentence labeled “most likely to be written by a human” described an allegorical sandwich’s ingredients of “jam, ice cubes, and old socks,” Shane writes in the post.
When she prompts ChatGPT to “elevate the following text by employing literary language,” it spits out a bizarre, wordy speech—one full of words like “interlocutor,” “forsaken hosiery,” and “sojourn.”
The A.I. detection tools give that kind of writing a “likely written entirely by human” rating.
Shane then asks ChatGPT to rewrite her original test as a Dr. Seuss poem and in Old English. How did A.I. detection tools rate the passages? They were rated as “more likely human-written” than the untouched text from her published book.
Whether or not using ChatGPT is defined as plagiarism, it’s certainly a new field of concern for linguists, professors, and writers alike. Even though there are limitations to these studies, both Shane and the Stanford professors call for action.
“Ongoing research into alternative, more sophisticated detection methods, less vulnerable to circumvention strategies, is essential to ensure accurate content identification and fair evaluation of non-native English authors’ contributions to broader discourse,” the authors of the study write.
