A paramedic student at Australian Catholic University had 84 percent of her essay flagged as AI-generated by Turnitin. She wrote it herself. The university's response was to demand her browser search history as proof of innocence. As the student told the ABC: "They're not police. They don't have search warrant to request your search history. But when [you're facing] the cost of having to repeat a unit, you just do what they want."
That same week — some week, every week, pick one — a self-identified 21-year-old stock trader in Nigeria was posting first-person melodramas on X about a nine-year-old boy heroically exposing a courtroom conspiracy. The stories were AI-generated start to finish. X's Creator Revenue Sharing program paid him for it.
One person writes her own work and gets punished by a machine. Another person has a machine write his work and gets paid by a platform. - The School Traitor
One person writes her own work and gets punished by a machine. Another person has a machine write his work and gets paid by a platform. If you designed this system as a parable about institutional rot, an editor would tell you it's too on the nose.
Here's what ties these together: a paper presented at ICML 2026, one of the top AI conferences, titled "Prompt Injection as Role Confusion." Researchers Charles Ye, Jasmine Cui, and MIT associate professor Dylan Hadfield-Menell demonstrated that large language models literally cannot tell who is talking to them. They can't distinguish their own internal reasoning from instructions a user has slipped into the prompt. The researchers spoofed the style of an LLM's chain-of-thought — a kind of internal scratch pad — and the models treated the fake notes as their own already-reached conclusions. As the researchers put it: "The rationale is transparently dumb, but the models don't evaluate it as an external claim to be scrutinized. They treat it as their already-reached conclusion, and simply act on it."
The AI cannot tell who it is.
And universities are trusting it to tell them who their students are.
Guilty Until Proven Literate
Research has found that AI detection tools carry significant false-positive rates for human-written essays, meaning students who write their own work can be flagged as cheaters. Studies suggest these detectors disproportionately misclassify essays by non-native English speakers, whose phrasing strikes the algorithm as "too predictable." International students, ESL students — the ones who already have the most to lose from an academic misconduct charge — are the ones most likely to be wrongly accused.
Turnitin itself tells universities its AI score "should not be used as the sole basis for adverse actions against a student." Vanderbilt University read that disclaimer and stopped using Turnitin entirely. Northwestern and the University of Texas did the same. UCLA declined to adopt it in the first place, citing "concerns and unanswered questions" about false positives. But hundreds of other institutions kept running the tool as if it were a breathalyzer — binary, definitive, inarguable.
Australian Catholic University reported nearly 6,000 cases of alleged cheating in 2024, with about 90 percent related to AI use. They finally stopped using Turnitin's AI detector this past March. After how many students were hauled in, forced to surrender their search histories, made to prove a negative?
Meanwhile, on X, the incentive structure runs the opposite direction. Before X cracked down in February 2026, many monetized creator accounts were flagged for engagement manipulation — posting low-effort AI-generated content specifically designed to farm impressions from verified users. The platform's stated goal for its revenue-sharing program is to encourage "authentic, high-quality content." The system rewarded the opposite for months.
This is the actual mechanism: universities use AI tools to detect AI, and the tools can't do it reliably. Social platforms use AI tools to promote "quality," and the tools reward low-effort content. And the ICML researchers have now shown us why — LLMs judge trustworthiness by style, not source. If text looks like a system prompt, the model treats it as a system prompt. If a student's essay looks statistically "smooth," the detector flags it as machine-made. The machine is pattern-matching all the way down, and nobody in a position of authority seems to care that the patterns don't mean what they think they mean.
The students care. They're the ones sitting in misconduct hearings, handing over their browser history to a deputy vice-chancellor, trying to prove they actually sat down and thought about paramedicine for six hours. They're doing the work the machines were supposed to make easier, and getting punished for it by the machines that were supposed to help.
If your university is still using an AI detector as the basis for academic misconduct charges, it is not protecting academic integrity. It is performing it, badly, at the expense of the students it claims to serve. Stop making students prove they're human to a machine that doesn't know what human means.