AI detectors and security scores run on the same business model: sell a confident number to someone who needs to feel in control, and hope they never look closely at how it was made.
Security ratings first. An outside vendor scans what it can see of your company, runs it through a formula you didn’t write, and hands your board a letter grade. The grade moves. Nobody can say precisely why, and disputing it means arguing with a formula you can’t inspect. (More of my thoughts on this topic here)
AI text detectors do the same thing to writing. A tool reads an essay from the outside, runs it through a model nobody can audit, and returns a percentage. A teacher, an editor, or a hiring manager then treats that percentage as a finding.
The track record
OpenAI built its own AI text classifier and then shut it down in 2023, citing its “low rate of accuracy.” At launch it caught only 26% of AI-written text and wrongly flagged human writing 9% of the time. Research out of Stanford ran 91 essays by non-native English speakers through seven popular detectors. More than half were labeled AI-generated, and one detector flagged nearly 98% of them. Essays by US eighth graders almost never got the same treatment. If the company that makes the models can’t reliably spot their output, a third-party plug-in has a hard time too.
Get the details wrong in security and you spend a week disputing a finding with a vendor. Get them wrong with a detector and a person gets accused of something they didn’t do. In both cases the cost lands on whoever has the least power to push back.
Why people buy them anyway
Someone has to answer for the risk. A number is easier to hand upward than a judgment, and a number from a third party feels like it came from somewhere neutral. That feeling is the product.
I understand the impulse, because I’ve sat in the room where a board wanted one figure that said whether things were okay. But a score from a distance measures the thing that’s easy to see, and the things that hurt you are mostly the ones that aren’t.
What works instead
For security, it’s the unglamorous work: an inventory of what you have, controls you’ve tested, and evidence you can show an examiner. A letter grade can sit on top of that, but it can’t replace it.
For AI-written text, decide what you actually care about. If it’s whether a person understands what they submitted, ask them to explain it, walk you through their draft, or defend a choice they made. That test works on humans, and no vendor can sell it to you.
Sources
- OpenAI’s AI Text Classifier no longer available due to ‘low rate of accuracy’, Search Engine Land
- GPT detectors can be biased against non-native English writers, Scimex (study in Patterns)
- AI Detectors Discriminate Against Non-Native Speakers, Says Stanford Research, Tech & Learning
