What AI detectors actually measure

We know exactly why they flag the US Constitution. It is the reason we do not sell you a number.

Run the United States Constitution through a popular AI detector. It will tell you a machine wrote it.

The Bible does the same thing. So do Britannica articles, chapters of well-known novels, and plenty of professional journalism. A Cal State professor ran writing they had done entirely by hand and got back a 98 percent AI probability.

None of it was written by AI. Most of it predates computers. So the detectors are measuring something, and it is worth knowing exactly what, because these scores are being used to judge real people.

We have spent a long time studying this. Here is what we found, including one finding that argues against our own product.

What a detector is actually doing

It does not read your writing. It has no idea what your sentences mean. It runs two measurements and turns them into a percentage.

Predictability. The detector pushes your text through its own language model and asks at every word: would I have guessed this one? When the answer is usually yes, the text scores as machine-written. This is called perplexity. The logic is reasonable, since language models generate text by picking likely next words, so their output is predictable by construction.

Variation. Human writing lurches. A long winding sentence stuffed with detail slams into a short one. AI settles into a groove and stays there, most sentences landing in the same narrow range. Detectors measure how much your sentence lengths and predictability scores move around. Not much movement reads as machine.

Newer tools bolt a trained classifier on top, a model shown millions of human and AI samples until it can sort them. Turnitin built theirs on BERT. Originality.ai uses modified BERT and RoBERTa. GPTZero runs several components at once, one of which just searches the internet to see if your text already exists somewhere.

That is the whole thing. Predictability, variation, pattern-matching.

This is also why Grain shows you what it changed in a rewrite rather than quoting you a score.

Why real writing gets caught

Look at what those measurements reward and the problem shows up immediately.

Predictable, evenly-paced, conventional prose scores as AI. That also describes a lot of perfectly good human writing. Legal writing is conventional on purpose. Encyclopedia entries are formal and even on purpose. Technical documentation is clear and consistent on purpose. Any well-edited article has had its rough edges deliberately sanded off by an editor.

The Constitution is the cleanest example. Edward Tian, who built GPTZero, explained it to Ars Technica himself:

The US Constitution is a text fed repeatedly into the training data of many large language models. As a result, many of these large language models are trained to generate similar text to the Constitution and other frequently used training texts. GPTZero predicts text likely to be generated by large language models, and thus this fascinating phenomenon occurs.

Read that twice. The document gets flagged because language models were trained on it. The detector is not catching a machine. It is catching text that resembles what machines produce, which naturally includes the text machines learned from in the first place.

The part that should worry people

Famous documents are the fun example. Students are the serious one.

Stanford researchers ran seven detectors over 91 TOEFL essays, every one written by a human, all by non-native English speakers. On average the detectors flagged more than half as AI-generated. A 2026 follow-up put the figure at 61.3 percent for essays by Chinese students, against 5.1 percent for American students in the same setup.

Then the researchers did something clever. They took the flagged essays and enriched the vocabulary, making the language more varied and elaborate. The false-positive rate dropped from 61 percent to 12 percent.

That one result explains everything. The detectors were never finding AI. They were finding limited, predictable word choice, which is what writing in a second language often looks like. A smaller English vocabulary produces more predictable prose, and predictable prose is what these tools flag.

What everyone else has concluded

A study in the International Journal for Educational Integrity tested fourteen detection tools, Turnitin included, and found them "neither accurate nor reliable." Every tool scored below 80 percent accuracy. Follow-up work put false-positive rates on human writing between 15 and 26 percent, meaning up to one paper in four.

Institutions moved. Vanderbilt switched Turnitin's AI detector off in 2023, pointing out that even a 1 percent error rate across 75,000 annual submissions means roughly 750 students wrongly flagged. Yale followed. So did Curtin. By 2026 more than a dozen universities had disabled the tools outright and over two dozen restricted their use. MIT Sloan published guidance titled "AI Detectors Don't Work."

In 2025 the Federal Trade Commission went after an AI detection company over its accuracy claims. The company advertised 98 percent. Independent testing measured 53. The FTC's Bureau of Consumer Protection director put it plainly: the product "did no better than a coin toss."

And here is the detail that says the most about the industry. A 2026 paper on academic integrity noted that several companies selling AI detectors also sell AI humanizers. They are selling the lock and the key, to different customers, at the same time.

Using a humanizer is now its own flag

This is the part to read carefully, because it works against what we sell.

Running AI-drafted text through a humanizer, including Grain, can make it more likely to be flagged as AI, not less.

Every competitor in this category advertises the opposite. Here is why they are wrong.

Turnitin added AI bypasser detection in August 2025. Their system now reports a separate category for AI-generated text that was AI-paraphrased. Think about what that does to a student who ran an essay through a rewriting tool before submitting it. The work comes back flagged twice, once for the AI and once for the paraphrasing. The second flag is the one that reads as intent to hide something. A false positive on the first is arguable. The second is much harder to explain away.

The research had already pointed here. A 2025 paper introducing a detector called DAMAGE (arXiv:2501.03437) trained specifically on humanized AI text and identified 97 to 98 percent of it. The same documents walked past GPTZero, which caught 34.5 percent. The researchers also found that mixing humanized examples into the training data at a rate of 0.68 percent was enough to bring detection back to nearly full strength, which means any vendor who decides to catch humanizer output can do it cheaply and quickly.

Two things follow from that gap. Detector quality varies far more than most people assume, so getting a clean result from one tool is close to meaningless as evidence about any other tool. And humanized writing carries a signature of its own, which a detector can learn to spot as soon as somebody trains for it. Somebody has.

Why rewriting cannot fix this

There is a result in the literature that explains the pattern, and it applies to every product in this category rather than to any one of them.

Sadasivan and colleagues showed that detection accuracy has a ceiling, set by how much human and machine writing genuinely overlap. As models get better at sounding like people, the two populations of text converge, and no detector can pull apart what has stopped being distinct. That reads like good news for anyone trying to slip through. Chakraborty and colleagues closed the gap at ICML in 2024 by showing that the ceiling rises again once the detector has more to work with, either a longer sample or more of them. Give it enough text and the difference becomes findable.

The word carrying the weight in both papers is distribution. What a detector responds to is the statistical shape of how a piece of writing came to exist. A rewrite works on the surface of the text, changing vocabulary, sentence boundaries, the rhythm of a paragraph, sometimes the order of the argument. None of that reaches the thing underneath, which is that a language model produced the substance. So paraphrasing buys cover that is temporary at best and fragile while it lasts, and a permanent guarantee against detection is not a thing anyone can sell you.

What this means if you use these tools

An AI score is not evidence of authorship. It is a statistical guess about how predictable your text is. Those are different things, and treating one as the other has consequences for real people.

A high score often just means the writing is clean. Formal, well-edited, conventional prose reads as predictable because it is predictable. Usually that signals care, not a machine.

The tools fail worst on the people least able to argue back. Non-native speakers get flagged at dramatically higher rates. So do students writing in formal registers they worked hard to learn.

And no tool can honestly promise you a score. Detectors update every few months, specifically to catch text written to avoid them. Anything guaranteeing a permanent result against a moving target is selling something it cannot deliver.

Where we stand

We make a tool that rewrites AI-drafted text so it reads like a person wrote it. The obvious thing to advertise would be a bypass rate. Most tools in this category do exactly that, with numbers like 99 percent and words like undetectable.

We do not, for the reason this whole article lays out. That number would be a promise we cannot keep, against systems we do not control, measured by tools that flag Britannica and the Constitution. It would also be worse than useless now that Turnitin reports paraphrasing as its own category, because the reader who trusted the promise is the one who gets caught by it.

So we will say the unhelpful thing plainly. If your goal is to get AI-drafted work past a detector, no tool can honestly promise you that, and running the text through one can leave you worse off than submitting it untouched. We would rather lose the sale than have someone find that out afterwards.

So we work on the writing instead. We keep a documented list of the habits that make text read as machine-written, and we cut them: padded transitions, abstract nouns hiding real verbs, that even metronomic rhythm, specifics blurred into generalities. Every rewrite comes with a receipt showing what changed.

Writing that reads as human because it genuinely is well-written does not expire when a detector updates. That is the part we can stand behind, and it is the part worth building.

If you want to see that in practice, the rewrite demo is free and shows every change it makes. The plans are there too.

Sources: Liang, Yuksekgonul, Mao, Wu & Zou, "GPT detectors are biased against non-native English writers," Patterns, 2023, with 2026 follow-up figures. Weber-Wulff et al., "Testing of detection tools for AI-generated text," International Journal for Educational Integrity, 2023. Jabarian & Imas, "Artificial Writing and Automated Detection," University of Chicago Booth, 2025. Edward Tian quoted in Ars Technica, July 2023. Vanderbilt University Center for Teaching statement, August 2023. FTC v. Workado, 2025. Detector-and-humanizer overlap noted in academic integrity research, arXiv, 2026. Turnitin AI bypasser detection announcement, August 2025. Pangram Labs, "Human-AI Collaborative Essay Detection" (DAMAGE), arXiv:2501.03437, 2025. Sadasivan, Kumar, Balasubramanian, Wang & Feizi, "Can AI-Generated Text be Reliably Detected?", Transactions on Machine Learning Research. Chakraborty, Bedi, Zhu, An, Manocha & Huang, "On the Possibilities of AI-Generated Text Detection," ICML 2024.