What AI detectors actually measure

We know exactly why they flag the US Constitution. It is the reason we do not sell you a number.

Run the United States Constitution through a popular AI detector. It will tell you a machine wrote it.

The Bible does the same thing. So do Britannica articles, chapters of well-known novels, and plenty of professional journalism. A Cal State professor ran writing they had done entirely by hand and got back a 98 percent AI probability.

None of it was written by AI. Most of it predates computers. So the detectors are measuring something, and it is worth knowing exactly what, because these scores are being used to judge real people.

We have spent a long time studying this. Here is what we found.

What a detector is actually doing

It does not read your writing. It has no idea what your sentences mean. It runs two measurements and turns them into a percentage.

Predictability. The detector pushes your text through its own language model and asks at every word: would I have guessed this one? When the answer is usually yes, the text scores as machine-written. This is called perplexity. The logic is reasonable, since language models generate text by picking likely next words, so their output is predictable by construction.

Variation. Human writing lurches. A long winding sentence stuffed with detail slams into a short one. AI settles into a groove and stays there, most sentences landing in the same narrow range. Detectors measure how much your sentence lengths and predictability scores move around. Not much movement reads as machine.

Newer tools bolt a trained classifier on top, a model shown millions of human and AI samples until it can sort them. Turnitin built theirs on BERT. Originality.ai uses modified BERT and RoBERTa. GPTZero runs several components at once, one of which just searches the internet to see if your text already exists somewhere.

That is the whole thing. Predictability, variation, pattern-matching.

Why real writing gets caught

Look at what those measurements reward and the problem shows up immediately.

Predictable, evenly-paced, conventional prose scores as AI. That also describes a lot of perfectly good human writing. Legal writing is conventional on purpose. Encyclopedia entries are formal and even on purpose. Technical documentation is clear and consistent on purpose. Any well-edited article has had its rough edges deliberately sanded off by an editor.

The Constitution is the cleanest example. Edward Tian, who built GPTZero, explained it to Ars Technica himself:

The US Constitution is a text fed repeatedly into the training data of many large language models. As a result, many of these large language models are trained to generate similar text to the Constitution and other frequently used training texts. GPTZero predicts text likely to be generated by large language models, and thus this fascinating phenomenon occurs.

Read that twice. The document gets flagged because language models were trained on it. The detector is not catching a machine. It is catching text that resembles what machines produce, which naturally includes the text machines learned from in the first place.

The part that should worry people

Famous documents are the fun example. Students are the serious one.

Stanford researchers ran seven detectors over 91 TOEFL essays, every one written by a human, all by non-native English speakers. On average the detectors flagged more than half as AI-generated. A 2026 follow-up put the figure at 61.3 percent for essays by Chinese students, against 5.1 percent for American students in the same setup.

Then the researchers did something clever. They took the flagged essays and enriched the vocabulary, making the language more varied and elaborate. The false-positive rate dropped from 61 percent to 12 percent.

That one result explains everything. The detectors were never finding AI. They were finding limited, predictable word choice, which is what writing in a second language often looks like. A smaller English vocabulary produces more predictable prose, and predictable prose is what these tools flag.

What everyone else has concluded

A study in the International Journal for Educational Integrity tested fourteen detection tools, Turnitin included, and found them "neither accurate nor reliable." Every tool scored below 80 percent accuracy. Follow-up work put false-positive rates on human writing between 15 and 26 percent, meaning up to one paper in four.

Institutions moved. Vanderbilt switched Turnitin's AI detector off in 2023, pointing out that even a 1 percent error rate across 75,000 annual submissions means roughly 750 students wrongly flagged. Yale followed. So did Curtin. By 2026 more than a dozen universities had disabled the tools outright and over two dozen restricted their use. MIT Sloan published guidance titled "AI Detectors Don't Work."

In 2025 the Federal Trade Commission went after an AI detection company over its accuracy claims. The company advertised 98 percent. Independent testing measured 53. The FTC's Bureau of Consumer Protection director put it plainly: the product "did no better than a coin toss."

And here is the detail that says the most about the industry. A 2026 paper on academic integrity noted that several companies selling AI detectors also sell AI humanizers. They are selling the lock and the key, to different customers, at the same time.

What this means if you use these tools

An AI score is not evidence of authorship. It is a statistical guess about how predictable your text is. Those are different things, and treating one as the other has consequences for real people.

A high score often just means the writing is clean. Formal, well-edited, conventional prose reads as predictable because it is predictable. Usually that signals care, not a machine.

The tools fail worst on the people least able to argue back. Non-native speakers get flagged at dramatically higher rates. So do students writing in formal registers they worked hard to learn.

And no tool can honestly promise you a score. Detectors update every few months, specifically to catch text written to avoid them. Anything guaranteeing a permanent result against a moving target is selling something it cannot deliver.

Where we stand

We make a tool that rewrites AI-drafted text so it reads like a person wrote it. The obvious thing to advertise would be a bypass rate. Most tools in this category do exactly that, with numbers like 99 percent and words like undetectable.

We do not, for the reason this whole article lays out. That number would be a promise we cannot keep, against systems we do not control, measured by tools that flag Britannica and the Constitution.

So we work on the writing instead. We keep a documented list of the habits that make text read as machine-written, and we cut them: padded transitions, abstract nouns hiding real verbs, that even metronomic rhythm, specifics blurred into generalities. Every rewrite comes with a receipt showing what changed.

Writing that reads as human because it genuinely is well-written does not expire when a detector updates. That is the part we can stand behind, and it is the part worth building.

Sources: Liang, Yuksekgonul, Mao, Wu & Zou, "GPT detectors are biased against non-native English writers," Patterns, 2023, with 2026 follow-up figures. Weber-Wulff et al., "Testing of detection tools for AI-generated text," International Journal for Educational Integrity, 2023. Jabarian & Imas, "Artificial Writing and Automated Detection," University of Chicago Booth, 2025. Edward Tian quoted in Ars Technica, July 2023. Vanderbilt University Center for Teaching statement, August 2023. FTC v. Workado, 2025. Detector-and-humanizer overlap noted in academic integrity research, arXiv, 2026.