Perplexity and Burstiness as Detection Signals

Perplexity and burstiness are two separate statistical measurements, and most confusion about how AI detection works starts with treating them as one idea. Perplexity measures something at the level of individual words: how surprised a language model is by each word given everything that came before it. Low perplexity means the model saw each word coming. A sentence like "The cat sat on the mat" produces an average perplexity in roughly the 5 to 15 range, because every word follows predictably from the last. A sentence like "The cat ricocheted off the radiator and landed in my soup" produces something closer to 80 to 200, because "ricocheted," "radiator," and "soup" are statistically unlikely choices given the words around them. The model did not see those words coming, and that surprise is what the number captures.

Burstiness works on a different scale, measuring how much sentence length varies across a document. GPTZero's original definition measured something slightly more specific: the variance in per-sentence perplexity scores across a document, not sentence length directly. The simplified version, measuring the standard deviation of sentence lengths, is what stuck in popular use, largely because it is easy to explain and easy to compute. A four-sentence paragraph with lengths of 13, 12, 11, and 11 words has a standard deviation of roughly 0.83: tight, uniform, flat. A human-typical version of the same paragraph, with lengths of 4, 19, 5, and 36 words, has a standard deviation closer to 14.3. The words themselves might be equally plain in both versions. What differs is the rhythm.

These two signals sit on different axes, and a text can land anywhere on the grid they form. Writing can have low perplexity and high burstiness, or high perplexity and low burstiness, or any other combination. Perplexity is a property of word choice. Burstiness is a property of sentence-level structure across the whole piece. Neither one, on its own, says anything about whether the writing is good, true, or worth reading. They measure pattern, not meaning, and that distinction is what the rest of this piece builds on.

Why LLMs default to low perplexity and low burstiness

Language models producing low-perplexity, low-burstiness text are simply doing what their training optimizes them to do. A model trained to predict the next most likely word, token after token, will generate sequences built from highly probable choices, because that is the entire objective. The output drifts toward safe, conventional phrasing because that phrasing is what gets rewarded during training. Phrases like "in today's rapidly evolving digital landscape" appear across completely different models because that kind of phrase is close to the statistical center of what a next-word predictor converges on.

The same logic explains flat sentence rhythm. A model samples its next token from a context window with a fixed statistical shape, and that process pulls output toward a mean clause length. A model doesn't get distracted, doesn't run out of breath mid-thought, doesn't suddenly clip a sentence short because an idea landed. Human writers vary sentence length for reasons that have nothing to do with probability: fatigue, emphasis, the rhythm of spoken thought carried onto the page. None of that is present in a token sampler. Unedited model output tends to produce paragraphs where every sentence clusters inside a narrow length band, because uniform rhythm is close to the lowest-energy outcome for a system built to predict one likely word after another.

This has a consequence beyond any single document. Research on LLM-based writing tools has pointed to a structural hazard: when many users rely on the same underlying model, their output converges toward a shared style, an effect researchers have called an algorithmic monoculture, producing writing that is, in that research's own phrase, "good but not colorful." The flatness is what happens at scale when a large share of the writing passing through the internet runs through a small number of shared models, each one pulling toward the same statistical center.

How detectors use these signals to produce a verdict

AI detectors return probabilities, not facts. A detector score is a likelihood that a given piece of text matches patterns associated with model output, built by combining several imperfect signals and weighting them against each other. No detector in wide use today relies on perplexity or burstiness by itself. Both get folded into a larger stack: token-level probability mapping, sentence embedding comparisons against known patterns of AI generation, checks for repetition (recycled sentence openers, recurring transition words, similar paragraph shapes), and formatting fingerprints such as heavy bullet use or bolded "key takeaway" phrasing that certain models default to. The single percentage a detector hands back looks precise, but it's an average of several rough signals, rounded to look more confident than the underlying math supports.

The rough zones that detectors in 2026 tend to treat as flags give some sense of scale, even though exact thresholds vary by tool and shift with updates. Average perplexity under roughly 40 tends to read as AI-typical, against a human-typical range closer to 50 and up, sometimes well above 200. A sentence-length standard deviation under roughly 4 tends to raise suspicion, where AI-typical writing usually falls between 0.5 and 3 and human-typical writing runs from 5 to 20. Lexical diversity, measured as a type-token ratio, under roughly 0.45 is a flag. Transition-word density above roughly 5 per 100 words is a flag too. Different detectors weight these ranges differently, and they were trained on different labeled datasets, so two tools can disagree on the same document. That disagreement means the instruments reading the signals aren't calibrated the same way, a gap that explains why some writing gets flagged incorrectly.

Where perplexity fails as a reliable signal

Perplexity gets more credit in detector explanations than its actual reliability earns. Low perplexity is not unique to machine output. It appears constantly in formal academic prose, legal writing, technical documentation, and writing by non-native English speakers, all of which lean toward conventional, predictable phrasing for reasons that have nothing to do with who or what produced them. Perplexity also moves with prompt style: asking a model to write in an unusual register shifts its output's perplexity substantially, so a raw perplexity threshold can move without any real change in who or what wrote the text.

Burstiness holds up better under testing. Across 47 long-form drafts run through GPTZero, Originality, and the 2025 version of Turnitin's detector, perplexity barely separated the drafts that cleared from the ones that flagged, usually by less than 4 points on GPTZero's scale. Burstiness moved every single time. Burstiness also has the advantage of being trivial to check: a single line of Python, np.std([len(s.split()) for s in sentences]), turns it into a measurable quality gate.

Writers can act on this directly, and four habits raise burstiness without wrecking the prose. Let a four-word fragment land right after a thirty-word setup. Break the second sentence in a paragraph short, on purpose. Cut em-dash bridges that smuggle two separate sentences into one clause, and take the period instead. Read each paragraph aloud and clip the sentence that makes the reader run out of breath. None of these are rules in the rigid sense. They are editing moves that put variance back into rhythm, and that variance is the thing detectors actually respond to, which matters more at scale than it does in a single essay.

Why first-generation detectors moved past raw perplexity and burstiness

Perplexity and burstiness were the foundation of the first wave of AI detectors, roughly 2022 through 2023. The field moved past raw versions of both signals because they weren't enough on their own to hold up against better-written or more deliberately shaped text. Three methods came in to supplement them. Curvature-based detection, introduced by DetectGPT at ICML 2023, works from the idea that machine-generated text tends to sit near a local peak in a model's log-probability landscape: perturb the text slightly and its probability drops sharply, while human text, never having sat on that peak to begin with, doesn't behave the same way. Fast-DetectGPT refined the idea using conditional probability curvature, trading some of DetectGPT's computational cost for speed.

A separate method, Binoculars, compares perplexity scores from two different models, an observer and a performer, rather than relying on a single raw perplexity number, and that ratio-based approach outperforms raw perplexity scoring by a wide margin. The shift from first-generation signal comparison to these layered methods tells writers and tool builders something important: if detection now stacks supervised classifiers and curvature analysis on top of basic perplexity and burstiness checks, then evasion strategies that only tweak surface sentence length are running out of road.

What the detection–evasion arms race reveals

The clearest record of this arms race comes from the joint PAN and ELOQUENT Voight-Kampff shared task at CLEF, an adversarial benchmark where systems compete to evade detector panels. The jump between its 2024 and 2025 editions was stark. In 2024, the best system's fool rate against the PAN '25 detector panel sat at 0.049. In 2025, that rate rose roughly 13-fold, to 0.654, driven by two prompting techniques that needed no retraining at all: literal Hindi back-translation combined with an explicit instruction to "be imperfect," and a rotation through nine different languages. Neither technique works by making text read more like something a person wrote. Both work by pushing the text outside the statistical distribution the detectors were trained to recognize.

A team from the University of Technology Sydney, competing at ELOQUENT 2026, named this asymmetry directly: pushing generated text out of a detector's training distribution reliably beats adversarial detection, while trying to pull text into that distribution, by mimicking human writing patterns, fails. Every detector family they tested, including ones fine-tuned specifically against adversarial examples, stayed vulnerable to these structural, out-of-distribution shifts. A separate framework called SHADE, also presented at ELOQUENT 2026, treats detector evasion as a reinforcement learning problem: a policy gets optimized to dodge detection. Full fine-tuning with a small KL regularization penalty pushed the surrogate evasion rate to 98.5%, up from a 1.5% baseline for the unmodified model.

But SHADE's writing itself is the important detail. The outputs that evaded detection most successfully were shorter, simpler, and less lexically varied than the baseline. High evasion rates did not track with more human-sounding prose. A separate line of 2026 research found that Claude Opus 5, operating inside a Claude Code harness, could orchestrate a local 32-billion-parameter OLMo-2 base model, routing up to 90% of output through that base model while losing almost no task accuracy. That result demonstrates a new category of attack against detection systems built on fine-tuned model outputs, and the researchers behind it urged detection providers to start training on base model output specifically. The lesson across all three findings points the same direction: real evasion comes from shifting the underlying distribution of the text, not from polishing its surface. That is a structural fix, not a cosmetic one, and it has nothing to do with whether the resulting writing is any good.

Why these metrics cannot tell quality from detectability

Perplexity and burstiness drive AI content detectors, but neither one measures writing quality. Both measure statistical pattern, and pattern and quality are not the same axis. A text can score as "more human" by a detector's math while being worse writing than the version that scored "more AI." The SHADE results make the point concretely: the RL-trained outputs that evaded detection most reliably were shorter, simpler, and less lexically diverse than the outputs that got flagged. The writing that fooled the detector was measurably worse prose than the writing that didn't.

The same mismatch runs in the other direction, too. Formal academic writing, legal drafting, technical documentation, and the work of skilled non-native English writers all tend to produce low perplexity and low burstiness, the exact profile a detector associates with machine output. Much of that writing is careful, precise, and hard to produce, and a detector will still flag it. Turnitin's own guidance on its AI-writing detection tool states that results "may not always be accurate" and should not serve as the sole basis for any adverse decision. Adding typos or scattering grammatical errors into a draft doesn't fix this either. Surface noise of that kind does not raise perplexity in the way detectors actually penalize it, because it is just noise sitting on top of an otherwise flat pattern.

What writing must have structurally to read as human

Writing that reads as human carries high burstiness as a byproduct of real structural variety, not as an effect bolted on after the fact. The variance has to live inside actual syntactic choices: a long, clause-heavy sentence that winds through two or three ideas before it lands, followed by four words, followed by something of medium length, followed by a fragment. That is burstiness functioning as a property of how the writing is built.

Lexical diversity works alongside it. Models tend to settle on a narrow set of preferred words and phrasings, where human writers range more widely across synonyms and alternate constructions, and that range is what pushes a type-token ratio into the human-typical zone. Transition density matters too: models overuse words like "furthermore," "moreover," "it is worth noting that," "in conclusion," "additionally," and "however." Cutting those close to zero removes one of the clearest formatting fingerprints detectors are trained to catch.

No single model produces this kind of variety on its own, because convergence toward a narrow style is the equilibrium its training settles into, not a quirk tied to any one prompt. The UTS research from ELOQUENT 2026 points to the architectural answer: the strategy that reliably defeats detection is pushing text outside the distribution a detector was trained on, and a single model, however it's prompted, stays inside a recognizable distribution because every output shares the same training lineage. Routing generation across multiple models, especially base models whose statistical fingerprints differ sharply from fine-tuned ones, produces a kind of distributional spread that no single-model setup can match on its own, a result the 2026 orchestration research demonstrated directly.

Without deliberate voice training, any model defaults to generic web English. Structural variety addresses detectability. Voice is the separate problem of making writing sound like it came from somewhere, and from someone, in particular.