When the Detector Is the Judge

NeurIPS, one of the two most selective machine learning venues in the world, desk-rejected 178 position papers this week using a commercial AI-detection tool called Pangram — no human review, no appeal. The decision has triggered a serious technical argument about whether the tool is reliable enough to carry that weight. Here is the mechanism underneath that argument, explained from first principles.

What AI detectors actually measure

Detectors do not read for meaning. They measure statistical patterns in text, specifically two signals: perplexity and burstiness.

Perplexity (the word comes from information theory) measures how surprised a language model is by a piece of text. A model assigns a probability to each word given everything before it. When the sequence is very predictable — the words flow exactly where you would expect — perplexity is low. When the writing is idiosyncratic, unexpected, or creative, perplexity is high.

AI-generated text tends to be low-perplexity because it is literally produced by a model that optimises for probable continuations. That is the whole mechanism: the model picks what comes next based on what usually comes next. A detector that flags low-perplexity text is, in a real sense, asking: "does this read like something a language model would produce?"

Burstiness measures variation in sentence complexity. Human writing tends to alternate — a short sentence, then a longer one, then a terse one. AI-generated text tends toward more uniform sentence length and structure. Detectors treat low burstiness as a secondary signal of machine generation.

Neither signal reads intent. Neither signal reads meaning. Both signals read the statistical shape of the prose.

Why academic writing sits in the danger zone

Here is the problem that makes the NeurIPS situation specifically interesting.

ML position papers are not creative writing. They use a constrained technical vocabulary, formulaic citation structures, and sentence constructions that are disciplined by precision requirements. You do not get to be surprising when you are defining a loss function or characterising a methodological limitation. The prose is constrained by convention.

That constraint compresses perplexity toward the AI-like range. A well-trained ML researcher writing a careful position paper may produce text that looks, statistically, like low-perplexity output — not because a model wrote it, but because the register demands it.

Burstiness is suppressed by the same force. Technical writing does not benefit from punchy short sentences alternating with sprawling complex ones. It benefits from consistent, precise, medium-length sentences. Which is also what AI-generated technical prose looks like.

This is not a philosophical objection. It is a specific, named failure mode in machine learning called distributional shift: a model trained on one distribution of text (general web prose, student essays) is applied to a different distribution (ML position papers), and its calibration does not transfer.

The threshold question

Any binary classifier — flag or pass — has a threshold. Above this score, flag; below, pass. The choice of threshold controls a trade-off between two kinds of error.

Set the threshold low and you flag more papers. You catch more genuine AI-generated text (high recall). But you also flag more human-written papers by mistake (lower precision, more false positives).

Set the threshold high and you flag fewer papers. Precision improves — most things you flag are genuinely problematic. But you miss more true violations (lower recall).

NeurIPS appears to have used a threshold considerably stricter than Pangram's default. Pangram's default sensitivity setting flags 42.7% of submissions. NeurIPS desk-rejected 18.4%. That gap — roughly 24 percentage points — suggests the chairs deliberately tuned toward precision over recall. In plain English: they tried to reduce false positives before pulling the trigger on outright rejections.

There is also a second tier. The 123 papers required to produce version-history evidence by June 15 presumably scored in a middle band — above the human-writing threshold but below the high-confidence rejection threshold. This two-tier structure is sensible classifier design: high-confidence flags get hard rejections, borderline cases get a human checkpoint. The problem is that "sensible design" and "correct calibration on this population" are different claims, and only the second one has been tested.

The validation gap

No public validation study for Pangram on ML position papers exists. This is the load-bearing technical objection in the backlash, and it is worth being precise about what it means.

Pangram is a commercial product. Its training data, the populations it was evaluated on, and its measured false-positive rates on domain-specific corpora are not documented in peer-reviewed literature. That does not mean the tool is wrong. It means no one outside Pangram has measured how often it misclassifies ML position papers specifically.

When you apply a classifier to a population it has never been validated against, you are extrapolating. The extrapolation may be fine. But "may be fine" is not the same as "we measured it and it is."

For most uses of a classifier, this is an acceptable risk — you tune as you learn, you iterate. For desk-rejecting papers at NeurIPS without appeal, the stakes are different. Acceptance at NeurIPS is a primary credential for academic hiring, visa applications, and grant eligibility in ML research. A false positive here is not a spam email landing in the inbox. It is a career consequence for someone who followed the rules.

The no-appeal problem as a systems-design question

All classifiers have a non-zero false-positive rate. This is not an indictment of Pangram specifically; it is a mathematical property of classifiers. At any non-zero false-positive rate, applying the classifier to a population large enough guarantees some genuine human-written papers are flagged. The question is not whether this happens — it does — but what mechanism exists to catch it.

Standard deployment guidance for classifiers in high-stakes settings (loan applications, medical triage, hiring) calls for human-in-the-loop review above some confidence threshold, precisely because the classifier's output is evidence, not ground truth. The classifier narrows the field. A human makes the final call.

NeurIPS's initial structure for the 178 desk-rejections offered no such step. The classifier's output was treated as the decision. That is unusual for high-stakes classification, and it is the procedural question underneath the technical one.

The checkpoint requirement for the 123 middle-tier papers does introduce a human review step, partially. If an author can produce timestamped drafts or version-control logs, a human presumably evaluates them. That is closer to the standard pattern. But it places the burden of proof on the author, not the tool — and the tool's reliability on this population has not been publicly established.

What this scales to

ICML and ICLR have not announced equivalent policies yet. If either venue adopts Pangram at default sensitivity settings, the 42.7% flag rate projects across their full submission pools. ICML receives roughly 10,000 submissions per year. At 42.7%, that is over 4,000 flagged papers. Even if the venues apply a stricter threshold and bring that down to NeurIPS's 18.4%, that is still 1,800 papers.

At that scale, the calibration question stops being an edge case. The false-positive rate on the NeurIPS-domain population, whatever it actually is, gets multiplied by submission volume. Getting that number right — with a real validation study, on real ML papers — matters more than the threshold choice.

What to watch

The June 15 checkpoint deadline will produce some data: how many of the 123 middle-tier papers can produce satisfactory evidence, and what the rejection rate looks like after human review. That is a partial signal on whether the detector's borderline band is well-calibrated or not.

The deeper question — whether Pangram's high-confidence tier has a measurable false-positive rate on ML position papers — will only be answered if the NeurIPS chairs or Pangram publish the analysis, or if researchers with access to both the scores and the original drafts conduct their own study. Neither has happened yet.

What NeurIPS has done, regardless of how this resolves, is set a precedent for AI-enforced authorship policy at the conference circuit's highest level. The mechanism that enforces the policy is, itself, an AI system whose reliability on this specific task has not been publicly documented. That is the recursion at the centre of this story, and it is worth understanding before the next venue makes the same call.

Footnotes

Further reading

• [Docs]: Pangram Labs product documentation (threshold configuration and sensitivity settings) — https://pangram.com/docs
• [Paper]: Sadasivan et al., "Can AI-Generated Text be Reliably Detected?" (2023) — https://arxiv.org/abs/2303.11156 — the foundational study on theoretical limits of AI-text detection, including false-positive rates under paraphrasing.
• [Paper]: Liang et al., "GPT Detectors Are Biased Against Non-Native English Writers" (2023) — https://arxiv.org/abs/2304.02819 — direct evidence on false-positive elevation for non-native writers.
• [Explainer]: Simon Willison, notes on AI-text detection reliability — https://simonwillison.net

¶