“Was This Written By AI?” Is Usually the Wrong First Question
Substack added an AI-detection scanner this month, built on Pangram.
Substack added an AI-detection scanner this month, built on Pangram. Open an eligible post, choose “Scan for AI text,” and it gives you an estimate of how much was written by a human versus with AI assistance. Writers can add a “How I make this” disclosure, contest a result they think is wrong, or turn detection off for a post. People seem to like having the number. I think it’s usually the wrong number to be asking for — though not for quite as simple a reason as I first thought.
Substack’s CEO has been explicit that Pangram isn’t a quality judgment — it can’t tell you whether human care went into a piece, or whether the piece is any good. It’s positioned narrowly, as information about authorship and reader expectations. That’s a fair and modest claim, and worth taking at face value.
But even taken on those modest terms, a detector answers a much thinner question than the one most readers actually have in mind when they reach for it. “Is this reliable?” and “is this substantially a person’s own expression?” are two different questions, and a detection score is weak evidence for the first and only loosely relevant to the second. The trouble isn’t that Substack is claiming otherwise — it’s that readers reach for the tool as a proxy for both, and it’s built to answer neither cleanly.





























































