The last post I published about this detector came from a machine. Detector knew. It said machine-made, and it was right. Then I spent one evening attacking that post, thirteen iterations, and drove the score down to 0.313. This post is the one-shot version. No iterations. Whatever byline says about it now is the final answer, and I am publishing it either way.
Generic detectors flagged eight of my posts as machine-generated at 0.95 confidence. All eight were mine, by hand. That was LLM-DetectAIve. It caught one genuine machine post and burned twenty three false positives to get there.
Binoculars did the opposite a week before. Zero separation. My posts and the machine posts sat in one interleaved band of scores.
Both failures had the same cause. Generic detectors classify style, not authorship, and my style does not exist in their training data. Terse. Technical. Full of sentence fragments. So I built my own, on my own ground truth.
The dataset was already public
Everything pre-2026 is mine. Everything in 2026 is machine output, prompted by me, some of it de-slopped to strip the tells.
Couldn’t see his face coz he was driving and I am sitting behind. He was happy, he told me he was happy.
That is from dad-was-happy. It sits in my bucket next to the marathon story and the hostel nights. The engineering notes sit in the other bucket. Forty four usable posts. Twenty eight mine, sixteen machine. Dates did the labeling. I did nothing. Used to think labeling was the hard part.
Why the generic ones failed
Perplexity is average model surprise, computed from per-token logprobs. Machine prose is low perplexity, low burstiness. Mine is messier. That is the theory.
Binoculars normalizes perplexity with cross-perplexity from a second model. Defaults are two Falcon 7B models, 28GB of RAM. My M2 Pro has 16GB. I swapped in Qwen 1.5B, capped CPU at 40 percent, and the signal vanished. OpenRouter was a dead end before that. Logprobs only for generated tokens, never for input.
LLM-DetectAIve has a trained head. Wrong training distribution. Academic essays. My debugging notes look like machine output to it, because in its world terse clean prose IS machine output.
The feature that did the work
Two tiers. Stylometric first. Sentence lengths, lexical diversity, punctuation, sentence-start diversity. That alone hit 0.886 LOO.
Second tier is the one doing the work. Instead of asking if prose looks machine-made in general, ask if it looks like my machine posts or my own posts specifically. TF-IDF, cosine to each class mean, five-nearest-neighbors vote. My k8s incident notes vote with each other. So do the newsletters. kNN fraction separates very cleanly on its own. My posts carry 0.83 same-class neighbors. Machine posts carry 0.075. The key decision does all that. LOO went 0.886 to 0.909, then 0.932 after pruning. AUROC 0.964.
Whole thing runs on 4 threads of CPU, no GPU, no cloud. Fast enough to scan the site in seconds.
Two features died
Perplexity from an on-box Qwen produced flipped signal. My writing is terse and predictable. Machine writing is expansive and more surprising. Theory backwards, because the theory assumes humans write the flowery prose. Two detectors failed. Why would a third be different?
NCD, compression distance, was worse. That stuff looked strong in isolation. 2018 marathon post exposed it. Raw oral-style prose. Short sentences. Repetitive. It compresses like machine text because smooth repetitive machine text compresses the same way. NCD measures compressibility, not authorship. Dropped it. The marathon post flipped. LOO went 0.909 to 0.932.
Full scan after pruning: 44 for 44! Machine posts at 0.85 to 1.00. Mine at 0.11 or below. A 0.74 margin between the classes. What does a 0.74 margin buy you? No ambiguity zone. Nothing sits in between.
The red team
Real test. Can a machine trying to sound like me get past it?
I read two pre-2026 posts, wrote two fakes in my voice. Human, said the detector. Twice! 0.018 and 0.039.
Let the service name be ‘banana’. Let the pods be of the service ‘pineapple’.
That opening is from my k8s-dns-fail notes. My fake tech post imitated its rhythm almost exactly. It still landed next to the original, 0.005, human.
Then I split my writing into two voices, because it genuinely is two. Tech posts are staccato, twenty-word sentences, incident narratives. Musings are flowing, thirty-two-word sentences, hedged, personal. One fake per style. Fake tech landed next to four real tech posts. Fake musing landed next to five real musings. 0.015.
Nearest neighbors gave it away. Similarity tier finds prose close to my corpus, close enough that the detector cannot tell if the words were mine or made to be close to mine. That boundary is the whole story.
The attack
Here is the part the first post never reached. The detector caught that post, the one about the detector, at 0.913. So I attacked it. Thirteen iterations in one session. Every round used the detector’s own explain output against it.
The moves that worked. Quote my own posts verbatim, and the kNN vote flipped from zero same-class neighbors to three of five. Kill repeated sentence openers, mine opened twenty sentences in a row with the same word. Add questions and exclamations, my posts run nine questions per thousand words and the machine drafts ran zero. Graft my vocabulary, k8s, cpu, stuff, well, very, don’t, key, word. Swap topic words that live in the machine centroid, the word itself appears in zero of my posts.
Weirdest discovery: words that appear in fewer than two corpus posts are invisible to the detector. ‘detector’, ‘LOO’, ‘kNN’ contribute nothing. Only corpus vocabulary counts.
Score went 0.942 to 0.313. The attacked post read more like me than the version I wrote first. That is the uncomfortable part. The attack made it better.
The skill
I packaged the playbook as a skill. byline-beater, the adversarial twin of remove-ai-smell. Quotes first, openers second, punctuation third, vocabulary fourth. Iterate against the scorer, never guess.
The loop is closed on this blog now. Detector. Attack skill. Post about both. And the detector still cannot separate the attack from my own writing. Which means the only thing it ever measured was how close the prose sits to my corpus.
What it all means
Reading prose alone has a hard ceiling, and I don’t think any amount of features moves it. Public writing plus one frontier LLM beats every prose-only detector. Property of the problem, not a bug. Well, mostly. Classifier learns the distribution. Imitator samples it.
Detectors are quality control, not forensics. Mine checks my own drafts before publishing. For a determined imitator you need evidence outside the prose. History. Drafts. Watermarks. The text itself is always forgeable.
This post was drafted by an LLM, in my voice, on my instructions. One shot, no iterations, no skill files loaded. Whatever score byline gives it, I am publishing it as is.
Postscript: the score was 0.479
byline read the finished post before publishing. Human-written, it said, at 0.479. Twenty one thousandths of a point away from the line. The threshold is 0.5. One shot, and I cleared it by the width of a rounding error.
Then I wrote this postscript, which changes the words, which changes the score. The number above was true for about ten minutes. Everything about this detector is a snapshot.
I keep coming back to the same thing. The detector measures proximity to my corpus, and a machine with my corpus can sit arbitrarily close to it. The score is not about who wrote the words. It is about how the words sit next to the other words. That is why 0.479 feels right. It is not confidence in me. It is a measurement of how close the imitation got.
The post promised the score, so here it is. 0.479. Human, barely, and I still cannot tell the difference without the tool. That is the whole journey in one number. Why would I run the playbook on this postscript? That would be cheating at my own game. Publishing it raw instead.