The results.
All sides of them.
No single accuracy score tells the full story. The installed detector does well on familiar data and struggles with other kinds of writing.
| Benchmark / sample size | AI hidden Correct · capture | AI visible Missed | Human hidden False positive | Human visible Correct · kept |
|---|---|---|---|---|
| Earlier regression5,620 AI / 24,416 human | 90.30%5,075 hidden | 9.70%545 visible | 0.97%236 hidden | 99.03%24,180 visible |
| Earlier regression5,620 AI / 24,416 human | 77.70%4,367 hidden | 22.30%1,253 visible | 0.06%15 hidden | 99.94%24,401 visible |
| RAID development2,094 AI / 2,094 human | 57.83%1,211 hidden | 42.17%883 visible | 14.09%295 hidden | 85.91%1,799 visible |
| RAID development2,094 AI / 2,094 human | 34.00%712 hidden | 66.00%1,382 visible | 1.77%37 hidden | 98.23%2,057 visible |
| Harder RAID2,293 AI / 2,293 human | 28.48%653 hidden | 71.52%1,640 visible | 14.09%323 hidden | 85.91%1,970 visible |
| Harder RAID2,293 AI / 2,293 human | 15.48%355 hidden | 84.52%1,938 visible | 2.09%48 hidden | 97.91%2,245 visible |
| MAGE standard27,957 AI / 28,594 human | 19.96%5,581 hidden | 80.04%22,376 visible | 10.15%2,903 hidden | 89.85%25,691 visible |
| MAGE standard27,957 AI / 28,594 human | 5.94%1,660 hidden | 94.06%26,297 visible | 1.13%322 hidden | 98.87%28,272 visible |
| MAGE unfamiliar domains677 AI / 562 human | 39.88%270 hidden | 60.12%407 visible | 9.07%51 hidden | 90.93%511 visible |
| MAGE unfamiliar domains677 AI / 562 human | 20.38%138 hidden | 79.62%539 visible | 1.42%8 hidden | 98.58%554 visible |
| BEEMO instruction writing2,165 AI / 2,165 human | 56.40%1,221 hidden | 43.60%944 visible | 28.36%614 hidden | 71.64%1,551 visible |
| BEEMO instruction writing2,165 AI / 2,165 human | 37.69%816 hidden | 62.31%1,349 visible | 12.06%261 hidden | 87.94%1,904 visible |
Each benchmark’s same texts are replayed under post and reply policies. They are not necessarily actual social posts or replies. Do not add the two policies’ sample counts together.
Download the benchmark counts (JSON) ↓What’s in each test?
The labels come from upstream datasets and generation records. They are evidence of provenance, not a guarantee of human identity or a measure of writing quality.
Earlier regression
A previously used collection of source-labelled human and generated writing. It was observed during development and is a familiar-data comparison, not independent confirmation.
RAID development & harder RAID
Different partitions of a benchmark designed to test detection across generators and writing conditions. The harder challenge exposes weaknesses concealed by familiar data. RAID dataset ↗
MAGE standard & unfamiliar domains
A multi-domain AI-writing benchmark. The unfamiliar-domain slice checks how performance changes when the type of writing shifts. MAGE dataset ↗
BEEMO
An instruction-writing comparison with human and AI answers. It differs from short conversational feeds and has now been inspected in development. BEEMO dataset ↗
What these numbers cannot tell you
They do not establish accuracy on a live X, Reddit, LinkedIn or YouTube feed, prove authorship, or classify whether writing is useful. No model here meets 80% AI capture with under 10% human false positives across all six sections. The shipped detector does not include the experimental 50-word gate.
Make room for the posts
you came here for.
Get Slop Blocker for Chrome ↗Free developer preview. Optional donations. Your call.