Tool · AI content provenance
AI Watermark Detection Rate Calculator
A watermark detector’s published accuracy describes one link in a chain of three. Real-world recall is the product of all three, so the share of AI content a provenance check actually finds runs far below the number on the box.Everything runs in your browser; nothing is uploaded.
provenance recall = coverage × survival × sensitivity
Set the inputs
The result
| Outcome | Pages |
|---|
How to use the calculator
- Set coverage. Estimate what share of the AI text you are checking came from a provider that watermarks its text output at all. ChatGPT does not, so this is usually well below half.
- Set survival. Estimate what share of those marks outlast normal handling: a human editing pass, a translation, a paraphrase, or truncation to a short passage.
- Set detector sensitivity. Enter the detector's published true positive rate on an intact mark. This is the only one of the three numbers vendors normally publish.
- Read provenance recall. The three multiply. The result is the share of genuinely AI-generated content the check will actually flag, which is far below the headline accuracy figure.
- Check both error directions. Read the verdict line at the bottom. It reports how often a hit is correct and how much of what the check cleared is AI-written anyway.
How this is calculated, and where it is soft
The three terms multiply. A detector quoted at 95% sensitivity is describing its behaviour on an intact mark from a model it can read. It says nothing about whether the content came from a marking provider, and nothing about whether the mark survived your editor.
False positives are applied to human text only. In reality, AI text carrying no intact mark can also trip a false positive, which would nudge the flagged count up and make the check look slightly better than it is. The simplification runs against the calculator’s own argument, so the recall figure here is if anything generous.
Coverage and survival are your estimates, not measurements. No published figure exists for either, which is the reason this is a calculator rather than a single number.
Precision and recall answer different questions. Precision tells you whether to trust a hit. Recall tells you whether to trust a miss. Most content workflows check a draft to confirm it passes, which is the recall question, and recall is the weak side of every watermark scheme.
Questions
What is provenance recall?
Provenance recall is the share of genuinely AI-generated content that a watermark or provenance check will actually flag. It is coverage times survival times detector sensitivity. Because all three are fractions, the result is much lower than any single published accuracy figure implies.
Why is a detector's published accuracy misleading on its own?
A published accuracy figure describes the detector's behaviour on an intact watermark from a model it can read. It says nothing about whether the content came from a provider that marks at all, and nothing about whether the mark survived editing. Those two questions are answered before the detector ever runs.
Why does the tool say a watermark check can convict but not clear?
A keyed watermark detector runs a statistical test against a secret key, so its false positive rate is tunable and can be set very low, which makes a hit trustworthy. Recall is the weak side, so a clean result is consistent with most of the AI content going undetected. Content teams usually check a draft to confirm it passes, which is the direction that fails.
Is any of my input sent to a server?
No. The calculator runs entirely in your browser using JavaScript. Nothing you enter leaves your device and nothing is uploaded or stored.
Where do the default values come from?
The defaults describe an agency auditing a client blog: roughly a third of the AI text from a marking provider, half the marks surviving an editing pass, and a 95% sensitive detector. Coverage and survival have no published values, which is exactly why this is a calculator rather than a single number. Change them to match your own situation.
Where does the paraphrase preset come from?
Sadasivan and colleagues (arXiv 2303.11156) measured watermark detection on 300-token passages falling from 99.3% to 9.7%, as true positive rate at a 1% false positive rate, under a recursive paraphrase attack. That tested a green-list watermarking scheme rather than Google's SynthID, so treat the preset as directional.
The full analysis behind this tool, with every source and the image-pipeline test that prompted it, is in Why an AI watermark can convict your content but never clear it.