How does Claude's text watermark work?
Anthropic recently shared their intent to watermark text so they can identify AI-generated text. This post is based on my understanding of how it works.
LLMs produce text word by word. At each step, they generate probabilities for the next likely word. Instead of sampling randomly from those words, the watermarking trick changes which words are allowed to be picked.
How to watermark a response?
Step 1: The model produces probabilities for the next word.
Step 2: Normally a random number generator picks one of the good candidates. With watermarking, a keyed function takes a secret key plus the previous few words and decides which candidates are valid to pick from.
Step 3: This repeats for the whole response. Places where there are multiple plausible choices carry the watermark signal.
How to detect a watermarked text?
Step 1: For any candidate word in the text, we check whether it is a valid choice based on the secret key and the few preceding words. If the word is valid, that is counted as a match.
Step 2: Run this across the entire text. Watermarked text matches far more often. The overall match rate can be treated as an AI-generated score.
I’m personally getting quite annoyed by the false negatives from all these AI text detection techniques, especially for technical writing.
What's your thoughts on AI text detection? Do you think AI text detection is useful, or will it create more problems?