Watermark Forensics for Generative Models: An Information-Theoretic Perspective· 生成模型的水印法医视角
A watermark in a generative model's output is usually asked only whether a text is machine-made. The same mark can do more: attribute it to the user who produced it, extract a hidden payload, or localize the part that survives editing. These form a forensic ladder, and we ask what each rung costs in the sample length $n$. One object organizes the answers. Let $S$ be the secret the mark carries (a user's identity or payload), and let the information profile $ν(t)=I(S;X_t\mid X_{<t})$ record how much the $t$-th token reveals about $S$ given the earlier ones. Its total mass pays for attribution and extraction; how that mass is spread pays for localization; and detection alone is paid for not by information but by presence, the distance from the marked to the unmarked distribution. The literature's two quality models, a mark subtle on every token and one that stamps a few tokens loudly, are two incomparable ways of capping this profile. Our main theorem settles the ladder's entropy column. For statistically distortion-free schemes, attributing a text to one of $N$ users costs $Θ(\log N/h)$ tokens over every stationary-ergodic source of entropy rate $h$, sharp to a $(1+o(1))$ factor: to our knowledge the first tight entropy-rate law for multi-user attribution (via exact alignment). The natural collision-counting analysis overcharges without bound; only a decoder thresholding each candidate by its own realized surprisal attains the rate while almost never implicating an innocent user. A matching converse makes the law two-sided, and extraction of an $\ell$-bit payload costs $Θ(\ell/h)$. Two gaps are real, not modeling artifacts: a $Θ(\log N)$-token window in which a text is provably machine-made yet unattributable, and a footprint-resolution uncertainty principle. Experiments on GPT-2, Pythia-410M, and Qwen2.5 recover the predicted constants.
生成模型输出水印的法医鉴定方法及其代价分析
- 核心方法
- 本文通过信息理论方法,定义了信息轮廓函数ν(t)来衡量生成模型输出的每个token对于携带的秘密S的泄露程度,并基于此分析了不同应用场景所需的样本长度。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 现有生成模型输出的水印主要用于识别文本是否由机器生成,而本文探索了水印在用户归属、隐藏载荷提取和编辑后定位方面的应用及其代价。
- 关键实验
- 通过在GPT-2、Pythia-410M和Qwen2.5上的实验验证了理论预测的常数。
- 主要贡献
- 提出了生成模型输出水印在多用户归属问题上的首个紧熵率定律,并且为载荷提取和编辑后定位提供了理论基础。
- 意义与局限
- 该研究为生成模型水印的应用提供了新的视角,解决了多用户归属和载荷提取的关键问题,但存在机器生成文本可识别但不可归属的时间窗口以及分辨率不确定性原理的局限。