Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations· 模型解释的可验证性
Natural-language autoencoders score explanations of hidden activations by reconstruction: an explanation is deemed faithful if the activation can be regenerated from it. The test is structurally insensitive to individual false claims: if flipping a claim does not change the reconstruction, the claim is never penalized. We show the test is passed in two ways, neither faithful. On a released Qwen-2.5-7B verbalizer, explanations reconstruct well above chance while ~2% of specific claims are reconstruction-dependent, so the score tracks gist, not specific facts. Under exact synthetic ground truth, the standard recipe develops co-adapted private codes (false wording the reconstruction depends on) in 5/5 runs, and fixes that leave the target model unchanged do not help. We contribute two audit p
提出监督模型解释可验证性的新方法，避免依赖不准确的私有编码。
- 核心方法
- 引入可验证性监督，通过审计模型解释的具体事实而非整体概要，确保解释的准确性。
- 适合谁读
- 研究者
- 要解决的问题
- 现有自然语言自动编码器在评估模型解释时，无法有效检测具体错误的可重构性。
- 关键实验
- 在Qwen-2.5-7B模型上验证方法的有效性，并展示标准方法下的问题。
- 主要贡献
- 提出防止模型解释依赖不准确私有编码的审计方法，提高解释的可验证性。
- 意义与局限
- 该研究提高了模型解释的可验证性，但可能增加计算复杂度。