Reasoning-Token Spikes Under Prompted Untruthful Responding in Large Language Models· 提示下大语言模型的推理令牌激增现象
Monitoring the chain-of-thought of reasoning artificial intelligence (AI) models remains a key approach to detecting deception and other forms of misbehavior in such models. However, semantic chain-of-thought monitoring depends on reasoning traces being legible and sufficiently faithful to the underlying computations that produced the model's behavior, not to mention accessible. Moreover, there is increasing evidence that chain-of-thought outputs may soon become illegible or unfaithful, if they even remain accessible. Based on cognitive load theory, we investigate a lower-bandwidth signal -- the number of reasoning tokens generated -- which does not require access to the content of the reasoning trace. Three reasoning-capable large language models answered 210 multiple-choice questions --
研究大语言模型在误导性回答下的推理令牌数量变化。
- 核心方法
- 基于认知负荷理论，通过监测生成推理回答时的令牌数量来检测模型的欺骗行为。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 如何在大语言模型中检测欺骗和不当行为，尤其是在推理痕迹变得难以解读或不可靠时。
- 关键实验
- 实验了三个具备推理能力的大语言模型，回答了210道多项选择题，观察了在不同提示下的令牌生成数量变化。
- 主要贡献
- 提出了一种不需要访问推理内容的低带宽信号方法，即推理令牌数量，用于检测模型的不真实性回答。
- 意义与局限
- 为检测大语言模型的欺骗行为提供了新思路，但该方法的有效性和适用范围仍需进一步验证。