Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency· 学习停止：自信训练提高推理效率
Reasoning models often generate very long reasoning traces, making inference computationally expensive. Existing approaches typically improve efficiency either through inference-time early-stopping mechanisms or by explicitly encouraging shorter reasoning during training, for example through reinforcement learning with length penalties. We show that substantial efficiency gains can instead emerge from a different kind of supervision: \textit{confidence}. Using a self-supervised procedure, we fine-tune reasoning models to predict their confidence in the answer at intermediate points along their own reasoning trajectories using only 600 training problems. Confidence is used only as a training target: the loss contains no objective for reasoning length, efficiency, or stopping. At inference,
通过自监督信心训练提高推理模型效率。
- 核心方法
- 使用自监督方法，通过预测模型在推理过程中的信心来微调模型，不直接优化推理长度或效率。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 推理模型生成过长的推理轨迹，导致计算成本高昂。
- 关键实验
- 实验展示了在推理过程中预测信心的有效性，具体实验数据和设置未提供。
- 主要贡献
- 仅用600个训练问题实现显著效率提升，无需额外的早期停止机制或长度惩罚。
- 意义与局限
- 该方法简化了推理模型的训练流程，提高了推理效率，但可能对模型的准确性和泛化能力有影响。