Direct Pass@$k$ Optimization for Thought Pruning: Balancing Quality and Diversity in Test-Time Reasoning
Test-time scaling has emerged as a key inference strategy for reasoning models, where a model generates a large number of candidate solutions in the hope of finding at least one correct solution. Running all candidate solutions to completion can be computationally demanding, so recent work prunes the candidate pool after generating only a short prefix, committing further computation only to the surviving candidates. Existing pruning strategies divide broadly into two categories: Quality-based methods assign a score to each partial solution and discard those predicted to ultimately lead to an incorrect final solution, while diversity-based methods select partial solutions that follow different reasoning trajectories and are therefore likely to cover different final solutions. The tension be
通过直接优化 pass@$k$ 来平衡测试时推理的质量和多样性。
- 核心方法
- 提出了一种直接优化 pass@$k$ 的方法，结合质量打分和多样性选择，通过仅保留最有可能产生正确解的候选路径来实现有效的剪枝。
- 适合谁读
- 适合研究者和工程师阅读，特别是关注推理效率和资源优化的人群。
- 要解决的问题
- 如何在测试时推理中有效地平衡生成候选解的质量和多样性，以提高准确率同时减少计算资源的消耗。
- 关键实验
- 在多个推理任务的数据集上进行了实验，展示了该方法在减少计算量的同时保持甚至提升模型性能的效果。
- 主要贡献
- 引入了直接优化 pass@$k$ 的新策略，平衡了推理过程中的质量与多样性，并显著提高了推理效率。
- 意义与局限
- 该方法为测试时推理提供了更有效的剪枝策略，有助于在资源有限的环境中部署复杂的推理模型。但可能会因过早剪枝而漏掉一些潜在的正确解。