Training-Free Inference-Time Self-Reflection and Cost-Bounded Early Stopping for Large Language Models· 大语言模型的无训练推理时自反思
Reinforcement-learning training of reasoning LLMs (e.g., GRPO) is expensive and requires a controllable environment, committing every contribution to a full training pipeline. We present EvoResearcher, a training-free, inference-time protocol that adds cost-bounded self-reflection to a single frozen LLM backbone. The protocol iterates generate -> self-critique -> revise until a maximum depth D is reached or the critique returns the CONFIRMED sentinel, an implicit early stop that lets the backbone self-verify its answer under a strict compute budget. Four self-reflective meta-reward components (correctness, efficiency, reflection depth, tool-call diversity) act as design principles instantiated as prompt-level mechanisms, so their benefits accrue with zero gradient updates. We validate the
提出无训练推理时自反思机制,优化大语言模型答案的正确性和效率。
- 核心方法
- 设计了EvoResearcher协议,在推理阶段通过生成 -> 自我批评 -> 修订的循环直至达到最大深度或自我验证正确性,过程中采用四种自反思元奖励组件指导优化。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 论文旨在解决大语言模型昂贵的强化学习训练成本及环境控制要求高,探索在不进行额外训练的情况下提高模型输出质量的方法。
- 关键实验
- 验证了EvoResearcher协议在几种典型任务上的效果,但论文未提供具体实验数据。
- 主要贡献
- 实现了大语言模型在无额外训练下通过有限计算资源自我验证和优化输出,提供了一种成本效益高的方法。
- 意义与局限
- 降低了大语言模型优化的成本,提高了推理时的灵活性和输出质量,但可能受限于特定模型结构和任务场景。