CAFE: Self-Improving Search Agents Need Co-Evolving Feedback· 共进化的搜索代理需要协同进化的反馈
Outcome-supervised search agents learn when and how to retrieve evidence, but terminal rewards neither localize intermediate errors nor redirect an ongoing trajectory before those errors compound. Treating corrective feedback as a learned in-trajectory intervention couples the two roles: the agent must decide when to request and use feedback, while the critic must infer useful corrections from outcome-confounded rollouts whose failure patterns shift as the agent improves. We introduce CAFE (Coupled Agent--Feedback Evolution), a framework in which a shared-parameter model alternates between search-agent and critic roles. CAFE initializes feedback-conditioned recovery from trajectories built around the base agent's own failures, then couples online and offline optimization. During online RL,
通过协同进化的代理和反馈机制，提高搜索代理的自学习与纠错能力。
- 核心方法
- CAFE框架通过共享参数模型，在搜索代理和批评者角色之间交替，实现在线和离线优化的结合，利用搜索失败的轨迹生成反馈以促进代理的自改进。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 现有的结果监督搜索代理无法在过程中定位错误或提前调整轨迹，导致错误累积，限制了性能提升。
- 关键实验
- 在多个搜索任务上进行了实验，显示了CAFE框架相比传统方法在效率和准确性上的显著提升。
- 主要贡献
- 提出了一种新的搜索代理自学习框架CAFE，能够有效利用错误轨迹和动态反馈来优化搜索策略和性能。
- 意义与局限
- CAFE框架为搜索代理的自学习和错误纠正提供了一种新的方法，有助于提高搜索任务的表现，但其有效性和泛化能力仍需在更多应用场景中进一步验证。