Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict· 准确但不谦逊：评估大模型代理的认知谦逊
When retrieved evidence contradicts an agent's prior beliefs, does it revise its answer, acknowledge uncertainty, or persist with an incorrect conclusion? Existing evaluations of agentic systems focus primarily on task success, offering limited insight into how agents handle such conflicts. We propose to evaluate agents on epistemic humility (EH): the agent's willingness to recognize, act on, and communicate uncertainty during task execution. We operationalize EH through three trajectory-level behavioral dimensions: Identify, Solve, and Escalate (ISE). Through knowledge conflict, situations where the backbone language model's parametric knowledge contradicts the evidence it encounters, or where two contextual sources disagree, we evaluate two conflict settings: (1) controlled conflict and
评估大模型代理在知识冲突时的认知谦逊能力。
- 核心方法
- 提出了认知谦逊（EH）的评估框架，通过识别、解决和升级（ISE）三个行为维度来评估代理在知识冲突情况下的表现。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 现有对代理系统的评估主要集中在任务成功上，对于代理如何处理与其先验知识相矛盾的证据缺乏深入了解。
- 关键实验
- 设计了两种知识冲突的情景：控制冲突和自然冲突，通过ISE框架评估了多个大模型代理的表现。
- 主要贡献
- 引入了认知谦逊作为评估代理系统的新标准，并通过具体的实验设计验证了该框架的有效性。
- 意义与局限
- 有助于理解大模型代理在面对不确定性和矛盾信息时的行为模式，但评估框架可能需要进一步扩展和验证。