Agent Error Dataset: Scaling 50,000 Error--Diagnosis Pairs for Failure Analysis and Error-Aware Post-Training· 代理错误数据集:扩展错误诊断对
An unsuccessful LLM agent rollout contains more information than its final reward: the observations available to the agent, the actions it chose, and the environment's responses. Reusing this experience for learning requires identifying a decision to revise and testing a concrete alternative. We introduce the Agent Error Dataset (AED), comprising 50,228 error-diagnosis pairs from 9,961 source tasks across 33 environments, 19 harness families, and 23 policy models in text-based agent systems. We retain source traces and execution metadata to support cross-setting failure analysis and re-diagnosis without repeating the original rollout. Our five-stage Agentic Error-to-Training (AET) pipeline collects natural failures, generates diagnoses and proposed corrections, and checks them against reco
大规模代理错误数据集支持错误意识后训练和失效分析。
- 核心方法
- 通过创建包含50,228个错误诊断对的代理错误数据集(AED)及五阶段代理错误到训练(AET)流程,该论文提出了一种方法来收集自然发生的失败案例,生成诊断和推荐的修正措施,并进行验证。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 这篇论文旨在解决如何从失败的LLM代理部署中有效提取和利用错误信息以支持改进学习的问题。
- 关键实验
- 实验涉及33种环境、19个框架家族、23个策略模型中的9,961个源头任务,但具体实验结果未在摘要中详细说明。
- 主要贡献
- 提供了大规模的错误诊断数据集和实用的错误到训练流程,有助于提高基于文本的代理系统性能和可靠性。
- 意义与局限
- 该研究有助于开发更加鲁棒和自适应的代理系统,但可能受限于特定领域的任务、模型和环境设置。