ai.hackcv
论文精选 60arXiv

The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agents· 回归税:技能对LLM代理的影响

Adding procedural skills to an LLM agent is typically evaluated by average improvement in task success. However, this metric hides an important cost: skills can also make agents worse. We measure both sides by comparing agents with and without skills across nearly 6,000 runs spanning two office automation benchmarks and three model harness stacks. This allows us to distinguish two outcomes. A regression is a task solved without skills but failed after skills are added. A residual failure is a task that fails both with and without skills. We find that regressions are substantial enough that the best performing skills outperform others primarily by regressing less, not by gaining more. We identify three causes of regression: (i) skill description osmosis, a skill changes an agent's behavior

AI 解读论文

技能的双刃剑效应及影响因素分析

核心方法
通过对比有无技能的代理在两办公室自动化基准测试中几乎6,000次运行的表现,量化技能带来的正面与负面影响
适合谁读
研究者、工程师
要解决的问题
如何全面评估技能对LLM代理性能的影响,包括正面与负面作用
关键实验
在两个办公室自动化基准测试中进行了近6,000次运行,涉及三种模型架构
主要贡献
揭示了技能对LLM代理的负面影响,并确定了回归现象的三个原因
意义与局限
该研究提醒开发人员在设计和添加技能时需注意潜在的负面影响,这对提高LLM代理的实际应用效果具有重要意义
领域:cs.AI作者:Darshan Tank、Baran Nama
相关推荐

本站内容由 LLM 精选聚合,原文版权归 arXiv 所有 · 摘录仅供参考