ContinualSkillBench: Can LLM Agents Truly Evolve Their Capabilities?· LLM代理能否真正进化其能力
Modern agent frameworks equip large language models with external skill libraries to solve complex tasks. However, it remains unclear whether these systems can effectively evolve their skills and whether the resulting skills improve task-solving capabilities. To bridge this gap, we introduce ContinualSkillBench, a dynamic evaluation framework for in-context continual skill learning. It covers five representative domains, each containing 100 interconnected subtasks ordered by increasing difficulty and opportunities for cross-task skill reuse. Our experiments show that sequential execution generally improves performance, but the gains vary substantially across models and domains. Moreover, in-context learning performs comparably to explicit skill maintenance on average, suggesting that much
评估大语言模型在复杂任务中能否进化其技能的动态框架。
- 核心方法
- 引入ContinualSkillBench，一个涵盖五个代表性领域、每个领域包含100个逐步难度递增的子任务的动态评估框架，用于评估上下文持续技能学习。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 现有大语言模型框架是否能有效进化技能及这些技能是否能提升任务解决能力。
- 关键实验
- 实验包括五个领域100个子任务，展示了不同模型和领域中的性能提升差异。
- 主要贡献
- 提供了系统性的方法评估大语言模型的技能进化能力，并发现顺序执行任务一般能提升性能，上下文学习与显式技能维护在平均表现上相当。
- 意义与局限
- 为理解大语言模型的技能进化机制提供了重要见解，但未涵盖所有可能的技能组合，存在一定的局限性。