ScienceClaw: Benchmarking Continual Self-Evolution of AI-for-Science Agents Across the Natural and Social Sciences· AI 科学代理的自我进化基准
Large language model agents are accelerating scientific automation, yet verified executions rarely become persistent program-level improvements, and existing evaluations do not examine this process across sequential tasks in both the natural and social sciences. We formalize ScienceClaw as fixed-parameter program self-evolution that unifies task solving, scientific verification, and program updates. ScienceClaw-Eval spans 23 disciplines and measures scientific correctness, evolutionary gain, retention, cross-dataset transfer, and evolution cost through sequential streams and independent reset evaluation. Our framework repairs executable workflows through multi-turn interaction, converts re-execution-verified failure--success trajectories into linked Skill and Operator candidates, and retai
AI科学代理的自我进化基准，跨学科测量进化效果。
- 核心方法
- ScienceClaw框架将任务解决、科学验证和程序更新统一为固定参数的程序自我进化过程，通过多轮交互修复可执行工作流，将重新执行验证的失败-成功轨迹转换为链接的技能和操作员候选。
- 适合谁读
- 适合AI研究者、科学自动化工程师和跨学科研究者阅读
- 要解决的问题
- 现有的AI科学代理在解决任务后，其验证执行结果很少转化为程序级别的持久改进，且缺乏跨自然科学和社会科学的任务序列评估。
- 关键实验
- 实验包括跨23个学科的顺序流评估和独立重置评估，未提供具体实验数据。
- 主要贡献
- 提出了ScienceClaw-Eval评测基准，覆盖23个学科，评估科学正确性、进化增益、保留性、跨数据集迁移和进化成本。
- 意义与局限
- 该研究为AI科学代理的自我进化提供了系统性评估方法，有助于推进科学自动化和AI应用的持续改进，但评估的全面性和实际应用效果仍需进一步验证。