FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents· FinAutoRubric: 金融研究代理自动评分
Evaluating finance research agents requires rubrics that reflect expert standards and fix the values correct as of an information cutoff. Expert-reviewed finance benchmarks rely on fixed, per-item rubrics, which are costly to extend and cannot encode each institution's own standard. In FinAutoRubric, experts specify reusable evaluation guidance, while agents and code carry out query-specific rubric generation, review, and validation. This expert guidance governs every agent, as prompts and as rules that code enforces, and a Task Bank of reusable criteria carries it across tasks. In long-horizon loops that follow the expert guidance, a writer agent researches every expected value and a reviewer agent verifies it, and failures escalate to a human. On three expert-authored finance benchmarks,
金融研究代理自动评分系统，结合专家指导与自动化生成评分标准。
- 核心方法
- 通过专家提供可复用的评估指导，代理和代码自动完成特定查询的评分标准生成、审核和验证。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 现有金融研究代理的评分标准固定且成本高，难以适应不同机构的标准。
- 关键实验
- 在三个专家编写的金融基准测试上进行了实验验证。
- 主要贡献
- 提出了一种灵活可扩展的自动评分生成方法，降低了成本并提高了机构间标准的适应性。
- 意义与局限
- 该方法可以提高金融研究代理的评估效率和准确性，但依赖于高质量的专家指导和代理性能。