FinRank: An Evidence-Grounded Benchmark for Financial Question Answering and Retrieval over SEC Filings· FinRank：针对 SEC 文件的金融问答和检索基准
Financial question answering is typically evaluated by answer correctness, yet in SEC filings a plausible and even numerically correct answer can be grounded in the wrong evidence. Similar facts and disclosures recur across sections of a filing, across reporting periods of the same firm, and across comparable firms. FinRank targets this provenance-sensitive retrieval problem by requiring systems to identify evidence for the intended entity, reporting period, and disclosure context. The benchmark contains 1185 manually authored question-answer records over the 10-K and 10-Q filings of 22 companies. Each record includes a reference answer, gold supporting passages, and hand-curated hard negatives drawn from confusable passages within filings, across reporting periods, and across comparable f
针对 SEC 文件的金融问答基准，强调正确证据的重要性。
- 核心方法
- 构建了一个包含 1185 个手动编写的问题-答案记录的基准数据集，每个记录包含参考答案、支持性段落和精心筛选的负例。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 现有的金融问答系统仅关注答案的正确性，而忽视了证据的来源和正确性。
- 关键实验
- 未提供
- 主要贡献
- 提出了 FinRank 基准，强调证据来源的准确性，有助于评估和改进金融问答系统。
- 意义与局限
- FinRank 可以帮助研究者和工程师开发更准确的金融问答系统，但在实际应用中仍需考虑不同公司和报告期间的复杂性。