FREA: A Multi-Source Expert Benchmark for Reaction Feasibility Verification· FREA：多源专家基准测试
As generative models and AI agents propose chemical reactions at a scale beyond expert review, feasibility verifiers decide which proposals enter synthesis planning. But do their decisions agree with chemists across different kinds of candidates? We introduce FREA, a benchmark of 751 reactions labeled by expert chemists under an explicit feasibility criterion, drawn from retrosynthesis model proposals, zero-yield experimental records, edits by large language models (LLMs), and five negative candidate generation methods. Our evaluation finds that no verifier leads across all sources: LLMs given only the criterion are competitive with dedicated verifiers, while forward models perform best on retrosynthesis proposals but reject most feasible edits of recorded reactions at the evaluated operat
FREA 是用于反应可行性验证的多源专家基准测试。
- 核心方法
- 构建了一个包含751个反应的多源专家基准测试FREA，这些反应来自逆合成模型提案、零产率实验记录、大语言模型的编辑以及五种负候选生成方法。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 验证当前反应可行性模型的性能是否与化学家的专业判断一致。
- 关键实验
- 评估了多种模型（包括大语言模型、专用验证器、前向模型）在不同来源反应上的表现，结果显示没有一种模型在所有来源上表现最优。
- 主要贡献
- 提供了一个全面的基准测试，评估了不同类型反应来源上的可行性验证器性能，揭示了不同模型的优势与不足。
- 意义与局限
- 该基准测试有助于改进反应可行性验证模型，提高化学合成规划的效率。局限在于数据量有限，可能无法完全反映所有类型的化学反应。