LG-Bench: A Graph-Structured Evaluation Benchmark for Life Sciences· LG-Bench：生命科学图结构评估基准
Traditional evaluation benchmarks reduce inherently interconnected scientific knowledge in life sciences into flat lists of questions, disregarding the underlying topological structure of the knowledge. We introduce, the first graph-structured benchmark for life sciences, featuring over 10,000 high-quality multiple-choice questions across medicine, biology, and chemistry. Our approach constructs a weighted evaluation graph using bidirectional matching and semantic similarity algorithms, where nodes represent questions and edge weights capture their semantic relationships. Leveraging this graph topology, we design two novel evaluation metrics. The Global Coherence Score (GCS) measures a model's consistency within semantically related neighborhoods, while Knowledge Balance Score (KBS) analyz
构建了首个生命科学图结构评估基准，涵盖超过10,000个高质量的多选题。
- 核心方法
- 通过双向匹配和语义相似算法构建加权评估图，使用GCS和KBS两种新评价指标衡量模型在语义相关区域的一致性和知识平衡。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 传统评估基准忽略了生命科学知识内在的互连性和拓扑结构，无法全面评估模型能力。
- 关键实验
- 未提供
- 主要贡献
- 提出LG-Bench，作为首个生命科学图结构评估基准，能够更全面地评估AI模型。
- 意义与局限
- 该基准有助于推动生命科学领域AI模型的开发与评估，但可能需要更多实验证明其有效性和通用性。