ExplorationBench: Measuring AI Systems' Exploration in Verifiable Alien Worlds· ExplorationBench：衡量AI系统在可验证外星世界中的探索
Scientific discovery begins where known problems end. There, AI systems must engage in exploration: framing hypotheses, designing experiments, and iterating on the results. However, evaluating this ability is difficult: (1) how to verify whether a genuinely new hypothesis holds, and (2) how to determine whether a system has discovered it through exploration or merely recalled related knowledge from pre-training data. To this end, we introduce ExplorationBench, which turns the wicked problem of evaluating scientific exploration into a concrete and tractable framework built on verifiable Alien Worlds: their rules are executable, so every answer can be checked exactly, and they conflict with familiar knowledge, so recall alone cannot solve the tasks. The benchmark contains two sandboxes, Alie
提出ExplorationBench，评估AI系统在外星世界的探索能力。
- 核心方法
- 创建了基于可验证外星世界的框架，包括两个沙盒环境（Alien Math和Alien Biology），确保AI通过探索而非回忆解决问题。
- 适合谁读
- 研究者
- 要解决的问题
- 如何有效评估AI系统在未知环境中的探索能力，特别是在科学发现方面。
- 关键实验
- 未提供
- 主要贡献
- 提供了一个新颖的评估工具，能够有效衡量AI在科学探索方面的表现。
- 意义与局限
- 为AI领域的科学探索研究提供了新的方向和评估标准，但受限于人工设定的外星世界，可能无法完全反映真实的科学探索过程。