When Does Test-Time Search Help World Action Models? An Empirical Audit of Candidate Selection and Simulator Integrity
World-action models increasingly apply test-time scaling, spending extra inference computation on candidate selection and adaptive allocation to improve control. Such gains are credible only when candidates carry exploitable signal, nominal compute savings become real speedup, and evaluation stays reliable. We audit all three on Cosmos Policy and RoboCasa with a premise-first, pre-registered methodology. On clean data, a learned value selector is 1.66 pp worse than random; a hindsight oracle gains only 1.40 pp, with an episode-level interval spanning zero; and 82% of states show identical candidate outcomes, leaving little selection headroom. Token pruning cuts per-forward FLOPs by up to 47% but yields at most a measured 1.15$\times$ end-to-end speedup on the deployed path (1.25$\times$ on
测试时搜索对世界行为模型的控制能力提升有限
- 核心方法
- 通过Premise-first、Pre-registered方法,对Cosmos Policy和RoboCasa的候选选择和模拟器完整性进行审计
- 适合谁读
- 研究者
- 要解决的问题
- 探讨测试时搜索是否能有效提高世界行为模型的性能
- 关键实验
- 在Cosmos Policy和RoboCasa上进行实验,显示学习的价值选择器比随机选择低1.66个百分点,后视预言机仅提高1.40个百分点,且82%的状态候选结果相同;Token pruning最多减少47%的FLOPs,但端到端的加速比最多为1.15倍
- 主要贡献
- 揭示了在清洁数据下,测试时搜索对性能提升的局限性,强调了候选选择的有效性和模拟器可靠性的重要性
- 意义与局限
- 该研究明确了测试时搜索在某些条件下可能无法显著提高世界行为模型的性能,对模型设计和评估方法提出了新的思考,但实验仅限于特定环境,可能不具普遍性