The Surprising Effectiveness of Approximate Value Iteration in Self-Play· 自对弈中近似价值迭代的惊人效果
Combining search with function approximation has driven major advances in game-playing programs, making self-play algorithms more competitive than ever. Still, the computational overhead of the most popular methods, based on Monte Carlo Tree Search (MCTS), can be substantial. In this work, we investigate whether simpler methods remain competitive in non-trivial, moderately sized games such as Connect Four, Hex(7x7) and synthetic games. We train a minimal self-play implementation of Approximate Value Iteration (AVI) and use ground-truth oracles for exact evaluation. Contrary to expectations, our results demonstrate the surprising effectiveness of AVI: it learns more accurate value functions than those learned by AlphaZero, while its one-step-lookahead greedy policies remain competitive with
近似价值迭代在自对弈中的表现超越了AlphaZero，学习更准确的价值函数。
- 核心方法
- 使用近似价值迭代（AVI）结合真实值评估器，训练最小化自对弈实现。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 研究如何在减少计算开销的情况下，保持自对弈算法在中等规模游戏中的竞争力。
- 关键实验
- 关键实验包括Connect Four、Hex(7x7)和合成游戏中的自对弈表现对比。
- 主要贡献
- 证明了AVI在中等规模游戏中的有效性，与AlphaZero相比，学习更准确的价值函数并且策略竞争力强。
- 意义与局限
- 研究结果表明，在特定条件下，更简单的方法也可以达到或超过复杂方法的效果，对降低游戏AI的计算成本有重要意义。