World Modeling in Transformers· Transformer 中的世界建模
Behavioral failures can make a transformer appear to lack a world model even when it has learned faithful representations of its environment. We demonstrate this in TaxiGPT, a transformer trained on random walks through Manhattan whose failures have been interpreted as evidence of an incoherent internal map. Through mechanistic analysis and causal interventions, we show that the model represents intersections and streets, tracks its position, and uses a goal compass to navigate. We trace its failures to interference between superposed intersection features, which disrupts localization within the internal map. Affordance packing, which groups representations of intersections with the same legal moves, helps limit the consequences of these errors. Finally, we propose mechanistic indicators t
研究揭示Transformer在世界建模中的潜在能力和失败原因。
- 核心方法
- 通过对TaxiGPT的机制分析和因果介入，展示Transformer如何表示环境特征和使用目标指南针导航，以及这些机制如何因特征干扰出错。
- 适合谁读
- 适合AI研究者和工程师阅读，特别是关注Transformer模型应用和世界建模的读者。
- 要解决的问题
- Transformer在特定任务中表现出的行为失败，被认为缺乏内部世界模型。
- 关键实验
- 在TaxiGPT模型上进行了机制分析和因果干预实验，验证了模型表示环境并导航的能力。
- 主要贡献
- 提出了影响Transformer世界建模表现的具体指标和改善方法，如能力打包。
- 意义与局限
- 加深了对Transformer内部机制的理解，为改进模型的世界建模能力提供了方向。局限在于研究仅限于特定任务和模型。