Adaptive Multi-Token Prediction: How far should we go?· 自适应多 token 预测：我们应该走多远？
Multi-token prediction (MTP) can accelerate language-model decoding by drafting several tokens and verifying them in a single target-model pass. However, the best speculation horizon changes with context: longer drafts offer greater amortization when reliable, but can waste compute on suffixes discarded after a rejection. Unlike autoregressive drafting, where a stopping rule can decide token by token, MTP must fix the horizon before drafting, as all heads fire in the same forward pass. We therefore formulate horizon selection as an online control problem, using verification feedback to learn how far the draft can be trusted in each context, and introduce Adaptive Multi-Token Prediction (AMTP): a lightweight controller that requires no retraining and leaves the exact verifier unchanged. To
提出自适应多 token 预测以优化语言模型解码速度和计算资源利用。
- 核心方法
- 将预测范围选择视为在线控制问题，引入 AMTP 控制器，根据验证反馈自适应调整每次预测的 token 数量。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 多 token 预测在语言模型解码中如何确定最佳预测范围，以减少计算浪费。
- 关键实验
- 未提供
- 主要贡献
- AMTP 不需要重新训练模型，且不改变验证器，能够灵活适应不同上下文。
- 意义与局限
- AMTP 可以提高语言模型解码效率，减少计算资源浪费，但需要在实际应用中验证其效果。