CARVE: Verified Expansion for Variable-Length Generation in Diffusion Language Models· CARVE:扩散语言模型中的可验证变长生成
Masked diffusion language models predict tokens from a partially observed response canvas, enabling bidirectional conditioning and parallel token refinement. Yet standard masked-diffusion decoders use a rigid inference interface: the number of masked positions allocated to the answer is fixed before generation begins. Choosing this length is difficult. A short canvas can truncate reasoning or code, while a long canvas wastes computation and can perturb denoising. We introduce CARVE (Counterfactual-Aware Reveal with Verified Expansion), a training-free variable-length algorithm for masked diffusion LMs. Starting from a shorter canvas, CARVE can grow the response during decoding by inserting additional [MASK] positions. Rather than keeping every insertion, CARVE tests a candidate expanded ca
扩散语言模型生成长度可调的新方法。
- 核心方法
- CARVE算法通过从较短的画布开始,在解码过程中根据需要动态插入[MASK]位置,并通过验证测试来决定是否保留这些扩展,实现了生成长度的动态调整。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 扩散语言模型在生成文本时,预先固定的生成长度可能导致资源浪费或生成内容不完整。
- 关键实验
- 未提供
- 主要贡献
- 提出了一种无需额外训练的可验证变长生成算法CARVE,能够在保持生成质量的同时优化计算资源的使用。
- 意义与局限
- 意义在于提升了扩散语言模型在实际应用中的灵活性和效率,但未提供具体实验数据验证其效果。