ai.hackcv
论文精选 83arXiv

Oilbird: Training-Free Speculative Decoding with Keys the Verifier Already Computes· Oilbird:利用验证者已计算键的免训练推测解码

Training-free speculative decoding drafts by matching an exact suffix of the context against a pool of earlier context. That lookup misses correct drafts already in the pool, most visibly on tool-calling traffic, where a request repeats almost everything but the few values minted for it, and where one rejected token discards the correct continuation behind it. We diagnose the failure position by position across ten benchmarks and find it to be a problem of addressing rather than of coverage: on our densest tool-calling benchmark, about half of what the strongest exact-match drafter misses is present in the pool yet unreachable by exact matching. We therefore propose a second, semantic draft source: the same pool, re-keyed by the hidden state the verifier has already computed at each commit

领域:cs.AI作者:Tao Jin、Phuong Minh Nguyen、Zhenzhu Yan
相关推荐

本站内容由 LLM 精选聚合,原文版权归 arXiv 所有 · 摘录仅供参考