ai.hackcv
论文精选 78arXiv

Flide: Accelerating Decision Inference with Exact State Forking· Flide:加速决策推理

Decision models such as Jev have emerged as a highly promising paradigm for LLM agent systems, providing lightweight and reliable categorical reasoning for rapid System One execution. However, existing generation-oriented inference engines fail to support these workloads effectively, suffering from severe prefix recomputation overheads or distorted output probabilities. To address this challenge, we propose Flide, an efficient and exact inference engine that decouples the logical tree structure of a decision request from its physical execution within a single flattened token stream. Through ForkAttention, Flide navigates request dependencies using GPU-resident indices across both standard attention and recurrent Gated DeltaNet layers, guaranteeing in exact arithmetic that every query branc

领域:cs.CL作者:Bobo Li、Jiajia Song、Hao Fei
相关推荐
论文精选 82

Elucidating the Design Space of LM Data Attribution· 阐明大模型数据归属的设计空间

Data attribution methods for Language Models (LMs) are typically evaluated using downstrea…

领域:cs.CL作者:Steven Yu、Sheng-Yu Wang、Kimia Hamidieh
📎 arXiv🕒 10-30 01:59🔗 arxiv.org

本站内容由 LLM 精选聚合,原文版权归 arXiv 所有 · 摘录仅供参考