ai.hackcv
论文精选 60arXiv

ACCT: Asymmetric Cross-LLM Context Transfer· ACCT:不对称跨大模型上下文传输

In real-world deployments, large language model (LLM) systems are increasingly evolving from single-model invocation toward heterogeneous deployments in which different models are dynamically invoked for inference. When the same context is reused across models, however, each newly selected target model typically has to process it again, repeatedly incurring substantial computational costs. Therefore, we propose ACCT (Asymmetric Cross-LLM Context Transfer), which targets KV cache transfer between models of different scales within the same model family. ACCT transforms the KV cache produced by a source model into sparse KV states that can be directly consumed by a target model, thereby enabling cross-model reuse of context computation and decoupling context computation from downstream infere

AI 解读论文

提出 ACCT,实现跨大模型的上下文计算复用,降低计算成本。

核心方法
通过将源模型产生的KV缓存转换为稀疏KV状态,实现不同规模模型间的上下文传输复用。
适合谁读
研究者 / 工程师
要解决的问题
在异构大模型部署中,不同模型间重复处理相同上下文导致计算成本高。
关键实验
未提供
主要贡献
减少跨模型上下文处理的计算负担,提高推理效率。
意义与局限
对异构大模型系统的优化有重要意义,但现阶段可能仅限于相同模型家族内的应用。
领域:cs.AI作者:Yuzhe Liu、Haolin Tian、Tonghan Wang
相关推荐

本站内容由 LLM 精选聚合,原文版权归 arXiv 所有 · 摘录仅供参考