Rethinking Heterogeneous System Disaggregation for Subquadratic Attention· 重新思考异构系统解构以实现次二次注意力
Frontier language models are more aggressively using subquadratic attention to reduce the memory footprint and compute requirements during inference while still delivering frontier accuracy. While existing systems make dense attention-centric disaggregated serving decisions, we show that disaggregating inference around the unique arithmetic intensity and memory footprint of subquadratic attention LLMs can achieve significant throughput and energy efficiency gains on emerging DRAM-based and SRAM-only heterogeneous systems. We introduce SQD (SubQuadratic Disaggregation), a fine-grained heterogeneous disaggregation scheme that splits decode by quadratic and subquadratic attention rather than by operator type, and that applies across subquadratic attention variants. For sparse attention LLMs,
提出了一种针对次二次注意力模型的异构系统解构方案，提高推理效率。
- 核心方法
- 引入SQD方案，按照二次和次二次注意力而非操作类型来分割解码过程，适用于不同次二次注意力模型。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 现有系统在处理具有独特算术强度和内存占用的次二次注意力大模型时，效率低下。
- 关键实验
- 未提供
- 主要贡献
- SQD方案在新兴的基于DRAM和仅SRAM的异构系统上实现了显著的吞吐量和能效提升。
- 意义与局限
- 该研究对优化次二次注意力模型的推理性能具有重要意义，特别是在特定异构计算环境中，但没有提供实验验证。