ai.hackcv
论文精选 65arXiv

The Transformer Revolution, Part 1: Dynamic Processing through Output- Weight Interconnections· Transformer革命:输出权重互联的动态处理

This paper offers a new interpretation of the Transformer during inference. Against the "stochastic parrot" view that large language models merely reproduce statistical regularities learned in training, we argue that Transformers construct and apply prompt-dependent transformations whose parameters are generated during inference. We call this form of computation SIDPP: Sequence-level Interactive Dynamic Parallel Processing. The Transformer is interpreted as a system that transforms concepts by means of concepts. Token vectors are the concepts to be transformed; parameterized transformations defined by matrices and vectors are the transforming concepts. These may be static, when fixed through training, or dynamic, when generated from the input sequence. Mechanically, they correspond to grou

AI 解读论文

重新解读Transformer,提出SIDPP动态处理新机制

核心方法
提出了SIDPP(序列级交互式动态并行处理)的概念,其中Transformer在推理时根据输入序列生成动态参数化的转换,用这些转换来处理和构建输入序列的概念
适合谁读
研究者
要解决的问题
论文试图解决大型语言模型在推理过程中只是简单地重现训练过程中学到的统计规律,而非真正进行动态处理的问题
关键实验
未提供
主要贡献
提供了一种新的Transformer工作机制理解,强调了其在推理阶段的动态处理能力,而非静态的模式复制
意义与局限
此研究有助于深入理解Transformer模型的内部工作原理,推动更高效和灵活的语言模型设计;但缺乏实验证据支持理论的有效性
领域:cs.AI作者:Marco Giunti、Fabrizia Giulia Garavaglia
相关推荐

本站内容由 LLM 精选聚合,原文版权归 arXiv 所有 · 摘录仅供参考