Deep Noir: Autonomous Steering Discovery via Architectural Chronometry in Transformer Models· Deep Noir: 通过架构时序在大模型中自主发现转向参数
Activation steering modifies LLM behavior at inference time, but identifying where and how strongly to steer remains manual. We introduce Deep Noir, a framework that uses Logit Lens convergence and causal head-level attribution to autonomously discover optimal steering parameters. Across three scales (1B x 3, 2-3B x 2, and 7-9B x 4), our engine achieves 16.7 percentage-point improvement on spam at 1B (standard deviation 4.7; 39 runs), with gains increasing to 21 to 42 percentage points at 7-9B across four architectures. On SST-2 sentiment, it achieves a 13.1 percentage-point improvement with zero code changes. Mechanistic grounding enables automated discovery of intervention points that generalize across tasks and architectures. On sentiment, RepE without head masking fails to improve over
Deep Noir 自动发现大模型中的最佳转向参数。
- 核心方法
- 通过Logit Lens收敛和因果头级归因，自主发现最优转向参数，实现模型行为的修改。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 如何自动化地确定大型语言模型中激活转向的具体位置和强度。
- 关键实验
- 在1B、2-3B和7-9B三个规模的模型中进行了实验，分别在垃圾信息识别和情感分析任务上取得了16.7%至42%和13.1%的性能提升。
- 主要贡献
- 提出了一种新的框架Deep Noir，能够在不同规模和架构的模型中自动发现转向参数，显著提高模型性能。
- 意义与局限
- 减少了人工干预，提高了模型的通用性和适应性，但可能受限于特定任务的复杂度。