A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients· 统一分词门控家族
Per-token gating of forward/reverse KL losses has become a standard technique for on-policy knowledge distillation (OPD), but existing methods such as EOPD (Jin et al., 2026) and ToDi (Jung et al., 2025) each fix a single gating signal and a single gating direction, and the two have never been compared directly. We introduce a four-coefficient parameterization lambda_t = sigma(a * h_t + b * u(x) + c + d * gap_t) in which direction-aligned proxies of EOPD and ToDi appear as one-dimensional (1D) restrictions, and which adds multi-channel composition and an explicit bias as further degrees of freedom. On TweetEval (Barbieri et al., 2020) emotion and hate, with a Qwen3-32B teacher and a Qwen3-4B student, configurations in the full family reach higher accuracy than the matched-magnitude single-
通过扩展分词门控技术，提高策略蒸馏模型的性能。
- 核心方法
- 提出了一种四系数参数化方法，结合了多通道组合和显式偏置，扩展了现有方法的限制。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 现有的分词门控技术在策略蒸馏中固定了单一的门控信号和方向，限制了模型的灵活性和性能。
- 关键实验
- 在TweetEval数据集上，使用Qwen3-32B作为教师模型，Qwen3-4B作为学生模型，验证了新方法的有效性。
- 主要贡献
- 提供了更灵活的分词门控方法，使策略蒸馏模型在TweetEval等任务上达到更高的准确率。
- 意义与局限
- 该方法显著提高了策略蒸馏的性能，为未来的优化提供了新的方向，但目前仅在特定任务和模型上进行了验证。