Online Label-Free Gradient Matching for Continual Language Model Training· 在线无标签梯度匹配的持续语言模型训练
Language models in customer-facing products must be continually updated, since the data they serve shifts over time. As retraining on the full accumulated data is impractical, each update should instead run on a small set of examples carrying the training signal of all data seen so far. The strongest current methods build this set by gradient matching, which synthesizes examples whose training gradients match those of the full data. However, they require ground-truth labels, run once on a fixed snapshot rather than on a growing stream, and are slow on large data. We introduce \textbf{OLGA}, which lifts these limitations. It distills unlabeled data by drawing the gradient matching targets from the model's own next-token predictions. Across five text classification benchmarks with phi-1.5, i
提出在线无标签持续训练语言模型的方法 OLGA。
- 核心方法
- OLGA 通过从模型自身的 next-token 预测中抽取梯度匹配目标，实现对未标注数据的蒸馏，以少量样本携带所有历史数据的训练信号。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 解决语言模型在客户产品中需要持续更新，但重新训练完整数据集不切实际的问题。
- 关键实验
- 在五个文本分类基准数据集上使用 phi-1.5 进行实验，验证了 OLGA 的有效性。
- 主要贡献
- 1) 实现在线无标签的梯度匹配；2) 适用于大规模数据的高效更新；3) 能够处理不断增长的数据流。
- 意义与局限
- OLGA 为解决语言模型的持续训练问题提供了新的思路，尤其适用于无法重新标注或数据持续增长的场景，但在复杂性和资源消耗上可能仍有局限。