CUSUM-Shaped Inference-Time Monitoring and Targeted Re-Decoding for Quantized Small Language Model Reasoning· CUSUM 形状推理时监控与目标重解码
Quantized small autoregressive reasoning models can enter long, repetitive, or unproductive trajectories, yet inference-time compute is usually allocated without observing how a trajectory develops. Building on an earlier token-level e-CUSUM controller, we develop MGT-B (Monitoring-Guided Test-time Backtracking), a revised external controller that maps overlapping windows of pre-sampling uncertainty and degeneration features to position-conditional empirical tail probabilities, accumulates mixture betting factors with a CUSUM-shaped reset, and responds to an alarm by estimating a rollback point, restoring token and key-value-cache state, and performing constrained re-decoding. To audit whether the effect persists on problem identities first observed after the manual choice of log threshold
提出 MGT-B 控制器优化小模型推理轨迹
- 核心方法
- 通过CUSUM形状的监控机制和目标重解码技术,动态调整推理过程
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 量化的小自回归模型在推理过程中可能出现重复或无效的轨迹
- 关键实验
- 实验验证了MGT-B在不同问题上的效果,但未详细列出
- 主要贡献
- 引入了位置条件下的经验尾概率计算和约束重解码,有效优化了推理效果
- 意义与局限
- 提高了小模型的推理效率和质量,但仍需进一步测试不同阈值的选择