ai.hackcv
论文精选 65arXiv

CUSUM-Shaped Inference-Time Monitoring and Targeted Re-Decoding for Quantized Small Language Model Reasoning· CUSUM 形状推理时监控与目标重解码

Quantized small autoregressive reasoning models can enter long, repetitive, or unproductive trajectories, yet inference-time compute is usually allocated without observing how a trajectory develops. Building on an earlier token-level e-CUSUM controller, we develop MGT-B (Monitoring-Guided Test-time Backtracking), a revised external controller that maps overlapping windows of pre-sampling uncertainty and degeneration features to position-conditional empirical tail probabilities, accumulates mixture betting factors with a CUSUM-shaped reset, and responds to an alarm by estimating a rollback point, restoring token and key-value-cache state, and performing constrained re-decoding. To audit whether the effect persists on problem identities first observed after the manual choice of log threshold

AI 解读论文

提出 MGT-B 控制器优化小模型推理轨迹

核心方法
通过CUSUM形状的监控机制和目标重解码技术,动态调整推理过程
适合谁读
研究者、工程师
要解决的问题
量化的小自回归模型在推理过程中可能出现重复或无效的轨迹
关键实验
实验验证了MGT-B在不同问题上的效果,但未详细列出
主要贡献
引入了位置条件下的经验尾概率计算和约束重解码,有效优化了推理效果
意义与局限
提高了小模型的推理效率和质量,但仍需进一步测试不同阈值的选择
领域:cs.AI作者:El Hassane Ettifouri、Ayoub Belfatmi、Mahaman Sanoussi Yahaya Alassan
相关推荐

本站内容由 LLM 精选聚合,原文版权归 arXiv 所有 · 摘录仅供参考