ai.hackcv
论文精选 82arXiv

Decoupling Is Not Identification: Supervised Evidential Learning in Next-Token Prediction· 解耦不代表识别:监督证据学习在下一 token 预测

A next-token probability says what a model predicts, not how much training support lies behind it. A Dirichlet head can represent this distinction by separating mean $m$ from concentration $S$, but decoupling does not identify what $S$ means. Here we propose an Evidential Next-Token Prediction (ENTOP) framework to audit this gap on character-level Moby-Dick, using exact 8-gram count as a reproducible lexical-support label and withholding count regression from 20% of context types. Standard implicit evidential training carries essentially no count signal beyond confidence on held-out-label types (partial Spearman $ρ= 0.001 \pm 0.014$), whereas explicit supervision generalizes ($ρ= 0.201 \pm 0.010$; matched-pair win $= 0.822 \pm 0.021$). CE predictive entropy is at chance for unseen 8-grams

领域:cs.AI作者:Ge Wang
相关推荐
论文精选 65已解读

Simulated Control Evals· 模拟控制评估

Control evaluations measure whether safeguards can prevent an AI model from causing undesi…

领域:cs.AI作者:Jeffrey Olmo、David Zhang、Tyler Tracy
📎 arXiv🕒 10-23 01:26🔗 arxiv.org

本站内容由 LLM 精选聚合,原文版权归 arXiv 所有 · 摘录仅供参考