Type-Safe Is Not Error-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to It· 类型安全不等于无错误
Typed decision models are built for settings where model outputs are consumed directly by software. Instead of generating free-form text, they return a decision over a predefined set of options. By construction, every output conforms to the required schema. Yet this guarantee does not tell us whether the model interprets the options as intended. We study Jev and two Jev-like models with open weights by changing how option names are assigned to rubrics. Each option consists of an option name and a textual rubric that defines what the option means. We change only which option name is assigned to each rubric; the question, state, rubric wording, and set of option names remain exactly the same. On 1200 workflow decisions with task-specific rubrics, renaming the two options from 0/1 to no/yes c
类型安全决策模型的选项命名影响模型理解，导致预测错误。
- 核心方法
- 通过改变选项名称而不改变其对应描述（rubric），研究模型对选项理解的影响。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 类型安全的决策模型虽然保证输出符合预定义的格式，但可能误解选项的实际意义，从而产生错误决策。
- 关键实验
- 在1200个带有任务特定描述的工作流决策上，将两个选项从0/1重命名为no/yes进行实验。
- 主要贡献
- 揭示了类型安全决策模型中选项命名对模型理解的重要性，提出了评估模型理解误差的方法。
- 意义与局限
- 强调了在设计类型安全决策模型时，不仅要关注输出格式的正确性，还要确保模型对选项的实际意义有正确理解。此研究对提高模型的解释性和可靠性具有重要意义。