Trace Integrity for LLM Data Agents: A Vision for Auditable Structured Reasoning in Real-World Systems· LLM数据代理的Trace Integrity
Answer accuracy is an insufficient reliability signal for LLM data agents. In structured-data tasks, a benchmark-correct answer can be produced by an invalid trace. This paper introduces Trace Integrity, a deployment reliability criterion for evaluating whether the computation recorded behind an answer is explicit, executable, schema-valid, operator-faithful, replayable, answer-consistent, and auditable. We identify the Structure Gap as the deployment failure mode that makes Trace Integrity necessary: natural-language reasoning and free-form rationales do not reliably specify the operator-level programs required by real-world systems. We operationalize Trace Integrity with execution contracts, structured artifacts that bind user intent to schema elements, operator plans, assumptions, execu
LLM数据代理的可靠性评估新标准：Trace Integrity。
- 核心方法
- 引入Trace Integrity作为评估标准，确保计算记录明确、可执行、符合模式、忠实于操作、可重放、答案一致且可审计。通过执行契约（execution contracts）将用户意图与模式元素、操作计划、假设和执行绑定。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 在结构化数据任务中，LLM数据代理可能通过不正确的推理路径产生正确的答案，导致可靠性评估不足。
- 关键实验
- 未提供
- 主要贡献
- 定义了一种新的可靠性评估标准Trace Integrity，并提出了执行契约来实现这一标准，提高了LLM数据代理在实际系统中的可审计性和可靠性。
- 意义与局限
- 增强了LLM数据代理在实际应用中的透明度和可靠性，但需要实际系统中的进一步验证和应用。