Beyond the Verdict: Agentic Code Judging with Verifiable Feedback· USACO-JUDGE：具有可验证反馈的代理代码评判
Test-time scaling for large language models (LLMs) can involve generating multiple candidate solutions for selection or revising an earlier attempt. To guide selection and revision, we need a verifier that can estimate candidate correctness and provide evidence of failure. In competitive programming, however, official judging systems provide authoritative verdicts under finite submission budgets, without revealing failure explanations or failing inputs. To provide informative feedback without consuming official submissions, we formulate a task in which a model judges and diagnoses a candidate program using only the problem statement and source code, with generated counterexamples serving as checkable evidence. To evaluate this task, we build USACO-JUDGE from the USACO 2026 contests, using
使用 USACO 数据集，构建代理代码评判系统，提供可验证的错误反馈。
- 核心方法
- 开发一个模型，能够仅基于问题描述和源代码，生成可检查的反例作为错误证据，帮助评估和诊断代码。
- 适合谁读
- 竞争编程领域的研究者和工程师
- 要解决的问题
- 在有限的提交次数下，为竞争编程提供有效的错误诊断和评判，而无需消耗官方提交次数。
- 关键实验
- 使用 USACO 2026 比赛数据集进行了实验，验证了模型的有效性和反馈的可验证性。
- 主要贡献
- 构建了USACO-JUDGE系统，提供了一个新的评估框架，使模型能够在不使用官方提交次数的情况下，给出详尽的反馈信息。
- 意义与局限
- 该研究提高了竞争编程中代码评审的效率和质量，但模型的准确性和反例的生成能力仍有待进一步提升。