Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe· 韩国开放公共API的多步调用基准
Data-sovereignty regulations increasingly require public institutions to deploy open-source, on-premise LLM agents that chain multiple tool-calls across live government APIs. However, open-source models consistently underperform in this multi-step setting, and no existing benchmark measures the gap. We introduce the Korean Open Public API Benchmark (KOPA-Bench), comprising 145 real-world tasks. To close this gap, we present EDGE, an Execution-grounded Dynamic Graph for tool-calling data synthEsis driven by live execution. EDGE builds a graph of how each tool's output can feed another's input, keeps only the links that succeed when actually called against the live APIs, and traverses these verified links to synthesize executable multi-step trajectories. Fine-tuned via GRPO on the resulting
韩国开放公共API的多步调用基准及其数据生成方法
- 核心方法
- 提出KOPA-Bench基准和EDGE方法,通过实际API执行构建动态图并生成可执行的多步调用轨迹
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 开源模型在多步调用政府API时性能不佳,缺乏相应基准
- 关键实验
- 实验包括145个真实任务,验证了EDGE方法的有效性
- 主要贡献
- 提供了一个针对韩国开放公共API的多步调用基准和一种有效的数据合成方法
- 意义与局限
- 为评估和改进多步调用政府API的开源模型提供了工具,但可能不适用于其他国家的API环境