Interactive Reward Agent: GUI Task Evaluation via Environment-State Verification· 交互式奖励代理：环境状态验证的GUI任务评估
Graphical user interface task evaluation aims to determine whether a GUI agent has successfully completed a user instruction. Automated GUI task evaluation has received increasing attention because the evaluation results can serve as reward signals for both test-time scaling and post-training. However, reliable GUI task evaluation remains challenging because the judgments often require access to environment states, such as system configurations, file data, and application settings, beyond the screenshots of execution trajectories. In this paper, we propose an interactive reward agent (IRA) based on a propose-then-verify framework to acquire and verify evidence from the post-execution environment. Given a task instruction and a GUI environment after the GUI agent execution, IRA first propos
通过环境状态验证评估GUI代理任务完成情况。
- 核心方法
- 提出了一种基于提出-验证框架的交互式奖励代理(IRA)，从执行后的环境获取并验证证据。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 自动化GUI任务评估难以获得执行后环境的状态信息，导致评估结果不可靠。
- 关键实验
- 实验对比了IRA与仅依赖截图的传统评估方法，展示了IRA在多种GUI任务中的优越性。
- 主要贡献
- 1. 引入了环境状态验证机制；2. 提高了GUI任务评估的准确性；3. 可作为强化学习的奖励信号。
- 意义与局限
- 本方法提高了自动化GUI评估的可靠性，对GUI测试和代理的持续学习有重要影响。局限在于需要额外的环境状态访问权限。