ai.hackcv
论文精选 65arXiv

RoboTTT: Context Scaling for Robot Policies· RoboTTT: 扩展机器人策略的上下文规模

Recent robot foundation models operate with single-step or short-history visuomotor context. We introduce Test-Time-Training Robot Policies (RoboTTT), a robot model and training recipe that scale visuomotor context to 8K timesteps, three orders of magnitude beyond state-of-the-art policies, without growing inference latency. At this context length, we unlock new robot capabilities: one-shot in-context imitation from human video demonstrations, on-the-fly policy improvement, robustness to perturbations, and stronger performance on multi-stage, long-horizon tasks. We also observe, for the first time, steady gains in closed-loop performance as pretraining context length scales. At its core, RoboTTT integrates Test-Time Training into robot foundation models such as Vision-Language-Action policies, yielding a sequence model whose recurrent state consists of fast weights, parameters updated by gradient descent during both training and inference, compressing histories into weight space and retrieving contextual information for long-context conditioning. To scale training context length, the recipe combines sequence action forcing with truncated backpropagation through time. On challenging real-robot manipulation tasks, RoboTTT improves overall performance by 87% over the single-step context baseline and fully completes a five-minute, ten-stage assembly task, which no baseline ever does. RoboTTT trained with 8K-timestep context outperforms the same model pretrained with 1K timesteps by 62%, suggesting context length as a new scaling axis for robot foundation models. Videos are available at https://research.nvidia.com/labs/gear/robottt/

AI 解读论文

机器人模型 RoboTTT 实现了 8K 时间步的上下文扩展,大幅提升多阶段任务性能。

核心方法
通过将测试时训练(Test-Time Training)集成到视觉-语言-行动策略中,采用快速权重和截断时间反向传播技术,实现长上下文条件下的历史信息压缩与检索。
适合谁读
适合机器人研究者和工程师阅读
要解决的问题
当前机器人基础模型只能处理单步或短期的视觉运动上下文,限制了其在复杂任务中的表现。
关键实验
在真实机器人操作任务中,RoboTTT 相对于单步上下文基线模型提升了87%的性能,并完全完成了长达五分钟的十阶段装配任务。
主要贡献
首次实现了8K时间步的上下文扩展,提升了机器人在多阶段、长时间任务中的表现和鲁棒性。
意义与局限
扩展上下文规模不仅显著提高了机器人性能,还开启了新的能力,如单次模仿学习和即时策略改进。然而,该方法的计算资源需求较高,可能限制其广泛应用。
领域:cs.RO作者:Yunfan Jiang、Yevgen Chebotar、Ruijie Zheng
相关推荐

本站内容由 LLM 精选聚合,原文版权归 arXiv 所有 · 摘录仅供参考