AcrossVAM1.0: Particle World Modeling for Text-Assisted Robot Video Prediction· AcrossVAM1.0: 文本辅助机器人视频预测的粒子世界建模
Predicting robot videos requires both precise motion reasoning and preservation of high-frequency appearance, yet monolithic pixel models entangle these objectives and often conceal their progress behind a strong last-frame baseline. We present AcrossVAM1.0, a lightweight, text-assisted video action model that factorizes future prediction into object-centric motion and dense appearance. A frozen SAM3-DLP codec decomposes four context frames into semantic particles for the robot, arm, and gripper, together with a background latent. A 0.28M-parameter spatio-temporal Transformer aligns particle identities, rolls their states forward, and is modulated by a frozen OpenCLIP instruction embedding through FiLM. A causal dual-stream decoder combines particle-rendered motion with appearance encoded
文本辅助分解视频预测任务，提高机器人预测视频精度与细节。
- 核心方法
- AcrossVAM1.0 采用粒子世界建模，通过语义粒子分解视频的上下文帧，并使用因果双流解码器结合运动与外观信息。
- 适合谁读
- 研究者 / 工程师
- 要解决的问题
- 现有的像素级模型在预测机器人视频时混合了运动和外观的目标，导致预测不准确且细节丧失。
- 关键实验
- 未提供
- 主要贡献
- 提出了轻量级的文本辅助视频动作模型，能够分别处理对象中心运动和密集外观，提高了预测的准确性和细节度。
- 意义与局限
- 有助于提升机器人行为预测的准确性和自然性，但模型的具体实验验证结果尚不明确。