Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstruction· Scal3R: 提升在线 3D 重建效率
Online 3D reconstruction models perform poorly on long videos. This happens because regressing poses relative to a fixed first-frame anchor forces extrapolation far beyond the training distribution. Small drifts accumulate and amplify into significant geometric collapse. However, we observe that per-frame depth remains stable throughout this failure. The backbone's local geometry remains intact; only the global pose head breaks down. Motivated by this decoupling, we introduce Scal3R. This approach reformulates online reconstruction as multi-reference relative pose querying. We use lightweight learnable tokens, which make up about ~1% of the parameters, and inject them into a completely frozen backbone via asymmetric attention. This setup queries poses relative to multiple past keyframes. An online pose-graph optimization system with loop closure suppresses long-range drift. Scal3R reaches convergence in 8 hours on a single GPU. It reduces the average ATE by over 60% on KITTI compared to the online baseline. It also achieves state-of-the-art performance across Virtual KITTI, Sintel, TUM-Dynamic, ScanNet, and 7-Scenes. Project page: https://linjohnss.github.io/scal3r/
提出 Scal3R 以多参考相对姿态查询方式提升在线 3D 重建效率。
- 核心方法
- Scal3R 将在线重建重新定义为多参考相对姿态查询，使用轻量级可学习令牌并通过非对称注意将其注入完全冻结的主干模型，相对多个过去的关键帧查询姿态，并采用带有闭环的在线姿态图优化系统来抑制长距离漂移。
- 适合谁读
- 适合计算机视觉研究者和工程师阅读
- 要解决的问题
- 在线 3D 重建模型在处理长视频时表现不佳，因为相对于固定的第一帧基准的姿态回归会导致超出训练分布的外推，小偏差累积放大导致显著的几何坍塌。
- 关键实验
- 关键实验包括在 KITTI、Virtual KITTI、Sintel、TUM-Dynamic、ScanNet 和 7-Scenes 数据集上的性能评测，证明了 Scal3R 的有效性和优越性。
- 主要贡献
- Scal3R 在单个 GPU 上仅需 8 小时即可收敛，相比于在线基准在 KITTI 数据集上平均 ATE 降低超过 60%，并在多个数据集上达到最先进性能。
- 意义与局限
- 这项研究提高了在线 3D 重建的效率和准确性，对于实时应用如自动驾驶和增强现实具有重要意义。其局限性在于依赖于姿态图优化系统和闭环机制，可能在动态环境中效果不佳。