Rule-Compliant Visual Spatial Planning for Multimodal Large Language Models· 多模态大模型的规则合规视觉空间规划
Multimodal large language models (MLLMs) combine linguistic reasoning with visual perception, yet their ability to perform visual spatial planning under explicit or previously unseen rule constraints remains underexplored. This setting requires models to jointly understand spatial layouts, interpret natural-language rules, and plan valid actions accordingly. To address this gap, we introduce RuleMaze, a controllable benchmark in which MLLMs must navigate mazes while obeying natural-language rules of varying complexity. RuleMaze isolates rule-compliant spatial planning by requiring accurate perception, rule interpretation, and constrained action planning. To enable scalable and systematic rule construction, we propose Language-Logic-Function Hybridization, which automatically generates natu
研究多模态大模型在规则约束下的视觉空间规划能力。
- 核心方法
- 提出RuleMaze基准测试和Language-Logic-Function Hybridization方法，前者用于评估模型在迷宫导航中遵守自然语言规则的能力，后者用于自动生成复杂的规则。
- 适合谁读
- 适合多模态大语言模型的研究者和开发人员阅读。
- 要解决的问题
- 现有的多模态大语言模型在处理需要遵守规则的视觉空间规划任务时表现不佳，尤其是在规则复杂或之前未见过的情况下。
- 关键实验
- 通过RuleMaze基准测试评估了多个多模态大语言模型在不同复杂度规则下的表现，结果表明新方法提高了模型的规划能力。
- 主要贡献
- 1. 引入了RuleMaze，一个用于评估模型规则合规视觉空间规划能力的基准测试；2. 提出Language-Logic-Function Hybridization方法，实现了规则的自动生成。
- 意义与局限
- 本研究填补了多模态大语言模型在规则约束下的视觉空间规划研究空白，为未来的研究提供了工具和方法，但目前应用仍限于基准测试环境。