Property-driven Causal Abstractions for Markov Decision Processes· 用于马尔可夫决策过程的属性驱动因果抽象
Markov Decision Processes (MDPs) are widely used as decision-making models, commonly specified over factored state spaces through state variables and their valuations. The exponential blowup in the number of states renders many reasoning tasks in MDPs challenging. Abstractions are promising techniques to reduce MDPs and thus mitigate scalability issues. In this work, we introduce a notion of causality on factored MDPs and a novel property-driven causal abstraction technique that retains many characteristics of the original MDP model. For this, we rely on causal relations over state variable predicates and identify those states that share the same reasons for fulfilling or violating a given abstraction property. We theoretically and empirically compare various causal MDP abstractions using
提出了一种基于因果关系的属性驱动抽象方法，减少 MDP 的状态空间。
- 核心方法
- 通过定义状态变量谓词上的因果关系，识别和抽象出具有相同属性影响状态的集合。
- 适合谁读
- 研究者、工程师
- 要解决的问题
- 解决马尔可夫决策过程中由于状态空间指数增长导致的可扩展性问题。
- 关键实验
- 理论上和实证上比较了不同的因果 MDP 抽象技术。
- 主要贡献
- 引入了属性驱动因果抽象的概念，保留了原 MDP 模型的许多特性，提高了处理大规模 MDP 的效率。
- 意义与局限
- 该方法可以显著减少状态空间，提高 MDP 模型的分析和求解效率，但可能对某些特定问题的精确度有所牺牲。