Title

A CONTEXT-AWARE MODEL FOR STOCHASTIC PLANNING IN ENVIRONMENTS WITH HIDDEN STATES

Abstract

Abstract

Partially Observable Markov Decision Processes (POMDP) have been used extensively to formalize representations for decision-theoretic planning problems to be solved under uncertainty. In this setting, autonomous agents do not have the perfect state information. For this, agents need to store a memory for keeping track of which state it is in depending on the observations. Combined with huge state spaces in most of the domains, policy generation becomes quite costly. To overcome this problem, compact representations using propositional logic and/or grammar-based models are highly beneficial. These representations benefit from the underlying state-action relationship in a given problem setting and use state-variables to represent a state. However, plain POMDPs do not encode these relationships. Existing exact solution algorithms for POMDP planning are inefficient at determining a useful policy in task with huge state space. Based on this motivation, in this thesis, we take our inspiration from an earlier work and propose a new grammar-based model called Context-Aware POMDP (CA-POMDP) for the purpose of representing Markovian sequential decision making problems in a more structured manner in partially observable environments. CA-POMDP changes and augments POMDP facilities by integrating causal relationships between states, actions and observations thereby enabling structural, compact if possible, representation of the tasks. To show the expressive power of CA-POMDP, we give the theoretical bounds for complexity of conversion between POMDP and CA-POMDP. Second, we enhance a policy generation algorithm for fully observable domains to reveal the way for solution procedures for partially observable domains which uses the local relationships for improved performance. We give theoretical definition and analysis of our solution algorithm then present our conducted experiments on numerous problems.

Supervisor(s)

Supervisor(s)

OMER EKMEKCI

Date and Location

Date and Location

2021-09-13 10:00:00

Category

Category

PhD_Thesis