📝 Abstract:
Reinforcement Learning (RL) provides a powerful framework for autonomous deci-sion making; however, its real-world applicability is often bottlenecked by the as-sumption of perfect environmental observability. While Partially Observable Markov Decision Processes (POMDPs) provide a more realistic formulation for these prob-lems, they lead perceptual aliasing which makes long-horizon planning highly unsta-ble for the agent. Landmark-based task decomposition offers a promising solution by utilizing critical environmental waypoints to break complex, global navigation prob-lems into localized, manageable sub-tasks. Despite this, a fundamental challenge per-sists: once landmarks are identified, determining the optimal sequence to visit them remains mathematically and computationally difficult. Current baseline approaches typically rely on passive, stochastic random walks to build transitional associations. This passive methodology aitificially restricts topological discovery, struggles with stochastic sparsity, and suffers from reward agnosticism, fundamentally failing to capture optimal trajectories. To overcome these limitations, this thesis introduces Graph-Based Chain Construe- tion (GBCC), a novel hierarchical decomposition framework that shifts the learn-ing paradigm from passive association to active, concurrent exploration. Rather than relying on a single agent's random walk, GBCC dynamically spawns a swarm of goal-conditioned sub-agents at each discovered landmark, effectively transforming the global exploration burden into localized sub-tasks. These agents systematically construct a directed, value-embedded reachability graph of the environment. Com-prehensive empirical evaluations in complex grid-world domains demonstrate that GBCC significantly outperforms existing benchmark methods. The proposed frame-work achieves complete and robust topological coverage in dense connectivity scenar-ios, successfully identifying optimal, high-yield paths while managing computational trade-offs, thereby establishing a highly scalable approach.