📝 Abstract:
This study presents an investigation on adaptive parameter optimization techniques for Reinforcement Learning-based Apache Spark job scheduling. Traditional Rein- forcement Learning-based scheduling approaches suffer from the limitations of fixed hyperparameter configurations, requiring extensive manual tuning and often failing to adapt optimally to diverse workload characteristics. The research develops and evaluates adaptive mechanisms that enhance Proximal Policy Optimization (PPO) ef- fectiveness through dynamic parameter adjustment. Four novel adaptive approaches are proposed: adaptive clipping that dynamically adjusts policy update constraints based on Kullback-Leibler divergence feedback, adaptive learning rate mechanisms that modulate optimization step sizes according to training progress, a combined ap- proach leveraging both techniques simultaneously, and enhanced Generalized Advan- tage Estimation for improved value function approximation. The experimental evaluation is conducted within a comprehensive discrete-event sim- ulator that accurately models Apache Spark execution semantics. The proposed mech- anisms are tested using Transaction Processing Performance Council - High Perfor- mance (TPC-H) workloads across multiple random seeds to ensure statistical rigor and reproducibility. The adaptive mechanisms are formulated under the assumptions of policy gradient optimization theory and incorporate feedback-based parameter ad- justment strategies. Sample problems are considered, and the solutions obtained for adaptive mechanisms are compared with those achieved by baseline implementa- tion. The results reveal that, with proper adaptive parameter adjustment, the proposed mechanisms may become advantageous over traditional fixed-parameter approaches in terms of convergence stability, exploration effectiveness, and optimization quality.