Advancing Decision-Making in Dynamic Environments

Abstract

This paper introduces the Adaptive Thompson Sampling (AdaTS) algorithm, specifically designed for the challenges posed by opportunistic bandit problems. Unlike conventional multi-armed bandit scenarios where the optimality of actions remains relatively static, opportunistic bandit environments are significantly influenced by fluctuating external conditions. These variations necessitate a highly adaptive strategy for decision-making. The AdaTS algorithm meets this requirement through a novel integration of real-time system load assessments, which dynamically adjust the balance between exploration and exploitation. This method not only ensures more effective adaptation to changing conditions but also considerably enhances performance metrics. By incorporating system load into the decision-making process, AdaTS effectively reduces regret and maximizes reward, particularly under scenarios of variable load intensities. This is particularly evident in environments characterized by binary-valued loads and stochastic reward distributions. The results highlight AdaTS's robustness and efficiency, making it a promising approach for complex environments where adaptability is crucial. This research marks a significant advancement in the field of adaptive algorithms for opportunistic bandit problems.