A Novel Adaptive Call Admission Control Scheme for Distributed Reinforcement Learning Based Dynamic Spectrum Access in Cellular Networks

Abstract:
This paper introduces a novel Q-value based adaptive call admission control scheme (Q-CAC) for distributed reinforcement learning (RL) based dynamic spectrum access (DSA) in mobile cellular networks, which provides a good quality of service (QoS) without the need for spectrum sensing. A DSA algorithm has been developed in this paper using the stateless Q-learning algorithm with Win-or-Learn-Fast (WoLF) learning rates. Its performance was analysed using the spatial distribution of the probabilities of call blocking (BP) and dropping (DP) across the network and compared to that of a 100% accurate spectrum sensing based DSA scheme. The Q-CAC scheme demonstrated good controllability of the blocking probability using a Q-value based call admission threshold parameter. It significantly reduced spatial fluctuations in BP and DP, thus providing more cells with acceptable quality of service (QoS).