Optimizing cloud elasticity : A Deep Reinforcement Learning approach enhanced by transfer learning.

Abstract

Cloud elasticity enables providers to dynamically scale application resources in response to fluctuating demand. Traditional scaling mechanisms often rely on simple heuristics, which can lead to suboptimal performance and resource utilization. This work proposes a Deep Reinforcement Learning based controller designed to manage cloud resources more efficiently. Although RL-based controllers have been explored previously, they often suffer from poor initial performance, which limits their practical applicability in real-world scenarios. To address this issue, an investigation into transfer learning is performed, and two distinct transfer learning techniques are explored: Sim-to-Real Transfer and Learning from Demonstrations, which significantly enhance initial controller performance, making RL viable for cloud elasticity management from the outset. Sim-to-Real Transfer utilizes simulation-based training to embed the model with prior knowledge, while Learning from Demonstrations leverages expert behaviors to significantly improve early-stage performance, thereby reducing the time required for effective scaling. The proposed model was evaluated using CloudSim Plus, a well established cloud simulation tool. The results demonstrate substantial performance improvements over traditional heuristic methods, with both transfer learning techniques notably improving the initial deployment phase of the RL controller. Specifically, these advancements render the use of RL in cloud elasticity scenarios not only viable but also highly advantageous. These findings open avenues for further exploration of RL-based cloud management strategies and demonstrate the potential of transfer learning to make RL models suitable in scenarios where it was previously unfeasible.