Age-Aware Edge Caching and Multicast Scheduling Using Deep Reinforcement Learning

The temporal nature of data in Internet of Things (IoT) networks necessitates periodic updates of cached content at edge devices, while multicasting dynamic content can enhance network efficiency. This paper addresses the challenge of joint cache updating and multicast scheduling in a cache-enabled, queue-equipped small base station (SBS) with limited cache capacity, which accesses a macro base station (MBS) to download (update) uncached (cached) content and serves requests through multicasting. We formulate a two-stage optimization problem to minimize the average age of information (AAoI) per request, subject to constrained average queueing delay and access rate. The first stage employs the Lyapunov drift-plus-penalty method at the SBS to schedule multicasting and downloading (updating) uncached (cached) content. The second stage, implemented at the MBS, leverages deep reinforcement learning (DRL) to determine the content replacement policy. Simulation results show that the DRL-based cache replacement policy yields up to 50%, 59%, and 60% improvements in AAoI compared to the maximum age, least-recently-used, and least-frequently-used baseline policies, respectively.