Meta reinforcement learning for rate adaptation

Abstract

Adaptive bitrate (ABR) schemes enable streaming clients to adapt to time-varying network/device conditions to achieve a stall-free viewing experience. Most ABR schemes use manually tuned heuristics or learning-based methods. Heuristics are easy to implement but do not always perform well, whereas learning-based methods generally perform well but are difficult to deploy on low-resource devices. To make the most out of both worlds, we develop Ahaggar, a learning-based scheme running on the server side that provides quality-aware bitrate guidance to streaming clients running their own heuristics. Ahaggar's novelty is the meta reinforcement learning approach taking network conditions, clients' statuses and device resolutions, and streamed content as input features to perform bitrate guidance. Ahaggar uses the new Common Media Client/Server Data (CMCD/SD) protocols to exchange the necessary metadata between the servers and clients. Experiments on an open-source system show that Ahaggar adapts to unseen conditions fast and outperforms its competitors in several viewer experience metrics.