Computer Science

Permanent URI for this collectionhttps://hdl.handle.net/10679/43

Browse

Now showing 1 - 16 of 16

Metadata only
Adaptive streaming of content-aware-encoded videos in dash.js
(IEEE, 2022-05) Beğen, Ali Cengiz; Akçay, Mehmet Necmettin; Bentaleb, A.; Giladi, A.; Computer Science; BEĞEN, Ali Cengiz; Akçay, Mehmet Necmettin
In Hypertext Transfer Protocol (HTTP) adaptive streaming, the client makes rate adaptation decisions based on the measured network bandwidth and buffer fullness. This simplifies the adaptation logic; however, it often produces noticeable quality fluctuations during the streaming session. With content-aware encoding (CAE), one can improve the visual quality without increasing the total number of bits spent by carefully choosing where the bits are spent based on human perception. However, an adaptation logic that is unaware of the resulting variable-size segments may cause more stalls, defeating the purpose of improving viewer experience through CAE. This article explains the design steps of a size-aware rate adaptation (SARA) logic for one of the most popular Dynamic Adaptive Streaming over HTTP (DASH) clients, namely dash.js, and shows the improvements in rebuffering behavior and fetching top-resolution segments as a result of applying this logic.
Open Access
Bandwidth prediction in low-latency media transport
(ACM, 2023-06-16) Bentaleb, A.; Akçay, Mehmet Necmettin; Lim, M.; Beğen, Ali Cengiz; Zimmermann, R.; Computer Science; BEĞEN, Ali Cengiz; Akçay, Mehmet Necmettin
Designing a robust bandwidth prediction algorithm for low-latency media transport that can quickly adapt to varying network conditions is challenging. In this paper, we present the working principles of a hybrid bandwidth predictor (termed BoB, Bang-on-Bandwidth) we developed recently for real-time communications and discuss its use with the new Media-over-QUIC (MOQ) protocol proposals.
Metadata only
Benchmarking the second edition of the omnidirectional media format standard
(IEEE, 2022) Kara, Burak; Akçay, Mehmet Necmettin; Beğen, Ali Cengiz; Ahsan, S.; Curcio, I. D. D.; Kammachi-Sreedhar, K.; Aksu, E.; Computer Science; BEĞEN, Ali Cengiz; Kara, Burak; Akçay, Mehmet Necmettin
Omnidirectional MediA Format (OMAF) is the first worldwide virtual reality (VR) standard to store and distribute immersive media, completed in 2019. Later, in 2021, the second edition of this standard (OMAF v2) was published. The second edition kept all the features defined in the first OMAF edition while introducing some new ones, such as overlays and multi-viewpoints. OMAF v2's Tile Index Segments that contain metadata to track fragment data per segment and quality levels create a bandwidth overhead. During the OMAF v2 standardization, multiple methods for the track fragment run representation were studied to deal with this overhead. This paper presents the implementation of one of these methods, the compressed box method using the DEFLATE algorithm (OMAF v2*). It also provides comprehensive test results of OMAF v1, OMAF v2 and OMAF v2∗ with various combinations of three tile grids (6x4, 8x6 and 12x8), three segment durations (300 ms, 900 ms and 3 s), two videos (RollerCoaster and Timelapse), two bitrate groups (each group with four different bitrates) and two HTTP versions (HTTP/1.1 and H2).
Open Access
The benefits of server hinting when DASHing or HLSing
(ACM, 2022-03-17) Lim, M.; Akçay, Mehmet Necmettin; Bentaleb, A.; Beğen, Ali Cengiz; Zimmermann, R.; Computer Science; BEĞEN, Ali Cengiz; Akçay, Mehmet Necmettin
Streaming clients almost always compete for the available bandwidth and server capacity. Not every client's playback buffer conditions will be the same, though, nor should be the priority with which the server processes the individual requests coming from these clients. In an earlier work, we demonstrated that if clients conveyed their buffer statuses to the server using a Common Media Client Data (CMCD) query argument, the server could allocate its output capacity among all the requests more wisely, which could significantly reduce the rebufferings experienced by the clients. In this paper, we address the same problem using the Common Media Server Data (CMSD) standard that is work-in-progress at the Consumer Technology Association (CTA). In this case, the incoming requests are scheduled based on their CMCD information. For example, the response to a request indicating a healthy buffer status is held/delayed until more urgent requests are handled. When the delayed response is eventually transmitted, the server attaches a new CMSD parameter to indicate how long the delay was. This parameter avoids misinterpretations and subsequent miscalculations by the client's rate-adaptation logic. We implemented the server and client understanding/processing CMCD and CMSD, respectively. Our experiments show that the proposed CMSD parameter effectively eliminates unnecessary downshifting while reducing both the rebuffering rate and duration.
Open Access
BoB: Bandwidth prediction for real-time communications using heuristic and reinforcement learning
(IEEE, 2023) Bentaleb, A.; Akçay, Mehmet Necmettin; Lim, M.; Beğen, Ali Cengiz; Zimmermann, R.; Computer Science; BEĞEN, Ali Cengiz; Akçay, Mehmet Necmettin
Bandwidth prediction is critical in any Real-time Communication (RTC) service or application. This component decides how much media data can be sent in real time. Subsequently, the video and audio encoder dynamically adapts the bitrate to achieve the best quality without congesting the network and causing packets to be lost or delayed. To date, several RTC services have deployed the heuristic-based Google Congestion Control (GCC), which performs well under certain circumstances and falls short in some others. In this paper, we leverage the advancements in reinforcement learning and propose BoB (Bang-on-Bandwidth) — a hybrid bandwidth predictor for RTC. At the beginning of the RTC session, BoB uses a heuristic-based approach. It then switches to a learning-based approach. BoB predicts the available bandwidth accurately and improves bandwidth utilization under diverse network conditions compared to the two winning solutions of the ACM MMSys'21 grand challenge on bandwidth estimation in RTC. An open-source implementation of BoB is publicly available for further testing and research.
Open Access
Catching the moment with LoL + in twitch-like low-latency live streaming platforms
(IEEE, 2022) Bentaleb, A.; Akçay, Mehmet Necmettin; Lim, M.; Beğen, Ali Cengiz; Zimmermann, R.; Computer Science; BEĞEN, Ali Cengiz; Akçay, Mehmet Necmettin
Our earlier Low-on-Latency (dubbed as LoL) solution offered an accurate bandwidth prediction and rate adaptation algorithm tailored for live streaming applications that targeted an end-to-end latency of up to two seconds. While LoL was a significant step forward in multi-bitrate low-latency live streaming, further experimentation and testing showed that there was room for improvement in three areas. First, LoL used hard-coded parameters computed from an offline training process in the rate adaptation algorithm and this was seen as a significant barrier in LoL's wide deployment. Second, LoL's objective was to maximize a collective QoE function. Yet, certain use cases have specific objectives besides the singular QoE and this had to be accommodated. Third, the adaptive playback speed control failed to produce satisfying results in some scenarios. Our goal in this paper is to address these areas and make LoL sufficiently robust to deploy. We refer to the enhanced solution as LoL+ which has been integrated to the official dash.js player in v3.2.0.
Open Access
Common media client data (CMCD): Initial findings
(Association for Computing Machinery, Inc, 2021-07-16) Bentaleb, A.; Lim, M.; Akçay, Mehmet Necmettin; Beğen, Ali Cengiz; Zimmermann, R.; Computer Science; BEĞEN, Ali Cengiz; Akçay, Mehmet Necmettin
In September 2020, the Consumer Technology Association (CTA) published the CTA-5004: Common Media Client Data (CMCD) specification. Using this specification, a media client can convey certain information to the content delivery network servers with object requests. This information is useful in log association/analysis, quality of service/experience monitoring and delivery enhancements. This paper is the first step toward investigating the feasibility of CMCD in addressing one of the most common problems in the streaming domain: efficient use of shared bandwidth by multiple clients. To that effect, we implemented CMCD functions on an HTTP server and built a proof-of-concept system with CMCD-Aware dash.js clients. We show that even a basic bandwidth allocation scheme enabled by CMCD reduces rebuffering rate and duration without noticeably sacrificing the video quality.
Metadata only
Content-aware playback speed control for low-latency live streaming of sports
(The ACM Digital Library, 2021) Aladağ, Ö. F.; Uğur, Deniz; Akçay, Mehmet Necmettin; Beğen, Ali Cengiz; Computer Science; BEĞEN, Ali Cengiz; Uğur, Deniz; Akçay, Mehmet Necmettin
There are two main factors that determine the viewer experience during the live streaming of sports content: latency and stalls. Latency should be low and stalls should not occur. Yet, these two factors work against each other and it is not trivial to strike the best trade-off between them. One of the best tools we have today to manage this trade-off is the adaptive playback speed control. This tool allows the streaming client to slow down the playback when there is a risk of stalling and increase the playback when there is no risk of stalling but the live latency is higher than desired. While adaptive playback generally works well, the artifacts due to the changes in the playback speed should preferably be unnoticeable to the viewers. However, this mostly depends on the portion of the audio/video content subject to the playback speed change. In this paper, we advance the state-of-the-art by developing a content-aware playback speed control (CAPSC) algorithm and demonstrate a number of examples showing its significance. We make the running code available and provide a demo page hoping that it will be a useful tool for the developers and content providers.
Metadata only
Could head motions affect quality when viewing 360° videos?
(IEEE, 2023-04-01) Kara, Burak; Akçay, Mehmet Necmettin; Beğen, Ali Cengiz; Ahsan, S.; Curcio, I. D. D.; Aksu, E. B.; Computer Science; BEĞEN, Ali Cengiz; Kara, Burak; Akçay, Mehmet Necmettin
Measuring quality accurately and quickly (preferably in real time) when streaming 360° videos is essential to enhance the user experience. Most quality-of-experience metrics have primarily used viewport quality as a simple surrogate for such experiences at a given time. While this baseline approach has been later augmented by some researchers using pupil and gaze tracking, head tracking has not been considered in enough detail. This article tackles whether head motions can influence the perception of 360° videos. Inspired by the latest research, this article conceptualizes a head-motion-aware metric for measuring viewport quality. A comparative study against existing head-motion-unaware metrics reveals sizeable differences. Motivated by this, we invite the community to research this topic further and substantiate the new metric's validity.
Metadata only
Evaluating the performance of apple’s low-latency HLS
(IEEE, 2020-09-21) Durak, Kerem; Akçay, Mehmet Necmettin; Erinç, Yiğit Kemal; Pekel, Boran; Beğen, Ali Cengiz; Computer Science; BEĞEN, Ali Cengiz; Durak, Kerem; Akçay, Mehmet Necmettin; Erinç, Yiğit Kemal; Pekel, Boran
In its annual developers conference in June 2019, Apple has announced a backwards-compatible extension to its popular HTTP Live Streaming (HLS) protocol to enable low-latency live streaming. This extension offers new features such as the ability to generate partial segments, use playlist delta updates, block playlist reload and provide rendition reports. Compared to the traditional HLS, these features require new capabilities on the origin servers and the caches inside a content delivery network. While HLS has been known to perform great at scale, its low-latency extension is likely to consume considerable server and network resources, and this may raise concerns about its scalability. In this paper, we make the first attempt to understand how this new extension works and performs. We also provide a 1:1 comparison against the low-latency DASH approach, which is the competing low-latency solution developed as an open standard.
Metadata only
Head-motion-aware viewport margins for improving user experience in immersive video
(ACM, 2022-01-10) Akçay, Mehmet Necmettin; Kara, Burak; Ahsan, S.; Beğen, Ali Cengiz; Curcio, I.; Aksu, E.; Computer Science; BEĞEN, Ali Cengiz
Viewport-dependent delivery (VDD) is a technique to save network resources during the transmission of immersive videos. However, it results in a non-zero motion-to-high-quality delay (MTHQD), which is the delta time from the moment where the current viewport has at least one low-quality tile to when all the tiles in the new viewport are rendered in high quality. MTHQD is an important metric in the evaluation of the VDD systems. This paper improves an earlier concept called viewport margins by introducing head-motion awareness. The primary benefit of this improvement is the reduction (up to 64%) in the average MTHQD.
Metadata only
Improving server and client-side algorithms for adaptive streaming of non-immersive and immersive media
(The ACM Digital Library,, 2021) Akçay, Mehmet Necmettin; Akçay, Mehmet Necmettin
HTTP adaptive streaming is a technique widely used in the internet today to stream live and on-demand content. Server and client-side algorithms play an important role in achieving a better user experience in terms of metrics such as latency, rebufferings and rendering quality. In this doctoral study, we propose and evaluate a number of new algorithms for both non-immersive and immersive media in different settings ranging from low-latency live to on-demand streaming.
Metadata only
Meta reinforcement learning for rate adaptation
(IEEE, 2023) Bentaleb, A.; Lim, M.; Akçay, Mehmet Necmettin; Beğen, Ali Cengiz; Zimmermann, R.; Computer Science; BEĞEN, Ali Cengiz; Akçay, Mehmet Necmettin
Adaptive bitrate (ABR) schemes enable streaming clients to adapt to time-varying network/device conditions to achieve a stall-free viewing experience. Most ABR schemes use manually tuned heuristics or learning-based methods. Heuristics are easy to implement but do not always perform well, whereas learning-based methods generally perform well but are difficult to deploy on low-resource devices. To make the most out of both worlds, we develop Ahaggar, a learning-based scheme running on the server side that provides quality-aware bitrate guidance to streaming clients running their own heuristics. Ahaggar's novelty is the meta reinforcement learning approach taking network conditions, clients' statuses and device resolutions, and streamed content as input features to perform bitrate guidance. Ahaggar uses the new Common Media Client/Server Data (CMCD/SD) protocols to exchange the necessary metadata between the servers and clients. Experiments on an open-source system show that Ahaggar adapts to unseen conditions fast and outperforms its competitors in several viewer experience metrics.
Metadata only
Quality upshifting with auxiliary I-Frame splicing
(IEEE, 2023) Akçay, Mehmet Necmettin; Kara, Burak; Beğen, Ali Cengiz; Ahsan, S.; Curcio, I. D. D.; Kammachi-Sreedhar, K.; Aksu, E.; Computer Science; BEĞEN, Ali Cengiz; Akçay, Mehmet Necmettin; Kara, Burak
This paper introduces the Auxiliary I-Frame Splicing method to reduce bandwidth waste in adaptive streaming. This method involves fetching a high-quality I-frame and splicing it into the already downloaded low-quality segment, resulting in a higher-quality rendering at a lower overhead than replacing the entire low-quality segment. In our experiments with three videos and four quantization parameters, the results show that the bandwidth can be saved up to 87% while still increasing the peak signal-to-noise ratio score by 20% and the video multi-method assessment fusion score by 73%. In the demo, we demonstrate the visual differences between the original and spliced videos.
Metadata only
Rate-adaptive streaming of 360-degree videos with head-motion-aware viewport margins
(IEEE, 2022) Akçay, Mehmet Necmettin; Kara, Burak; Beğen, Ali Cengiz; Ahsan, S.; Curcio, I. D. D.; Aksu, E.; Computer Science; BEĞEN, Ali Cengiz; Akçay, Mehmet Necmettin; Kara, Burak
Efficient use of available bandwidth is vital when streaming 360-degree videos as users rarely have enough bandwidth for a pleasant experience. A promising solution is the combination of viewport-dependent streaming using tiled video and rate adaptation, where the goal is to spend most of the available bandwidth for the viewport tiles. However, head motions resulting in a change in the viewport tiles briefly cause low-quality rendering until the new tiles can be replaced with high-quality versions. Previously, viewport margins-fixed regions around the viewport rendered at a medium quality-were proposed to make the viewport changes less abrupt. Later on, Head-motion-aware Viewport Margins (HMAVM) were implemented to further smooth the transitions at the expense of increased bandwidth consumption. In this paper, we manage the overall bandwidth cost of HMAVMs better by first developing a set of algorithms that trade off the quality of some viewport tiles and then making the margin selection part of the rate-adaptation algorithm.
Metadata only
When they go high, we go low: low-latency live streaming in dash.js with LoL
(The ACM Digital Library, 2020-05) Lim, M.; Akçay, Mehmet Necmettin; Bentaleb, A.; Beğen, Ali Cengiz; Zimmermann, R.; Computer Science; BEĞEN, Ali Cengiz; Akçay, Mehmet Necmettin
Live streaming remains a challenge in the adaptive streaming space due to the stringent requirements for not just quality and rebuffering, but also latency. Many solutions have been proposed to tackle streaming in general, but only few have looked into better catering to the more challenging low-latency live streaming scenarios. In this paper, we re-visit and extend several important components (collectively called Low-on-Latency, LoL) in adaptive streaming systems to enhance the low-latency performance. LoL includes bitrate adaptation (both heuristic and learning-based), playback control and throughput measurement modules.

Browse

Browsing by Author "Akçay, Mehmet Necmettin"