ABR logic in video and audio

[MardenSSS]

Hello,

I'm curious if the ABR logic also applies to audio (selecting the appropriate one among multiple audio bitrates) when playing a video source that contains both sound and images in dash.js, not just video. If so, I want to know if the ABR logic makes decisions and downloads for audio chunks and video chunks synchronously. In other words, will the ABR logic make bitrate decisions and send requests for the i-th audio chunk and the i-th video chunk at the same time? If the i-th audio chunk has been downloaded, but the i-th video chunk has not yet been downloaded, will ABR wait for the audio chunk to be downloaded before synchronously selecting the bitrate for the i+1-th video chunk and the _i+1-th_ audio chunk?

Any help would be much appreciated.

[dsilhavy]

• Yes, the ABR logic is also applied for audio
• There are two separate instances of the ScheduleController class, one for video and one for audio. The current logic kind of synchronizes them by aiming for an audio buffer that is equal to the video buffer. This is done by checking the current videoBufferLevel and comparing it to the audioBufferLevel. I am linking two functions below that might help to understand the logic:

  • dash.js/src/streaming/controllers/ScheduleController.js

    Line 235 in ca4c7aa

    function getBufferTarget() {

  • dash.js/src/streaming/controllers/ScheduleController.js

    Line 284 in ca4c7aa

    function _getBufferTargetForAudio() {

As a consequence, the request for audio segment i+1 should not be done before the i-th video segment request completed (assuming they have a similar duration)