Common Media Client Data (CMCD) implementation #1461
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
peaBerberian
added
work-in-progress
peaBerberian
force-pushed
the
feat/cmcd
branch
10 times, most recently
from
035468a
to
9c097d3
Compare
peaBerberian
added
the
Priority: 2 (Medium)
peaBerberian
changed the title
peaBerberian
removed
the
work-in-progress
peaBerberian
added a commit
that referenced
this pull request
Jun 21, 2024
While working on CMCD (#1461), I saw an opportunity for doing a refactoring that I initially postponed: moving what was called the `DownloadingQueue` from the `RepresentationStream` to the `fetchers` code. The modules involved -------------------- The `DownloadingQueue` is a class allowing to put the current list of wanted segments for a given media type (audio, video, text) into a FIFO queue so they can be requested from the most urgent (usually the one coming right next in the corresponding media buffer) to the least urgent (usually the ones consecutive to that segment, in chronological order) - through a simple interface. The `DownloadingQueue` does both the queueing (the list of wanted segments is given to it as input) and the requesting (by relying on an instanciated `SegmentFetcher` - defined in the `fetchers` code) as well as some optimizations such as requesting the initialization segment at the same time than the initially-wanted media segment when possible. Previously, that `DownloadingQueue` was part of the `RepresentationStream` (in the `src/core/stream/representation` directory). The `RepresentationStream` is documented as the module whose role is to find the right segments to load, to orchestrate their requests and then to push them into the right `SegmentSink` (`src/core/segment_sinks`). In terms of responsibility and readability of the code it can make sense to put the `DownloadingQueue` in the `RepresentationStream`, but I always felt that it would be more at its place in the `fetchers` code for segments instead (in `src/core/fetchers/segment). The `fetchers` code for segments's role is to actually perform the segment requests, and do so in a transport-agnostic way (the transport-specific part being defined in `src/transports` which is directly used by the `fetchers` code). What I did here --------------- Basically, I moved the `DownloadingQueue` file into the `src/core/fetchers/segment` directory and renamed it `SegmentQueue` as it made more sense to me to remove the `Downloading` part from the name now that it is implied from its new location, and add the `Segment` part, which is sort of a convention in the `fetchers` to separate manifest-related code to segment-related code. I then made the `SegmentQueue` one of the main modules to be used by code external to the `fetchers` (here principally the `RepresentationStream`) instead of the `PrioritizedSegmentFetcher` like before. I also tried to simplify the whole `fetchers` API for segments so it can be straightforward to use from other modules without having to understand logic specific to that part. What's the relation with CMCD ----------------------------- As written at the start, this was done in relation with my work on CMCD, because that scheme can optionally provide to the CDN hints about what segment it is going to load next (with the previous segment's request). Previously, this would have meant an akward supplementary parameter somewhere asking for information on the next wanted segment for what's in the end very specific. By instead updating the `fetchers` (which is the module directly exploiting CMCD) so its API is already aware of the current queue of wanted segments (and not just the most urgent one like before), implementing "next segment hints" can be done without having to involve any other module.
peaBerberian
added a commit
that referenced
this pull request
Jun 21, 2024
While working on CMCD (#1461), I saw an opportunity for doing a refactoring that I initially postponed: moving what was called the `DownloadingQueue` from the `RepresentationStream` to the `fetchers` code. The modules involved -------------------- The `DownloadingQueue` is a class allowing to put the current list of wanted segments for a given media type (audio, video, text) into a FIFO queue so they can be requested from the most urgent (usually the one coming right next in the corresponding media buffer) to the least urgent (usually the ones consecutive to that segment, in chronological order) - through a simple interface. The `DownloadingQueue` does both the queueing (the list of wanted segments is given to it as input) and the requesting (by relying on an instanciated `SegmentFetcher` - defined in the `fetchers` code) as well as some optimizations such as requesting the initialization segment at the same time than the initially-wanted media segment when possible. Previously, that `DownloadingQueue` was part of the `RepresentationStream` (in the `src/core/stream/representation` directory). The `RepresentationStream` is documented as the module whose role is to find the right segments to load, to orchestrate their requests and then to push them into the right `SegmentSink` (`src/core/segment_sinks`). In terms of responsibility and readability of the code it can make sense to put the `DownloadingQueue` in the `RepresentationStream`, but I always felt that it would be more at its place in the `fetchers` code for segments instead (in `src/core/fetchers/segment). The `fetchers` code for segments's role is to actually perform the segment requests, and do so in a transport-agnostic way (the transport-specific part being defined in `src/transports` which is directly used by the `fetchers` code). What I did here --------------- Basically, I moved the `DownloadingQueue` file into the `src/core/fetchers/segment` directory and renamed it `SegmentQueue` as it made more sense to me to remove the `Downloading` part from the name now that it is implied from its new location, and add the `Segment` part, which is sort of a convention in the `fetchers` to separate manifest-related code to segment-related code. I then made the `SegmentQueue` one of the main modules to be used by code external to the `fetchers` (here principally the `RepresentationStream`) instead of the `PrioritizedSegmentFetcher` like before. I also tried to simplify the whole `fetchers` API for segments so it can be straightforward to use from other modules without having to understand logic specific to that part. What's the relation with CMCD ----------------------------- As written at the start, this was done in relation with my work on CMCD, because that scheme can optionally provide to the CDN hints about what segment it is going to load next (with the previous segment's request). Previously, this would have meant an akward supplementary parameter somewhere asking for information on the next wanted segment for what's in the end very specific. By instead updating the `fetchers` (which is the module directly exploiting CMCD) so its API is already aware of the current queue of wanted segments (and not just the most urgent one like before), implementing "next segment hints" can be done without having to involve any other module.
peaBerberian
added a commit
that referenced
this pull request
Jun 21, 2024
While working on CMCD (#1461), I saw an opportunity for doing a refactoring that I initially postponed: moving what was called the `DownloadingQueue` from the `RepresentationStream` to the `fetchers` code. The modules involved -------------------- The `DownloadingQueue` is a class allowing to put the current list of wanted segments for a given media type (audio, video, text) into a FIFO queue so they can be requested from the most urgent (usually the one coming right next in the corresponding media buffer) to the least urgent (usually the ones consecutive to that segment, in chronological order) - through a simple interface. The `DownloadingQueue` does both the queueing (the list of wanted segments is given to it as input) and the requesting (by relying on an instanciated `SegmentFetcher` - defined in the `fetchers` code) as well as some optimizations such as requesting the initialization segment at the same time than the initially-wanted media segment when possible. Previously, that `DownloadingQueue` was part of the `RepresentationStream` (in the `src/core/stream/representation` directory). The `RepresentationStream` is documented as the module whose role is to find the right segments to load, to orchestrate their requests and then to push them into the right `SegmentSink` (`src/core/segment_sinks`). In terms of responsibility and readability of the code it can make sense to put the `DownloadingQueue` in the `RepresentationStream`, but I always felt that it would be more at its place in the `fetchers` code for segments instead (in `src/core/fetchers/segment). The `fetchers` code for segments's role is to actually perform the segment requests, and do so in a transport-agnostic way (the transport-specific part being defined in `src/transports` which is directly used by the `fetchers` code). What I did here --------------- Basically, I moved the `DownloadingQueue` file into the `src/core/fetchers/segment` directory and renamed it `SegmentQueue` as it made more sense to me to remove the `Downloading` part from the name now that it is implied from its new location, and add the `Segment` part, which is sort of a convention in the `fetchers` to separate manifest-related code to segment-related code. I then made the `SegmentQueue` one of the main modules to be used by code external to the `fetchers` (here principally the `RepresentationStream`) instead of the `PrioritizedSegmentFetcher` like before. I also tried to simplify the whole `fetchers` API for segments so it can be straightforward to use from other modules without having to understand logic specific to that part. What's the relation with CMCD ----------------------------- As written at the start, this was done in relation with my work on CMCD, because that scheme can optionally provide to the CDN hints about what segment it is going to load next (with the previous segment's request). Previously, this would have meant an akward supplementary parameter somewhere asking for information on the next wanted segment for what's in the end very specific. By instead updating the `fetchers` (which is the module directly exploiting CMCD) so its API is already aware of the current queue of wanted segments (and not just the most urgent one like before), implementing "next segment hints" can be done without having to involve any other module.
peaBerberian
added a commit
that referenced
this pull request
Jun 22, 2024
While working on CMCD (#1461), I saw an opportunity for doing a refactoring that I initially postponed: moving what was called the `DownloadingQueue` from the `RepresentationStream` to the `fetchers` code. The modules involved -------------------- The `DownloadingQueue` is a class allowing to put the current list of wanted segments for a given media type (audio, video, text) into a FIFO queue so they can be requested from the most urgent (usually the one coming right next in the corresponding media buffer) to the least urgent (usually the ones consecutive to that segment, in chronological order) - through a simple interface. The `DownloadingQueue` does both the queueing (the list of wanted segments is given to it as input) and the requesting (by relying on an instanciated `SegmentFetcher` - defined in the `fetchers` code) as well as some optimizations such as requesting the initialization segment at the same time than the initially-wanted media segment when possible. Previously, that `DownloadingQueue` was part of the `RepresentationStream` (in the `src/core/stream/representation` directory). The `RepresentationStream` is documented as the module whose role is to find the right segments to load, to orchestrate their requests and then to push them into the right `SegmentSink` (`src/core/segment_sinks`). In terms of responsibility and readability of the code it can make sense to put the `DownloadingQueue` in the `RepresentationStream`, but I always felt that it would be more at its place in the `fetchers` code for segments instead (in `src/core/fetchers/segment). The `fetchers` code for segments's role is to actually perform the segment requests, and do so in a transport-agnostic way (the transport-specific part being defined in `src/transports` which is directly used by the `fetchers` code). What I did here --------------- Basically, I moved the `DownloadingQueue` file into the `src/core/fetchers/segment` directory and renamed it `SegmentQueue` as it made more sense to me to remove the `Downloading` part from the name now that it is implied from its new location, and add the `Segment` part, which is sort of a convention in the `fetchers` to separate manifest-related code to segment-related code. I then made the `SegmentQueue` one of the main modules to be used by code external to the `fetchers` (here principally the `RepresentationStream`) instead of the `PrioritizedSegmentFetcher` like before. I also tried to simplify the whole `fetchers` API for segments so it can be straightforward to use from other modules without having to understand logic specific to that part. What's the relation with CMCD ----------------------------- As written at the start, this was done in relation with my work on CMCD, because that scheme can optionally provide to the CDN hints about what segment it is going to load next (with the previous segment's request). Previously, this would have meant an akward supplementary parameter somewhere asking for information on the next wanted segment for what's in the end very specific. By instead updating the `fetchers` (which is the module directly exploiting CMCD) so its API is already aware of the current queue of wanted segments (and not just the most urgent one like before), implementing "next segment hints" can be done without having to involve any other module.
peaBerberian
added a commit
that referenced
this pull request
Jun 22, 2024
While working on CMCD (#1461), I saw an opportunity for doing a refactoring that I initially postponed: moving what was called the `DownloadingQueue` from the `RepresentationStream` to the `fetchers` code. The modules involved -------------------- The `DownloadingQueue` is a class allowing to put the current list of wanted segments for a given media type (audio, video, text) into a FIFO queue so they can be requested from the most urgent (usually the one coming right next in the corresponding media buffer) to the least urgent (usually the ones consecutive to that segment, in chronological order) - through a simple interface. The `DownloadingQueue` does both the queueing (the list of wanted segments is given to it as input) and the requesting (by relying on an instanciated `SegmentFetcher` - defined in the `fetchers` code) as well as some optimizations such as requesting the initialization segment at the same time than the initially-wanted media segment when possible. Previously, that `DownloadingQueue` was part of the `RepresentationStream` (in the `src/core/stream/representation` directory). The `RepresentationStream` is documented as the module whose role is to find the right segments to load, to orchestrate their requests and then to push them into the right `SegmentSink` (`src/core/segment_sinks`). In terms of responsibility and readability of the code it can make sense to put the `DownloadingQueue` in the `RepresentationStream`, but I always felt that it would be more at its place in the `fetchers` code for segments instead (in `src/core/fetchers/segment). The `fetchers` code for segments's role is to actually perform the segment requests, and do so in a transport-agnostic way (the transport-specific part being defined in `src/transports` which is directly used by the `fetchers` code). What I did here --------------- Basically, I moved the `DownloadingQueue` file into the `src/core/fetchers/segment` directory and renamed it `SegmentQueue` as it made more sense to me to remove the `Downloading` part from the name now that it is implied from its new location, and add the `Segment` part, which is sort of a convention in the `fetchers` to separate manifest-related code to segment-related code. I then made the `SegmentQueue` one of the main modules to be used by code external to the `fetchers` (here principally the `RepresentationStream`) instead of the `PrioritizedSegmentFetcher` like before. I also tried to simplify the whole `fetchers` API for segments so it can be straightforward to use from other modules without having to understand logic specific to that part. What's the relation with CMCD ----------------------------- As written at the start, this was done in relation with my work on CMCD, because that scheme can optionally provide to the CDN hints about what segment it is going to load next (with the previous segment's request). Previously, this would have meant an akward supplementary parameter somewhere asking for information on the next wanted segment for what's in the end very specific. By instead updating the `fetchers` (which is the module directly exploiting CMCD) so its API is already aware of the current queue of wanted segments (and not just the most urgent one like before), implementing "next segment hints" can be done without having to involve any other module.
peaBerberian
added a commit
that referenced
this pull request
Jun 24, 2024
While working on CMCD (#1461), I saw an opportunity for doing a refactoring that I initially postponed: moving what was called the `DownloadingQueue` from the `RepresentationStream` to the `fetchers` code. The modules involved -------------------- The `DownloadingQueue` is a class allowing to put the current list of wanted segments for a given media type (audio, video, text) into a FIFO queue so they can be requested from the most urgent (usually the one coming right next in the corresponding media buffer) to the least urgent (usually the ones consecutive to that segment, in chronological order) - through a simple interface. The `DownloadingQueue` does both the queueing (the list of wanted segments is given to it as input) and the requesting (by relying on an instanciated `SegmentFetcher` - defined in the `fetchers` code) as well as some optimizations such as requesting the initialization segment at the same time than the initially-wanted media segment when possible. Previously, that `DownloadingQueue` was part of the `RepresentationStream` (in the `src/core/stream/representation` directory). The `RepresentationStream` is documented as the module whose role is to find the right segments to load, to orchestrate their requests and then to push them into the right `SegmentSink` (`src/core/segment_sinks`). In terms of responsibility and readability of the code it can make sense to put the `DownloadingQueue` in the `RepresentationStream`, but I always felt that it would be more at its place in the `fetchers` code for segments instead (in `src/core/fetchers/segment). The `fetchers` code for segments's role is to actually perform the segment requests, and do so in a transport-agnostic way (the transport-specific part being defined in `src/transports` which is directly used by the `fetchers` code). What I did here --------------- Basically, I moved the `DownloadingQueue` file into the `src/core/fetchers/segment` directory and renamed it `SegmentQueue` as it made more sense to me to remove the `Downloading` part from the name now that it is implied from its new location, and add the `Segment` part, which is sort of a convention in the `fetchers` to separate manifest-related code to segment-related code. I then made the `SegmentQueue` one of the main modules to be used by code external to the `fetchers` (here principally the `RepresentationStream`) instead of the `PrioritizedSegmentFetcher` like before. I also tried to simplify the whole `fetchers` API for segments so it can be straightforward to use from other modules without having to understand logic specific to that part. What's the relation with CMCD ----------------------------- As written at the start, this was done in relation with my work on CMCD, because that scheme can optionally provide to the CDN hints about what segment it is going to load next (with the previous segment's request). Previously, this would have meant an akward supplementary parameter somewhere asking for information on the next wanted segment for what's in the end very specific. By instead updating the `fetchers` (which is the module directly exploiting CMCD) so its API is already aware of the current queue of wanted segments (and not just the most urgent one like before), implementing "next segment hints" can be done without having to involve any other module.
peaBerberian
added a commit
that referenced
this pull request
Jun 24, 2024
Previous iteration of CMCD [#1461] did not handle "nrr" nor... "nor" attributes. The former (nrr) gives a hint to the CDN of what the next byte range request going to be (if it's one) and the later (nor) indicates the relative URL based on the current one. Note that if it's the same segment, the relative URL is just the file name, as it seems logical for me - and as I didn't find an explicit mention of this frequent case in the CMCD specification. Even if the documentation for `nrr` indicates that an absent `nor` but a present `nrr` is assumed to mean the same segment.
peaBerberian
added a commit
that referenced
this pull request
Jun 24, 2024
Previous iteration of CMCD [#1461] did not handle "nrr" nor... "nor" attributes. The former (nrr) gives a hint to the CDN of what the next byte range request going to be (if it's one) and the later (nor) indicates the relative URL based on the current one. Note that if it's the same segment, the relative URL is just the file name, as it seems logical for me - and as I didn't find an explicit mention of this frequent case in the CMCD specification. Even if the documentation for `nrr` indicates that an absent `nor` but a present `nrr` is assumed to mean the same segment.
src/experimental/tools/VideoThumbnailLoader/video_thumbnail_loader.ts
Outdated
Show resolved
Hide resolved
Florent-Bouisset
approved these changes
Jun 28, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a work-in-progress and proposal for a CMCD integration inside the RxPlayer.
CMCD is for "Common Media Client Data", and it allows a media player to communicate metrics related to playback conditions and wanted segments to the CDN.
In turn, this can be then used on the CDN-side theoretically for things like monitoring (e.g. why and when users encounter rebuffering periods), QoE improvements (e.g. putting more priority on users with a low buffer health) and caching improvements thanks to the client hinting at what it's going to request next.
This communication can go through several means: here I implemented through the URL's query string by default or through headers (which might necessitate preflight requests because of CORS policies) optionally.
It will be opt-in and disabled by default.
Specifics:
The CMCD properties
norandnrr, used for segment pre-fetching be integrated is not yet relied on because it necessitates some refactoring firstFor now, CMCD is always integrated in all builds, as opposed to an optional feature to explicitly import
When in a multithreading mode, it is possible that the estimated current position that is relied on for several CMCD fields is actually behind the real current position, as we rely on the last communicated one there (we could alternatively estimate a more precize position by looking at adding the time difference from the point at which the object was constructed)