Skip to content

Conversation

CastagnaIT
Copy link
Collaborator

@CastagnaIT CastagnaIT commented Sep 24, 2025

Description

Currently ISA support to read the TS streams containing the audio stream into the same video stream

in other words, these HLS have a single video playlist stream in .ts, and the same video segments contains also the audio data (similar to local files)

in this case only a single TSSampleReader for video track is created, at each read the TSSampleReader is automatically able to distinguish between video and audio data and change automagically also the stream ID as needed
therefore, its internal state changes continuously, between video and audio and provide data in PTS order at each read

with MP4 this is different
the FragmentedSampleReader use AP4_LinearReader to read data, it that cannot switch automagically between audio/video data, it is strongly linked to AP4_Track's usually one AP4_Track for video, one for audio

therefore we must use/share the same AP4_LinearReader for both a/v tracks,
(otherwise create multiple AP4_LinearReader that point to the same AdaptiveStream (AP4_ByteStream) will cause a bad behaviors on data to be read)

since most of code is coupled to AP4_Track it is difficult to reuse the same FragmentedSampleReader for both tracks, because we need read packet data from both tracks, also initialize the codechandler and extract extradata

I thought about creating a FragmentedSampleReader clone by changing the AP4_Track (imo maybe the "easy" way)
where share the same AP4_LinearReader of video track,
then inject the sample reader to the CStream by setting the appropriate StreamId

Currently, this is a POC that demonstrates that both audio and video work,
OFC it is not complete and has various problems

Motivation and context

POC to solve
#1909
#343

also #1005 but require to include changes to DASH parser

How has this been tested?

tested with dailymotion videos such as
https://www.dailymotion.com/video/x9qgg9c
the m3u8 link expire, can be found also other videos
just take a look that playlist is with mp4 and not TS

Screenshots (if appropriate):

Types of change

  • Bug fix (non-breaking change which fixes an issue)
  • Clean up (non-breaking change which removes non-working, unmaintained functionality)
  • Improvement (non-breaking change which improves existing functionality)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that will cause existing functionality to change)
  • Cosmetic change (non-breaking change that doesn't touch code)
  • None of the above (please explain below)

Checklist:

  • I have read the Contributing document
  • My code follows the Code Guidelines of this project
  • My change requires a change to the Wiki documentation
  • I have updated the documentation accordingly

@CastagnaIT CastagnaIT added Type: Fix non-breaking change which fixes an issue WIP RFC PR submitted for gathering feedback Type: Improvement non-breaking change which improves existing functionality labels Sep 24, 2025
@CastagnaIT CastagnaIT force-pushed the fmp4_included_tracks branch from 88acbe4 to 51ab893 Compare October 13, 2025 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

RFC PR submitted for gathering feedback Type: Fix non-breaking change which fixes an issue Type: Improvement non-breaking change which improves existing functionality WIP

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant