Embedded Video Subtitle

[langaliamayank]

Media3 Exoplayer Not showing Embedded Video Subtitle for HLS(M3U8) Stream

https://flux1.csidigital.net/GPU4/hls_live/INSP_HD/playlist.m3u8
For getting render type = 5 - TRACK_TYPE_METADATA
mimetype - application/id3

I checked the above streaming URL in VLC player and The OPlayer.
We are getting subtitles but when I tried with Exo player media3 v1.4.1 not work.

DefaultTrackSelector.Parameters parameters = mTrackSelector.getParameters();
        DefaultTrackSelector.Parameters.Builder builder = parameters.buildUpon();
        final MappingTrackSelector.MappedTrackInfo mappedTrackInfo = mTrackSelector.getCurrentMappedTrackInfo();
        if (mappedTrackInfo == null) {
            return;
        }
        for (int i = 0; i < mappedTrackInfo.getRendererCount(); i++) {
            if (mappedTrackInfo.getRendererType(i) == C.TRACK_TYPE_TEXT) {
                builder
                        .clearOverridesOfType(i)
                        .setRendererDisabled(C.TRACK_TYPE_VIDEO, false)
                        .setSelectUndeterminedTextLanguage(true)
                .setPreferredTextRoleFlags(C.ROLE_FLAG_CAPTION);

                TrackGroupArray trackGroupArray = mappedTrackInfo.getTrackGroups(i);
                for (int trackIndex = 0; trackIndex < trackGroupArray.length; ++trackIndex) {
                    Format format = trackGroupArray.get(trackIndex).getFormat(0);
                    if (format.id != null && (format.sampleMimeType.equalsIgnoreCase("application/cea-608")
                            || format.sampleMimeType.equalsIgnoreCase("application/cea-708"))) {
                        builder.addOverride(new TrackSelectionOverride(trackGroupArray.get(trackIndex), 0));
                    } else {
                        builder.addOverride(new TrackSelectionOverride(trackGroupArray.get(trackIndex), 0));
                    }

                }


                mTrackSelector.setParameters(builder);
            }
        }
 
public int getTrackCountSubtitle() {
        if (mTrackSelector != null) {
            final MappingTrackSelector.MappedTrackInfo mappedTrackInfo = mTrackSelector.getCurrentMappedTrackInfo();
            if (mappedTrackInfo != null) {
                for (int rendererIndex = 0; rendererIndex < mappedTrackInfo.getRendererCount(); rendererIndex++) {
                    Logger.e("getTrackCountSubtitle" + mappedTrackInfo.getRendererType(rendererIndex));
                    if (mappedTrackInfo.getRendererType(rendererIndex) == C.TRACK_TYPE_TEXT) {
                        final TrackGroupArray trackGroups = mappedTrackInfo.getTrackGroups(rendererIndex);
                        return trackGroups.length;
                    }
                }
            }
        }
        return 0;
    }

getTrackCountSubtitle method return 0 because render type return 5 not 3.

[marcbaechinger]

Thanks for your question!

When I play this stream I see that there is an ID3 metadata track. That's something different to a text track though that has properties like a language to be available for proper selection.

I don't understand how the ID3 track should be turned into a text track available for selection tbh. I don't think this is supported by ExoPlayer. I also don't see any metadata being emitted by the stream.

Can you please point me to the spec that defines how subtitles are transported by ID3 metadata entries with HLS/ts? Or alternatively expand a bit on how you think this should work?

20:21:13.764 10124-10124 EventLogger        tracks [eventTime=1.47, mediaPos=25.96, window=0, period=0
20:21:13.764 10124-10124 EventLogger          group [
20:21:13.765 10124-10124 EventLogger            [X] Track:0, id=0, mimeType=video/avc, container=application/x-mpegURL, bitrate=730000, codecs=avc1.4d4029, res=480x270, fps=29.97, supported=YES
20:21:13.765 10124-10124 EventLogger            [X] Track:1, id=1, mimeType=video/avc, container=application/x-mpegURL, bitrate=1100000, codecs=avc1.4d4029, res=640x360, fps=29.97, supported=YES
20:21:13.765 10124-10124 EventLogger            [X] Track:2, id=2, mimeType=video/avc, container=application/x-mpegURL, bitrate=3000000, codecs=avc1.4d4029, res=960x540, fps=29.97, supported=YES
20:21:13.766 10124-10124 EventLogger            [X] Track:3, id=3, mimeType=video/avc, container=application/x-mpegURL, bitrate=5000000, codecs=avc1.640028, res=1920x1080, fps=29.97, supported=YES
20:21:13.766 10124-10124 EventLogger          ]
20:21:13.766 10124-10124 EventLogger          group [
20:21:13.766 10124-10124 EventLogger            [X] Track:0, id=0, mimeType=audio/mp4a-latm, container=application/x-mpegURL, codecs=mp4a.40.2, supported=YES
20:21:13.766 10124-10124 EventLogger          ]
20:21:13.766 10124-10124 EventLogger          group [
20:21:13.766 10124-10124 EventLogger            [X] Track:0, id=ID3, mimeType=application/id3, supported=YES
20:21:13.766 10124-10124 EventLogger          ]
20:21:13.766 10124-10124 EventLogger          Metadata [
20:21:13.766 10124-10124 EventLogger            HlsTrackMetadataEntry
20:21:13.766 10124-10124 EventLogger          ]
20:21:13.766 10124-10124 EventLogger        ]

[langaliamayank]

Yes, but it will work in VLC Player and The OPlayer. How are they managing subtitles and close captions?

Exo Player v2.16.0 was working fine but not working in Media3 v1.3.1.

Please help me to resolve this issue.

Thanks

[icbaker]

@langaliamayank Have you tried disabling HLS chunkless preparation?

See #1518 (comment)

[langaliamayank]

MediaItem mediaItem = new MediaItem.Builder().setUri(Uri.parse("https://flux1.csidigital.net/GPU4/hls_live/INSP_HD/playlist.m3u8"))
.build();
exoPlayer.setMediaItem(mediaItem);
exoPlayer.prepare();

I'm using media3 library. There is no option to disable hls chunkless preparation.

[icbaker]

I'm using media3 library. There is no option to disable hls chunkless preparation.

Yes there is

[langaliamayank]

After adding above solution it will work for HLS streaming but what about DASH ? There is no option available for dash streaming.

[icbaker]

ExoPlayer's DASH module has no concept of chunkfull vs chunkless preparation, and doesn't support MPEG-TS chunks. Do you have an example DASH stream you are having issues with? It is unlikely to be the same root cause as you're seeing here.