The state of streaming video support - HLS and DASH

[elemdeso]

There have been so many threads here over the years about the ins and outs of support for video, it would be great to have a summary of the current state of play, particularly support for HLS and DASH streaming.
I'm asking because years ago I made an iOS app for iPad to stream short video/audio (from AWS). It uses a basic stageVideo implementation to stream HLS encoded mp4 files (with m3u8 manifests). It's worked without problems right through to updates via the latest AIR SDK.
Now I'm trying to remake it for macOS. Everything is fine except the video playback, which hangs on Test and in a packaged/published app. The MP4 master file (H.264/AAC) however plays fine. What am I missing in the macOS app to enable native HLS playback?
Similarly, for Windows, which I haven't tried yet, what streaming media is supported?
Thanks.

I had a thought. I assumed that because the macOS app plays MP4 but not HLS, the failure must be lack of native HLS playback in macOS (Sonoma), which would be strange. But I played the iOS app on an M2 machine and the HLS stream played fine.
Does this mean that Air for iOS includes HLS support, and Air for Desktop does not?

[ajwfrost]

Hi

We don't have HLS (or DASH) support natively in AIR .. so when you say it works on iOS, I'm assuming you're not just passing in an m3u8 URL to a NetStream object?

There is a difference in the MP4/H.264 support in iOS vs the other platforms, it's a little more like StageVideo capabilities where it gets handed off to the platform. The classes used there are likely to support HLS but I don't think we would invoke them if using a URL that gives us a manifest rather than an MP4 file...

In terms of the plan .. our new multimedia framework would have the concept of a "source" component where currently we'd support files/URLs/blobs, but it would equally be possible to create an "HLS" or "DASH" source component that then passes the chunks of video in to the next stage. But this is still at a fairly immature stage at the moment.

On Android we enabled a mechanism for people to use the ExoPlayer component and display the outputs within an AIR display, so we could look at a similar option on iOS perhaps using the native AVPlayer and passing things through...

thanks

[elemdeso]

Hi Andrew, thanks so much for taking the time to explain so carefully. Except... I am just passing an m3u8 manifest URL into a NetStream object - and it works fine on iOS. I admit it's been a long time since I wrote the routine and I don't exactly remember how it all works, but here's a very cut down version:

var vidView:Rectangle = new Rectangle(32, 210, 1984, 1116);
var stageVideoAvail:Boolean;
var sv:StageVideo;
var nc:NetConnection = new NetConnection();
nc.connect(null);
var ns:NetStream = new NetStream(nc);
var netClient:Object = new Object();
ns.client = netClient;
function onAvail(e:StageVideoAvailabilityEvent):void {
	stageVideoAvail = (e.availability == StageVideoAvailability.AVAILABLE);
	initVideo();
}
stage.addEventListener(StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY, onAvail);
function initVideo():void {
	sv = stage.stageVideos[0];
	sv.addEventListener(StageVideoEvent.RENDER_STATE, onRender);
	sv.attachNetStream(ns);
}
function onRender(e:StageVideoEvent):void {
	sv.viewPort = vidView;
}
ns.play("https://www.example.com/example.m3u8");

You mention MP4 being handed off to the platform (presumably AVKit). That's what I assumed for HLS manifests as well, and why I'm puzzled it doesn't work similarly on MacOS.

[ajwfrost]

Oh yes - so for iOS, it checks the file extension and pushes the URL to the system component if it matches one of these:
.mov.m4v.mp4.f4v.3gp.3g2.m4a.m3u8
So that would be why this works :-)

On macOS, we don't do that, it's a different video mechanism. Not actually sure why that is! We can see whether it would be simple to do the same sort of thing here...

[elemdeso]

Ah ha! Andrew, you're a champion. Now it make sense. Porting to MacOS (Air for Desktop, I mean) would be great, and I'm sure useful for lots of people, being that everything Apple uses AVKit.

[ajwfrost]

On desktop, the current video set-up uses VDA (video decode acceleration framework) with OpenGL composition for StageVideo/VideoTexture. So actually changing that could be a bit of a challenge... but that (i.e. the pass-through to native objects) is the kind of use case we would look for with the newer air.media pipeline. So it might be simpler if we took this kind of approach (like had been bolted on for iOS) to display the AVPlayer instance (from AVFoundation; the AVKit stuff is a bit too new it seems..) and then the 'source' can just be a URL that is something the video renderer can handle directly..

thanks

[elemdeso]

Sounds like a plan...