Updates for automatic track selection for streams (ABR)

Hey everyone,

We're making some well needed changes to the automatic track selection within MistServer. Currently unless a track is masked from the specific process it will always default to the last added track. This decision worked pretty well when everything was single bitrate and the last added track would be whatever you added through an encoder, which you generally wouldn't unless it improved playback or usability.

Now however multibitrate is quite common and there's no guarantee that the last added track is the "best quality" to start on. Although that did bring up an interesting problem to solve, how does one determine the quality of a track through nothing but track information.

After all: A high resolution doesn't automatically mean a better picture, just like higher FPS or higher bitrate doesn't automatically mean the image/movement is more clear or crisp. When it really comes down to it, the only reliable source would be your own eyes. So while we haven't solved this issue, we think we've decided on something that is fairly "good" or at least better than picking the last added track.

When it comes to video MistServer now looks at:

• framerate (fps)
• resolution
• codec

Now you're probably thinking, why not bandwidth? Bandwidh is definitely an critical metric and we started with it included, however we quickly noticed that it was less of a reliable metric when combined with the others. Higher bandwidth does not automatically mean a "better" stream, this is especially true if you were to compare H264 video versus AV1. A 10mbps H264 could look pretty good, however a 3mbps AV1 would be of comparable quality. Now we were thinking of deciding what would be "normal" bandwidth per resolution, however when we really thought about it in the end most users will most likely have several resolutions, fps and codec qualities, but not the same resolution, fps and codec with just bandwidth being the difference. So in the end unless the quality difference is only in bandwidth we should be performing much better.

Additionally we did add some extra track selectors for audio, video, subtitles and metadata. You can now also use:

• source for grabbing the "source" track, do note that if you received a multibitrate stream all of them would qualify so you might need to add additional selectors
• last for grabbing the last track, effectively going back to the previous behaviour
• first for grabbing the first track

Expect these changes in the upcoming 3.11 release