Setting up a video on the web can be as simple as adding a video tag with the desired parameters. In the code snippet below, myvideo.mp4 will autoplay and loop forever (although it could be paused with the controls). It simply tells the browser to download the video and play it back:

<video src=”myvideo.mp4” autoplay muted loop playsinline width="360" controls ></video>

This is limiting. There is only one video file, which will be downloaded to every device, no matter the network type or screen size. Depending on the browser, the entire file might be downloaded, despite only a fraction of the video being consumed by the end user.

The solution to this is to use video streaming. Video streams are encoded in different sizes and bitrates, allowing content to be optimised for the device, and downloaded in segments of video, so only the video consumed (and a bit downloaded ahead of time for buffering) is downloaded across the network.

In previous posts, I’ve written about how HLS video streaming works, and how to optimise your manifest to ensure fast video startup. In brief, when multiple versions of a video are created, they are all referenced in a manifest file. This manifest file is delivered to a player in the browser, and the player makes decisions on which version should be played. The player is a required entity, but until now, is just a black box. How do HLS video players work? And what can we do to optimise them?

HLS video players

In this post, I’ll look at three popular JavaScript players, hls.js, video.js and Shaka player, and compare them to YouTube. I’ll configure them to stream, and then investigate how they interact with a stream in order to play back on the web. As you might expect, each player behaves slightly differently, and understanding the differences can help you decide which player to utilise.

The setup

In order to minimise the number of variables, I’ve hosted all of the JavaScript on the same server as the website (using the latest version available as of 10 April 2019). The HLS video is hosted at Cloudinary (and this hosted video is used in all tests except for YouTube). If you are curious, the repository is on GitHub, and each example is live on the corresponding GitHub page.

JavaScript file sizes and processing times

To playback streaming video, the JavaScript player must be downloaded and prepared in the browser. One key metric in video playback is the time to start the video, so getting the player file downloaded and parsed quickly is key. In my tests, I kept the player file local to prevent another DNS lookup, and also used minified versions of the default player (NB: Some players have ‘light’ versions that I did not investigate).

	Minified JavaScript size (KB)
hls.js	71.1
Shaka.js (+ mux.js)	140
video.js	133
YouTube	771 (includes fonts/CSS/picture)

Hls.js is half the size of the other two streaming players. Once the JS is downloaded, it must be parsed before the first video request can be made. In WebPageTest, this is seen as a delay between requests (the pink lines show CPU processing). Using a Motorola G4, we see a gap in content download from 1.7s to 2.3s as the JavaScript is being parsed:

Shaka and video.js are double the size of hls.js, and the table below shows that the parsing time is twice as long. YouTube is an interesting case. The WebPageTest runs did not autoplay, but we can see how much content is downloaded before video playback:

Since the YouTube playback performs differently in WebPageTest, I measured the time until the video playback begins using Chrome DevTools, limiting the network to a Fast 3G connection:

	Time after JS download before manifest request (ms) (WebPageTest: Moto G4)	Time to video start (Chrome DevTools, 3G Fast connection)in seconds
hls.js	250	5.83
Shaka Player (+mux.js)	565	7.15
video.js	575	9.54
YouTube	1300 (for moreYouTube files)	10.16

Compressing JavaScript files

In order to quickly deliver video content to the browser, we need to deliver the player quickly. Using the March 2019 HTTP Archive, we can examine how many of the players delivered were minified (based on the filename) and used gzip compression. (NB: It’s possible that not all files named video.js actually refer to the video.js library).

Between 3.4% and 16.4% of the files are not sent with gzip, increasing the KB usage, and download time. Looking at the 95th percentile of gzipped and not gzipped, we see savings from 350ms to 10923 ms in the downloads.

File	Total File Count	Not gzipped	DL time delta: 95th percentile (ms)
hls.js	280	33 (11.8%)	366
hls.min.js	1121	38 (3..4%)	5829
Shaka-Player	23	3 (13%)	2749
video.js	11692	1737 (14.9%)	10923
video.min.js	4163	684 (16.4%)	2946

Preconnecting the video stream

In my very simple test, one of the largest delays was in establishing the connection to Cloudinary to download the video file. Since the video streams are on a third party, there is an additional DNS lookup/connection/HTTPS handshake. Without any optimisation, this occurs serially, after the JavaSCript processing time. By using preconnect, we can make the connection to Cloudinary in parallel with the JavaScript, removing these timings from the critical path (Andy Davies has an excellent post on this).

Hls.js without preconnect:

Hls.js with preconnect:

In the case of hls.js, this moved the manifest file download to ~ 1.75s from ~2.2s (and pushed in front of the favicon), saving 450ms in the critical path.

Streams are responsive

One major reason for switching your videos to streams is to enable the responsiveness of videos. For images, media queries allow developers to serve resize images to different screens sizes. However, media queries are not supported for video (they were removed in Chrome 34), so using responsive video with static files would require some JavaScript to choose the correct video version. Luckily, responsiveness is built right into streaming video!

How responsive video works

In the manifest of the streaming video in my tests, there are six different streams, each with a different dimension (indicated in the green text “RESOLUTION”):

Note also that the Bandwidth for each stream is listed in bytes (the first stream in the list above has a bitrate of 341 Kbps).

In a previous post, I discussed how starting with the lowest bitrate in the list will speed up the startup of our video as the lowest bitrate will download faster. The expectation for streaming video is that the player will resize the stream to fit the window size on the screen — downloading just the right amount of video for the device. In practice, this only happens with one of the players tested:

When Video.js streams to a window set to 360 px wide, the player settles on the 480w x 270h stream, the size just larger than the assigned number of pixels on the screen. This results in 24 requests and 2.7 MB transferred during the video playback (down from 8.4 MB on a 720px window).

Not responsive by default

When the same test is run with Shaka, hls.js and YouTube, we see non-responsive results: all of these players optimise for the available bandwidth, but not for the defined video window, and the same video tonnage is downloaded for the 360px window as a 720px window. Even though it’s not responsive out of the box, we can add options to make these players responsive.

Making hls.js responsive

There is an option in hls.js called ‘capLevelToPlayerSize” that is set to false by default. Implementing this parameter results in the 960×540 video to be used — technically two sizes larger than the cap, but clocking in at a much reduced 5.1 MB of data transferred.

In the HTTP Archive from 1 March 2019, there are 74,000 responses with the terms “hls.js” or “hls.min.js” and the term “m3u8”. Of these, only 2871 (3.8%) also include ‘capLevelToPlayerSize,’ indicating that responsive video with hls.js is rarely implemented.

Making Shaka “responsive”

Shaka Player does not have a responsive option per se, but you can limit the maximum dimensions of the stream:

player.configure({
		abr: {
			restrictions: {
				maxWidth: 360}
			}
	});

This results in the 320px wide video playing in the 360px wide window — so lower quality than ideal, but using JavaScript to ensure the ‘next largest’ stream plays can solve this.

Making video.js responsive

In my tests the video.js example optimises for the video’s width automatically, but there is a parameter that can be added to ensure that the video plays back responsively:

var player = videojs('my_video_1', {	

			responsive: true

	});

When this parameter is added, there is no change from default usage in Chrome, but it probably is a good idea to add this, just in case.

Responsiveness of players to the video size on the screen:

	Width 720px	Width 360px	Width 360px(responsive option)
hls.js	13.9 MB	13.9 MB	4.1 MB
Shaka Player	15.3 MB	15.3 MB	1.7 MB
video.js	8.4 MB	2.7 MB	2.7 MB
YouTube	18.0 MB	18.0 MB	N/A

 
Responsiveness summary

Video.js is the only player to resize the video to the dimensions of the video window. Both Shaka and hls.js have options to make the video playback responsive, but hls.js still uses video two sizes larger than the window on the screen.

Bitrate adaptation to network changes

The other advantage of streaming video is that if there are changes in network quality, the player can adapt the stream to continue video playback. If the network becomes faster, a higher quality version can be displayed, and if the network gets slower or more congested, the player can adjust the quality downwards.

In order to find out how these four players react to changes in video quality, I performed the following test in Chrome DevTools:

1. Create a custom network profile “LTE” of 12 Mbps up/down, 70 ms latency (copied from WebPageTest)
2. Initiate stream on LTE, when the buffer is ~50% full (as judged from the playback window), lower the network speed to Fast 3G profile (1.6 MBPS down, 768 KBPS up, 150 ms latency)
3. Chart the DevTools result with the following annotations:
4. 1. Green highlight = LTE connection
   2. Yellow highlight = Fast 3G connection
   3. Purple bar = video bitrate change
   4. Red box = stall

The streams all show similar patterns:

1. Once the network has been throttled to a lower bitrate, the player continues downloading a segment at the high bitrate, and this takes longer. This is seen as a longer horizontal line in the DevTools waterfall.
2. The player then switches to a lower bitrate. Some players do a large drop to low quality immediately, while others make several jumps. (This may occur as the player was able to accurately determine the new bitrate more/less accurately from the real time download speed). These are marked with purple lines.
3. If the player does not jump fast enough…. it stalls!

If you look at the DevTools screenshots carefully, you may be able to see the bitrates being requested by the players in the URL.

Hls.js: The initial stream is 1920px wide, drops to 1280px and the 480px in two steps. No stalls were seen in three tests.

Video.js: The video begins streaming at 960px wide, and drops to 320px wide after the network change. No stalls seen in three tests.

Shaka Player: The stream begins at 1920px wide, and, after the network change, drops to 480px and then 320px. In all three tests, there was a stall before the switch could occur.

YouTube: I was unable to determine what bitrates are being used by YouTube. However, there is a 404 response in the data, and the  URL reads (in part): “videoplayback?sparams=altags%…..&fallbackcount=1….”, which indicates a bitrate change did occur (and the 404 indicates that perhaps YouTube is not accurately recording the fallback’s occurrence). No stalls were observed.

Summary of adaptive bitrate on network change

Hls.js, video.js and YouTube all successfully adapted to the bitrate change and continued displaying the video without any stalls. The Shaka Player always stalled on the drop in network bandwidth. What is interesting is that while Shaka and hls.js both request similarly sized segments, the hls.js appears to be more responsive to network changes — perhaps due to buffer size.

Looping video

When a video loops, I expect that the player will use cached video content to replay the video, rather than download the video files again. The video in my test was 50 seconds long, so my expectation was that there should be no requests after about a minute of the page being opened.

Interestingly, there is a wide variation on how the players handle looping the video:

	Single playback(MB)	10 minutes looping(MB)
hls.js	13.9 MB	13.9 MB
Shaka	15.3 MB	200 MB
video.js	8.4 MB	54 MB

With hls.js, no further requests are made after 15s of the page being opened. This is great from a data perspective. From a quality perspective, the initial segment is downloaded at 320px wide (and low quality), and the remaining are 1920px wide. On looping, the first several seconds of the video are always pixelated (using the low quality segment already in memory).

Shaka Player does not appear to cache any of the video in memory, and just downloads the entire video on each playback.

Video.js seems to download the first couple of segments on each loop, and then uses the remainder from the cache. This seems to fix the ‘low quality at loop start’ we see in hls.js, but does not use as much data as the Shaka Player.

Summary

In examining three popular video players used on the web today, along with YouTube, I found a wide range of behaviours.

1. Many video streaming implementations are not performing basic JavaScript compression or minification to deliver the players quickly.
2. JavaScript parsing adds a delay of 250 to 500ms to the autoplaying of videos (fastest video delivery was hls.js, which is also the smallest JavaScript file).
3. Only video.js was responsive (meaning video streams optimised to the size of the player were delivered) ‘out of the box’. Hls.js and Shaka have optional parameters to enable responsiveness, but they are disabled by default (and, based on HTTP Archive queries, are not widely implemented).
4. The hls.js responsive attribute did not use the ‘next biggest’ video size. It chose a video size two sizes larger, resulting in slightly more data usage.
5. When network quality dropped from LTE to Fast 3G, no stalls were seen in hls.js, video.js or YouTube, but were observed in Shaka.
6. Looping videos are cached 100% with hls.js, to a high degree with video.js, and not cached at all with Shaka.

In my (very) limited test of one video, hls.js appears to be the fastest video player for autoplay content. The ‘capLevelToPlayerSize’ option makes the player very nearly responsive, and it caches video on looping, preventing additional data usage.

However, it’s important to test your streams with various implementations to ensure that your streams are optimised to play back for your customers.