Beginning HTML5 Media. Make the most of the new video and audio standards for the Web (2015)

CHAPTER 2

image

Using and Manipulating HTML5 Video and Audio Elements

With the video files encoded we can now turn our attention to the HTML5 markup that plays and controls the video files. In this chapter we will thoroughly review the <video> and <audio> elements introduced in HTML5. We will also be reviewing the various user interfaces used by the modern browsers because they do change from browser to browser. As we move through the chapter you can find all of the examples presented at http://html5videoguide.net.

One final note: HTML5 video continues to be a work in progress. If you discover something isn’t quite working as you expect, you really should check the actual specification for any updates that have been added. The HTML5.0 specification can be found atwww.w3c.org/TR/html5/. The next version is being developed with new features at www.whatwg.org/specs/web-apps/current-work/multipage/.

Image Note  The authors would like to thank the School of Media Studies at Humber College in Toronto for permission to use the Vultures video clip presented in this chapter. We would also like to thank Joseph Labrecque for permission to use the audio clip used in the audio portion of this chapter.

Before we get going, let’s keep in mind that the modern browsers, as shown in Figure 2-1, only support specific codecs. This is important for you to know because there will be occasions where you look at a blank screen wondering why the video is missing. The odds are almost 100% the browser doesn’t support the codec or file format used.

9781484204610_Fig02-01

Figure 2-1. Browser support

As is so typical with the Web, standards don’t remain static. On the horizon are two new codecs—VP9 and H.265—specifically designed to exist in a world slowly moving toward 4K or ultra HD (high definition) video. H.265 is being designed to be a better compressor than H.264 rather than a compressor designed to squash more data into a small space with the inevitable loss of quality.

Another on the horizon is VP9. Unlike H.265, VP9 is open source and royalty free. Developed by Google as the successor to VP8, this also explains why Google has quietly added VP9 support into Chrome and YouTube. Firefox has also added VP9 support.

Though the new codecs may seem quite different, VP9 and H.265 have something in common: both halve the file size of 720p and 1,080p content, making it far easier to download or stream HD video over relatively slow connections.

The <video> element

HTML5 introduced the <video> element, which allows you to place your video content in a web page without the use of complicated code or <object> and <embed> tags. This element can be placed directly on a page or within a CSS-driven <div> on the page. Throughout this chapter we will be demonstrating the use of the various properties and attributes that apply to HTML 5 video. The markup template used throughout this chapter is really quite basic, as shown in Listing 2-1. Please note that the listing numbers relate to the example files that we have published onhttp://html5videoguide.net/ so you can more easily locate them.

Listing 2-1. A Bare Video Element

<body>
<div class=".container">

    <h1>Beginning HTML5 Media</h1>
    <h2>The basic code:</h2>
    <p>Here is an example of the Video element:</p>

    <video></video>

</div>
</body>

The video “magic” all happens between the <video></video> tags. As it stands right now, you will see absolutely nothing because there is no video content being pointed to.

Fallback Content

You may have noticed the <video> element has an opening and closing tag. There are two reasons for this.

First, other elements can be introduced as children of the <video> element—in particular the <source> and <track> elements. We will introduce you to the <track> element in Chapter 4, which deals with accessibility.

Second, anything found inside the <video> element that is not inside a specific child of the <video> element is regarded as “fallback content.” By that we mean if the browser doesn’t support an element, it will ignore it and, instead, display its contents—in this case the content in the audio and video element. It could be something as simple as the message shown in the code block in Listing 2-2.

Listing 2-2. A Video Element with Fallback Text

<video src = "Vultures.mp4">
        Your browser doesn’t support the HTML5 video element.
</video>

If this were to be loaded into the old Internet Explorer (IE) 6 browser the user would see that message in the block set aside for the video. In fact, it is a best practice to always include fallback content in any page containing HTML5 video.

You can add any HTML markup inside the <video> element including, as we said, <object> and <embed> elements. Thus, for example, you could provide a fallback that displays a Flash Video file that uses the mp4 or flv format, always assuming that a legacy browser without media element support will at least have Adobe Flash support.

Such video plug-ins will not support the JavaScript API (Application Programming Interface) of the HTML5 <video> element, but you can get JavaScript libraries that emulate some of the JavaScript API functionality and can be used to provide fallback under a variety of conditions. Example libraries, two of which we explore in the next chapter, are

·        mwEmbed: https://github.com/kaltura/mwEmbed

·        Video For Everybody: http://camendesign.com/code/video_for_everybody

·        Sublime Video: www.sublimevideo.net/

·        VideoJS: www.videojs.com/

·        JW Player: www.jwplayer.com/

Also keep in mind that even though you may be using a modern browser, if it does not support ogv, for example, it will not display the fallback content. You have to use JavaScript to catch this error and we’ll get into how to catch this load error in Chapter 3. This really is relevant only if you intend to use a single-media format and want to catch errors for browsers that don’t support your choice.

So much for introducing the <video> element; let’s move on to a review of all the content attributes of the <video> element to explore just how robust the <video> element has become.

@src

At its most basic form, all the <video> element needs is to be pointed to a video file and it takes over from there. Here, Listing 2-3, shows the code.

Listing 2-3. A Video Element with an @src Attribute

<video src="video/Vultures.mp4"></video>

By simply adding the @src attribute and pointing it to a folder on the site containing the Vultures .mp4 file, we can include a video file on a page. Figure 2-2 shows the result.

9781484204610_Fig02-02

Figure 2-2. A <video> element with only an @src attribute

As you may have noticed, the video looks like a simple image. It won’t play and only the first frame in the video is displayed—thus the look of an image. Use of the <video> element in this manner only makes sense if the video is to be controlled with JavaScript or your intention is to have the video start playing immediately after loading.

@autoplay

To have the video start automatically you only need to add the @autoplay attribute. Without being set to autoplay, a browser will only download enough bytes from the video to decide whether or not it is able to decode the video and the header. When you talk about HTML5 video, the header really is the video file’s metadata, containing information such as the resolution and framerate which we talked about in the last chapter.

When you supply the @autoplay attribute, the video will request more audio and video data after setting up the decode pipeline, buffering the data, and starting playback when sufficient data has been provided and decoded. This explains why some videos take a while to load and others start almost immediately. The browser is making sure it can play the video at the given data rate without draining the buffer.

When you get right down to it, adding an @autoplay attribute with the @src attribute is the bare minimum needed to play a video. The code, shown in Listing 2-4, would be as follows:

Listing 2-4. A Video Element with @autoplay

<video src="video/Vultures.webm" autoplay></video>

The @autoplay attribute is a “Boolean” attribute, meaning it only has two possible values: True or False, where “True” means that the attribute exists and “False” means it doesn’t. By adding @autoplay to the markup, the True value is enabled. With @autoplay enabled, a video will start playing immediately (see Figure 2-3) and stop at the end of the video. Be careful with this one: using @autoplay="false" also enables the attribute through its sheer existence. If you don’t want @autoplay, you have to remove it. Give it any value at all and it is enabled.

9781484204610_Fig02-03

Figure 2-3. A webm file with @autoplay enabled

The other thing to be aware of is that if the download speed of the video is not fast enough to provide smooth playback or the browser’s decoding speed is too slow, the video will stop and start as the buffers are refilled before continuing playback.

Finally, using @autoplay is not regarded as an industry best practice. If you have ever arrived at a site where an annoying video starts playing, you’ll understand this. Use this attribute carefully. One place where it can be used freely is in situations where the video is used as a full-page background video without audio and is more of a design element than media.

@muted

It is a best practice to leave the choice of whether or not to play the audio track in a video to the user. However, there will also be occasions where you simply don’t need the audio to play. This is accomplished through the @muted attribute as shown in Listing 2-5.

Listing 2-5. A Video Element with @muted

<video src="video/Vultures.webm" muted autoplay></video>

@muted, like @autoplay, is a Boolean attribute. If muted is present, the value is true and the audio track is turned off. If it isn’t, the audio track plays.

If you want to give the user control of the audio, use either the @controls attribute or the JavaScript API.

@loop

The @loop attribute enables a video to automatically restart after it has finished playing. Obviously, this attribute turns the video into a continuous loop. Listing 2-6 shows the code,

Listing 2-6. A Video Element with @loop

<video src="video/Vultures.webm" autoplay loop></video>

@loop, like @autoplay, is a Boolean attribute, which will play and replay the video continuously until the user either leaves the page or quits the browser. If the video is more of a design element (e.g., full-screen background video) or is relatively short, then feel free to use it. Otherwise this attribute should be used sparingly.

If you want to put a limit on how many times the video loops, you will need to use the JavaScript API.

@poster

When a video loads, the browser typically displays the first frame in the video. Sometimes this first frame has color bars or is all black or all white and is not a good representative for your video. HTML5 allows you to define an image, usually picked from further into the video, as the poster image for the video. You can see this in Figure 2-4. We have gone a few minutes into the video, extracted a picture of a baby vulture, and saved it as a .jpg image for use as the poster frame. When we add this picture in the @poster attribute, Figure 2-4 is the image the user will see when he loads the video.

9781484204610_Fig02-04

Figure 2-4. The screenshot to be used as the poster frame

Poster frames are especially useful for videos that will take some time to load and play back. This way the user isn’t staring at a blank area of a page. The neat thing about poster frames is they don’t have to be taken from the video. Provided the image has the same physical dimensions as the video, it could be any .jpg or .png image you create, including a picture with a title text for the video. Some web sites even choose to use a looped animated .gif to display a brief video summary.

Listing 2-7 shows the code that loads the BabyVulture.jpg image as the poster frame.

Listing 2-7. A Video Element with @poster

<video src="video/Vultures.mp4" poster="img/BabyVulture.jpg"></video>

There are some differences in the way that the various browsers work with a poster frame. Firefox, Chrome, Opera, and Safari will display the poster (see Figure 2-5) instead of the video and pause if there is no @autoplay attribute. IE is a bit more problematic. IE will display the poster while it is setting up the decoding pipeline, but, as soon as the pipeline is established, it will display a black frame and then transition to the first frame of the video.

9781484204610_Fig02-05

Figure 2-5. The .jpg image used as a poster frame

If you use the @autoplay attribute, the poster image will only appear briefly while the video’s metadata is loaded and before playback starts. A good rule of thumb is not to use the @poster attribute in conjunction with @autoplay.

@width,@height

To this point in the book we have been displaying the same two videos in the browser and you may have noticed the physical dimensions of the video mirror those of the original source file. How do browsers do that?

Essentially the dimensions are calculated from the first picture of the video resource, which could be obtained from the @poster attribute or the video itself. In the case of the HK_Traffic video used in this example the dimensions are 1,066 x 600 (this is the video’s intrinsic size).

If there are no poster or video dimensions available—video load errors or no @poster attribute—the video display area or “viewport” is displayed at 300 x 150 (the minimum display) or at its intrinsic size.

If the poster dimensions and the video dimensions are different, the video will initially display at the poster dimensions and later adapt to the video dimensions.

What you can gather from this is that there is a lot of scaling going on by default. This can actually create a performance bottleneck in the browsers and a disruptive display when the viewport suddenly changes size between a differently scaled poster image and the video. To avoid this, we recommend the use of the @width and @height attributes in the <video> element—or even better, the use of width and height CSS attributes.

What values should be used? The @width and @height attributes can be expressed as either pixels (px) or percentages (%). The only browser that won’t accept a percentage value is IE9, which interprets the percentage as a px value (subsequently fixed in IE10). In a responsive web environment, where percentages reign supreme, our suggestion is to use CSS for percentage.

Listing 2-8 shows the code that uses the @width and @height attributes.

Listing 2-8. A Video Element with @width and @height Percentages

<video src="video/HK_Traffic.mp4" width="50%" height="50%"></video>

If we were to use numbers instead of percentages the code line would be as shown in Listing 2-9.

Listing 2-9. A Video Element with @width and @height in Pixels

<video src="video/HK_Traffic.mp4" width="533px" height="300px"></video>

Though you can use px, a more common application is to simply forget the px and use numbers instead. Listing 2-10 shows an example.

Listing 2-10. A Video Element with @width and @height with No Measurement Units

<video src="video/HK_Traffic.mp4" width="533" height="300"></video>

The result, as shown in Figure 2-6, is a video scaled to one-half of its size in the viewport.

9781484204610_Fig02-06

Figure 2-6. The video is scaled to 50% of its original size

Naturally, there are issues and the bulk of them relate to the inclusion of percentages. Let’s look a little deeper. When a percentage value is used for both the poster and the video, Firefox will scale both the video and the poster to the value required. Not Safari and Chrome. They scale to the value of the height of the poster image.

Sizing by giving fixed pixel values doesn’t come without its “issues” because CSS pixels, not screen pixels, are used to define pixels. A CSS pixel is a unit of measurement that establishes an optical standard for the length of a pixel and is totally independent of the size of the screen of a device. In the W3C standard for CSS3, this unit is described as roughly 1/96th of an inch (before CSS3, a pixel was defined as one device pixel on a standard 96 dpi monitor positioned roughly arm’s length from the observer, which is now called a reference pixel (seewww.w3.org/TR/css3-values/#absolute-lengths). This explains why content on devices is sometimes rather fuzzy. The CSS pixel, in this case, may just be larger than the device pixel on the screen.

Another pain point you may encounter with the @width and @height attributes is the aspect ratio. If a video with an aspect ratio of 16:9 (1280 x 720) is reduced to a 4:3 ratio (640 x 480) the video will be placed in a letterbox or pillarbox. Note that IE9 uses black bars instead of the transparent ones used by the other browsers.

@controls

The ease of adding video to web pages has made the use of video somewhat ubiquitous. Along with ubiquity comes responsibility. To this point with @autoplay, the video starts playing and plays right through to the end. This is not a best practice. The user should have the ability to choose when to play the video.... not you. Which brings us to the most useful attribute in the arsenal: @controls.

Like @autoplay this one is a Boolean attribute. If specified, controls are displayed when the video is loaded. During playback, the controls are displayed when the user mouses over the video and hidden otherwise.

Listing 2-11 shows the code that uses the @controls attribute.

Listing 2-11. A Video Element with @controls

<video src="video/HK_Traffic.mp4" controls></video>

The result, shown in Figure 2-7, is that the video now sports a spiffy control bar at the bottom and will not play until the user clicks the Play button.

9781484204610_Fig02-07

Figure 2-7. The @controls attribute, shown here in Chrome, adds user controls to a video

@preload

The final attribute we need to look at is @preload.

@preload replaces an earlier attribute called @autobuffer, which was a Boolean attribute and was thus unable to distinguish between several different buffering requirements of users. This is why the @preload attribute was introduced. It allows web developers to give the browser more detailed information about what they expect regarding the user’s buffering needs.

The @preload attribute is an attribute that you will not ordinarily want to use unless you have very specific needs. Thus, the following paragraphs are only meant for advanced users.

When a web browser encounters a <video> element, it needs to decide what to do with the resource that it links to.

If the <video> element is set to @autoplay, then the browser needs to start downloading the video resource, set up the video decoding pipeline, start decoding audio and video frames, and start displaying the decoded audio and video in sync. Typically, the browser will start displaying audio and video even before the full resource has been downloaded, since a video resource is inevitably large and will take a long time to download. Thus, as the web browser is displaying the decoded video, it can, in parallel, continue downloading the remainder of the video resource, decode those frames, buffer them for playback, and display them at the right display time. This approach is called progressive download.

In contrast, if no @autoplay attribute is set on <video> and no @poster image is given, the browser will display only the first frame of the video resource. It has no need to immediately start a progressive download without even knowing whether the user will start the video playback. Thus, the browser only has to download the video properties and metadata required to set up the decoding pipeline, decode the first video image, and display it. It will then stop downloading the video resource in order not to use up users’ bandwidth with data that they may not want to consume. The metadata section of a video resource typically consists of no more than several kilobytes.

A further bandwidth optimization is possible if the <video> element actually has a @poster attribute. In this case, the browser may not even bother to start downloading any video resource data and may just display the @poster image. Note that, in this situation, the browser is in an information-poor state: it has not been able to discover any metadata about the video resource. In particular, it has not been able to determine the duration of the video, or potentially even whether it is able to decode the resource. Therefore, most browsers on laptop or desktop devices will still download the metadata and the first frame of the video, while on mobile devices, browsers more typically avoid this extra bandwidth use.

As a web developer, you may be in a better position than the web browser to decide what bandwidth use may be acceptable to your users. This decision is also an issue because a delayed download of video data will also cause a delay in playback. Maybe web developers do not want to make their users wait for the decoding pipeline to be set up.

Thus, the @preload attribute gives you the explicit means to control the download behavior of the web browser on <video> elements.

The @preload attribute can take on the values of “none,” “metadata,” or “auto.”

Let’s start with the “none” parameter shown in Listing 2-12.

Listing 2-12. A Video Element with @preload “None”

<video src="video/HK_Traffic.ogv" poster="img/Traffic.jpg" preload="none" controls></video>

You would choose “none” in a situation where you do not expect the user to actually play back the media resource and want to minimize bandwidth use. A typical example is a web page with many video elements—something like a video gallery—where every video element has a@poster image and the browser does not have to decode the first video frame to represent the video resource. In the case of a video gallery, the probability that a user chooses to play back all videos is fairly small. Thus, it is a best practice to set the @preload attribute to “none” in such a situation and avoid bandwidth wasting, but accept a delay when a video is actually selected for playback. You also accept that some metadata is not actually available for the video and cannot be displayed by the browser (e.g., the duration of the video).

Another choice is to preload the metadata, as shown in Listing 2-13.

Listing 2-13. A Video Element with @preload “Metadata”

<video src="video/HK_Traffic.mp4" poster="img/Traffic.jpg" preload="metadata" controls></video>

You will choose “metadata” in a situation where you need the metadata and possibly the first video frame but do not want the browser to start a progressive download. This could be, for example, a video gallery situation, particularly without poster attributes. In this case, you may want to choose “none” if you are delivering your web page to a mobile device or through a low-bandwidth connection but choose “metadata” on high-bandwidth connections. Also, you may want to choose “metadata” if you are returning to a page with a single video that a user has already visited previously, since you might not expect the user to view the video again, but you do want the metadata and first frame to be displayed. The default preload mode is “metadata”.

Listing 2-14 shows the final parameter, “auto.”

Listing 2-14. A Video Element with @preload “Auto”

<video src="video/HK_Traffic.webm" poster="img/Traffic.jpg" preload="auto" controls></video>

You will choose “auto” to encourage the browser to actually start downloading the entire resource (i.e., to do a progressive download even if the video resource is not set to @autoplay). The particular browser may not want to do this (e.g., if it is on a mobile device), but you as a web developer signal in this way to the browser that your server will not have an issue with it and would prefer it in this way to optimize the user experience with as little wait time as possible on playback.

Support for @preload is implemented in Firefox and Safari, such that “none” loads nothing and “metadata” and “auto” set up the video element with its metadata and decoding pipeline, as well as the first video frame as poster frame. Chrome and Opera support it, but you cannot use the same video with different preload strategies on different pages or they will freeze. IE doesn’t seem to support the attribute yet and ignores it.

As a recommendation, it is, in general, best not to interfere with the browser’s default buffering behavior and to avoid using the @preload attribute.

Before diving further into the functionality of the <video> element, we are going to take a couple of pages to introduce its brother, the <audio> element.

The Audio Element

One of the authors, when delivering media seminars, inevitably refers to audio as being the “red-headed child in a family of blondes.” By this he means, audio is inevitably overlooked as web designers and developers focus on the shiny new <video> element. This is a huge mistake because, if properly used, audio can actually “seal the deal.” Horror movies wouldn’t be creepy if there was a scene where the screen is dark and all you hear is shuffling footsteps.

<audio> shares a lot of markup and functionality with the <video> element, but it does not have @poster, @width, and @height attributes, since the native representation of an <audio> element is to not display visually.

At this point, we need to look at the supported audio codecs in HTML5. Table 2-1 displays the table of codecs supported by the main HTML5 media supporting web browsers.

Table 2-1. Audio Codecs Natively Supported by the Major Browsers

Table2-1

You may have noticed that there isn’t a single encoding format supported by all web browsers. It can be expected that IE may implement support for WAV, but as WAV is uncompressed, it is not a very efficient option and should be used only for short audio files. At minimum you will need to provide Ogg Vorbis and MP3 files to publish to all browsers.

@src

Listing 2-15 is a simple code example that will embed an audio resource in HTML5.

Listing 2-15. An Audio Element with @src

<audio src="audio/Shivervein_Razorpalm.wav"></audio>

<audio src="audio/Shivervein_Razorpalm.ogg"></audio>

<audio src="audio/Shivervein_Razorpalm.mp3"></audio>

This is the absolute minimum code needed to play an audio file. Due to the fact that the audio element has no controls, there will be no visual representation of the <audio> element. This is sensible only in two circumstances: either the <audio> is controlled through JavaScript (seeChapter 3) or the <audio> is set to start playback automatically, for which it requires an @autoplay attribute.

@autoplay

To make the audio autostart, you need to add the @autoplay attribute as used in Listing 2-16.

Listing 2-16. An Audio Element with @autoplay

<audio src="audio/Shivervein_Razorpalm.mp3" autoplay></audio>

The @autoplay attribute is a Boolean attribute and is supported by all browsers, just as it is with the <video> element. Providing it will make the audio begin playing as soon as the browser has downloaded and decoded sufficient audio data. The audio file will play through once from start to end. We recommend that you use this feature sparingly, since it can be highly irritating for users.

Pay attention to the words “sparingly” and “irritating.” There has to be a valid reason for its use such as a background audio file. If your reason is that”It’s a cool track,” then you might want to reconsider.

@loop

To make the audio automatically restart after finishing playback, you use the @loop attribute shown in Listing 2-17.

Listing 2-17. An Audio Element with @autoplay

<audio src="audio/Shivervein_Razorpalm.ogg" autoplay loop></audio>

The @loop attribute, in conjunction with the @autoplay attribute, provides a means to set continuously playing “background” music or sound on your web page. This is not recommended; it is just mentioned here for completeness.

If you accidentally or deliberately create several such elements, they will all play at the same time and over the top of each other, but not synchronously. In fact, they may expose a massive drift against each other since each <audio> element only follows its own playback timeline. InChapter 4 we will learn about the @mediagroup attribute that was defined to synchronize such elements. Unfortunately, it is not currently supported by most browsers. So, you have to use JavaScript to synchronize such elements. You would poll for the current playback time of each element and reset all elements to the same playback position at regular intervals. We will learn about the tools to do this in Chapter 3. For audio streams, you could also use the Web Audio API (see Chapter 6).

@controls

You may have noticed our hectoring around the use of the @autoplay and @loop attributes. If you are planning to display an audio resource on your web page for user interaction rather than for background entertainment, you will need to turn on @controls for your <audio> element as shown in Listing 2-18.

Listing 2-18. An Audio Element with @loop

<audio src="audio/Shivervein_Razorpalm.wav" controls></audio>

One thing you will notice is that the controls (see Figure 2-8) for various browsers use a different design. Their width and height are different and not all of them display the duration of the audio resource. Since the <audio> element has no intrinsic width and height, the controls may be rendered as the browser finds appropriate. This means that Safari uses a width of 200 px; the others all use a width of 300 px. The height ranges from 25 px (Safari, Opera) to 28 px (Firefox) to 30 px (Google Chrome) to 52 px (IE).

9781484204610_Fig02-08

Figure 2-8. The audio controls for a number of browsers

In Chapter 3 we show how you can run your own controls and thus make them consistent across browsers.

@preload

The @preload attribute for <audio> works like the one for <video>. You ordinarily should not have to deal with this attribute. As you can see in the code snippet in Listing 2-19, the @preload attribute accepts three different values: “none,” “metadata,” or “auto,” which are the same values for its <video> cousin.

Listing 2-19. An Audio Element with @preload

<audio src="audio/Shivervein_Razorpalm.mp3" controls preload="none"></audio>

Web developers may choose “none” in a situation where they do not expect the user to actually play back the media resource and want to minimize bandwidth use. A browser would typically load the setup information of the audio resource, including metadata, such as the duration of the resource. Without the metadata, the duration of the resource cannot be displayed. Thus, choosing no preload only makes sense when dealing with a large number of audio resources. This is typically only useful for web pages that display many audio resources—an archive of podcasts, for example.

Use “metadata” in a situation where you do not expect the user to actually play back the media resource and want to minimize bandwidth use, but not at the cost of missing audio metadata information. This is typically the default behavior of the web browser unless the element is set to autoplay, but can be reinforced by the web developer through this attribute if supported by the browser.

“auto” is typically used in a situation where you expect an audio resource to actually be played back and want to encourage the browser to prebuffer the resource (i.e., to start progressively downloading the complete resource rather than just the setup information). This is typically the case where the <audio> element is the main element on the page, such as a podcast page. The aim of using @preload with “auto” value is to use bandwidth preemptively to create a better user experience with a quicker playback start.

Support for @preload is implemented in most browsers, such that “none” loads nothing and “metadata” and “auto” set up the audio element with its metadata and decoding pipeline.

Now that you are familiar with both the <video> and <audio> elements, we need to examine the <source> element because it has a major role in how both video and audio files will play in the browser.

The Source Element

In the previous chapter you discovered that both the <video> and the <audio> elements, thanks to the browser vendors, do not have a universally supported baseline codec. Therefore, the HTML5 specification has created a means to allow specification of alternative source files through the <source> element. This allows a web developer to integrate all the required links to alternative media resources within the markup without having to test for browsers’ support and use JavaScript to change the currently active resource.

Obviously, using individual <video> elements for each of the video types—.mp4, .webm, and .ogg—is simply inefficient. The <source> element allows you to add all of the media files within a single <video> element.

Listing 2-20 shows an example of a <video> element with multiple resources.

Listing 2-20. A Video Element with Different Source Files

<video poster="img/BabyVulture.png" controls>
   <source src="video/Vultures.mp4"/>
   <source src="video/Vultures.webm"/>
   <source src="video/Vultures.ogv"/>
</video>

For <audio>, shown in Listing 2-21, it looks as follows:

Listing 2-21. An Audio Element with Different Source Files

<audio controls>
   <source src="audio/Shivervein_Razorpalm.mp3"/>
   <source src="audio/Shivervein_Razorpalm.ogg"/>
   <source src="audio/Shivervein_Razorpalm.wav"/>
</audio>

The <source> element is an empty element. It is not permitted to have any child elements and therefore doesn’t have a </source> closing tag. If such a closing tag was used, it would create another <source> element without any attributes, so don’t use it. It is, however, possible to add a slash “/” at the end of the <source> element start tag as in <source/>—HTML user agents will parse this—but it is not an HTML5 requirement. If you were using XHTML5, though, you would need to close the empty element in this way to conform with XML requirements.

@src

The list of <source> elements specifies alternative media resources for the <video> or <audio> element, with the @src attribute providing the address of the media resource as a URL (uniform resource locator).

A browser steps through the <source> elements in the given order. It will try to load each media resource and the first one that succeeds will be the resource chosen for the <media> element. If none succeeds, the media element load fails, just as it fails when the direct @src attribute’s URL of <audio> or <video> cannot be resolved.

For example, the current version of Opera can’t play an .mp4 video. In the previous example, it would inspect the .mp4 source and realize, ”Nope. Can’t use it.” Then it will move down the order until it finds one—.webm—that it can use and starts playing it. These file inspections are called content type sniffing. They consume valuable time that the browser could use to load a file that it can actually load. This is where the @type attribute comes in.

@type

The <source> element has a @type attribute to specify the media type of the referenced media resource. This attribute is a hint from the web developer and makes it easier for the browser to determine whether it can play the referenced media resource. It will skip over files it is certain it can’t load and only test those it has a chance to load.

The @type attributes, shown in Listing 2-22, contain a MIME type with optional codecs parameters.

Listing 2-22. A Video Element with Different Source Files and @type Attributes

<video poster="img/BabyVulture.png" controls>
   <source src="video/Vultures.mp4" type=’video/mp4’/>
   <source src="video/Vultures.webm" type=’video/webm; codecs="vp8, vorbis"’/>
   <source src="video/Vultures.ogv" type=’video/ogg; codecs="theora, vorbis"’/>
</video>

Note that you need to frame multiple parameters with double quotes and thus you have to put the @type value in single quotes or otherwise escape the double quotes around the @type attribute value. You cannot use single quotes on the codecs parameter, since RFC 4281 (www.ietf.org/rfc/rfc4281.txt) specifies that they have a special meaning. RFC 4281 is the specification that defines the codecs parameter on a MIME type.

Embedding audio (see Listing 2-23) with WAV, Ogg Vorbis, and MP3 formats and explicit @type is quite similar.

Listing 2-23. An Audio Element with Different Source Files and @type Attributes

<audio controls>
  <source src="audio/Shivervein_Razorpalm.mp3" type="audio/mpeg; codecs=mp3"/>
  <source src="audio/Shivervein_Razorpalm.ogg" type="audio/ogg; codecs=vorbis"/>
  <source src="audio/Shivervein_Razorpalm.wav" type="audio/wav; codecs=1"/>
</audio>

The browsers will parse the @type attribute and use it as a hint to determine if they can play the file. MIME types do not always provide a full description of the media resource. For example, if “audio/ogg” is provided, it is unclear whether that would be an Ogg Vorbis, Ogg Flac, Ogg Speex, or an Ogg Opus file. Or if “audio/mpeg” is given, it is unclear whether that would be an MPEG-1 or MPEG-2 audio file Layer 1, 2, or 3 (only Layer 3 is MP3). Also note that codecs=1 for audio/wav is PCM.

Thus, based on the value of the @type attribute, the browser will guess whether it may be able to play the media resource. It can make the following three decisions:

·     It does not support the resource type.

·     “Maybe”: there is a chance that the resource type is supported.

·     “Probably”: the web browser is confident that it supports the resource type.

A confident decision for “probably” can generally be made only if a codecs parameter is present.

The browser makes a decision for “maybe” based on information it has available as to which codecs it supports. This can be a fixed set of codecs as implemented directly in the browser, or it can be a list of codecs as retrieved from an underlying media framework such as GStreamer, Media Foundation, or QuickTime.

You can use the following code snippet, shown in Listing 2-24, to test your browser for what MIME types it supports:

Listing 2-24. How to Discover Which Video MIME Types Are Supported

<p>Video supports the following MIME types:
  <ul>
    <script type="text/javascript">
      var types = new Array();
      types[0] = "video/ogg";
      types[1] = ’video/ogg; codecs="theora, vorbis"’;
      types[2] = "video/webm";
      types[3] = ’video/webm; codecs="vp8, vorbis"’;
      types[4] = "video/webm";
      types[5] = ’video/webm; codecs="vp9, opus"’;
      types[6] = "video/mp4";
      types[7] = ’video/mp4; codecs="avc1.42E01E, mp3"’;
      types[8] = "video/mp4";
      types[9] = ’video/mp4; codecs="avc1.42E01E, mp4a.40.2"’;
      // create a video element
      var video = document.createElement(’video’);
      // test types
      for (i=0; i<types.length; i++) {
        var support = video.canPlayType(types[i]);
        if (support == "") support="no";
        document.write("<li><b>"+types[i]+"</b> : "+support+"</li>");
      }
    </script>
  </ul>
</p>

The canPlayType() function is from the JavaScript API, which we will look at in Chapter 3.

The browsers will return “maybe” when given a MIME type without codecs parameters and “probably” when given one with codecs parameters for a format that they support. Otherwise they return the empty string.

As shown in Listing 2-25, you can do the same test for audio.

Listing 2-25. How to Discover Which Audio MIME Types Are Supported

<p>Audio supports the following MIME types:
      <ul>
        <script type="text/javascript">
          var types = new Array();
          types[0] = "audio/ogg";
          types[1] = "audio/ogg; codecs=vorbis";
          types[2] = "audio/mpeg";
          types[3] = "audio/mpeg; codecs=mp3";
          types[4] = "audio/wav";
          types[5] = "audio/wav; codecs=1";
          types[6] = "audio/mp4";
          types[7] = "audio/mp4; codecs=mp4a.40.2";
          types[8] = "audio/x-m4b";
          types[9] = "audio/x-m4b; codecs=aac";
          types[10] = "audio/x-m4p";
          types[11] = "audio/x-m4p; codecs=aac";
          types[12] = "audio/aac";
          types[13] = "audio/aac; codecs=aac";
          types[14] = "audio/x-aac";
          types[15] = "audio/x-aac; codecs=aac";
          types[16] = "audio/ogg";
          types[17] = "audio/ogg; codecs=opus";

          // create a audio element
          var audio = document.createElement(’audio’);
          // test types
          for (i=0; i<types.length; i++) {
            var support = audio.canPlayType(types[i]);
            if (support == "") support="no";
            document.write("<li><b>"+types[i]+"</b> : "+support+"</li>");
          }
        </script>
      </ul>
</p>

While all browsers are moving to a so-called sniffing of content types by downloading a small piece of the file to check its type, older browser versions still rely on the MIME types of the files being served correctly. It’s best practice to deliver correct MIME types for your files anyway. So, make sure your web server reports the correct MIME types for your files. Usually, there is a mapping between file extensions and MIME types that the web server checks (e.g., for Apache it’s the mime.types file). In your browser page inspector, check the “content-type” HTTP header that the web browser downloads for a video file to confirm.

Even then, we have to admit we have no control over what screen—Smartphone to the flat-panel TV in your home—will be used to view or listen to your content. In this case you may need to create video files with a variety of resolutions, only one of which gets loaded once the browser figures out what screen is being used. This is where the @media attribute plays a huge role in today’s Responsive Web Design universe.

The “codecs” parameter in @type is optional and even though it helps browsers be more confident around whether they will be able to decode a resource, ultimately that test comes only with trying to load the resource. We therefore recommended that you use only the MIME type without the codecs parameters.

@media

The <source> element only exists to help the browser select the first acceptable media resource from a list. The @type attribute helps identify whether the browser supports that media format. The @media attribute further provides for associating so-called media queries with a resource.

To preserve your user’s sanity, encode your video in an appropriate resolution for the devices you are targeting—it will enable you to both target everything from 4K screen sizes to mobile devices—from the same video element. What you don’t do is to adjust the width and height of the video element. You provide copies of the different files called by the media query. You don’t want to deliver a huge HD video file to a small mobile screen—it causes the browser to have to download more data than it can display, decode a higher resolution than it can display, and then have to downsample for your actual device. Thus, even a high-quality encoded video will have a poorer rendering on a mobile device than an adequately sized video.

Media queries exist to specify that a specific resource is tailored to a specific (range of) output device(s). For example, a @media value of "min-width: 400px" specifies that the resource is targeted toward display areas of at least 400 px width.

Many different media queries are possible. Following are just a few examples used on media elements:

·     @media="handheld" to indicate that the media resource is appropriate for handheld devices.

·     @media="all and (min-device-height:720px)" to indicate that the media resource is appropriate for screens with 720 lines of pixels or bigger.

·     @media="screen and (min-device-width: 100px)" to indicate that the media resource is appropriate for screens with 100 lines of pixels or higher.

If you are only concentrating on working with browsers on desktops, laptops, and, in certain instances, tablets, this attribute should not be of much interest.

The Default Player Interfaces

We conclude this discussion with a quick look at the user interfaces the browsers have implemented for audio and video elements. These interface designs are still in flux—YouTube launches a new player interface roughly every six months—and it is likely that web browsers will make improvements and add features to their audio and video players for a while to come.

The default user interface for HTML5 audio and video elements is separated into two different types: the visible controls and the controls hidden in the context menu, usually reached through right-clicking the element. The design of the players is at the discretion of the browsers, which means each player is different.

Visible Controls

We have already encountered the visible controls of the main browsers. In this section we are going to look at them a little more closely in regard to the functionality each browser provides. Controls are mostly identical between audio and video elements.

We start with Firefox’s controls shown in Figure 2-9.

9781484204610_Fig02-09

Figure 2-9. The Firefox controls

Firefox’s controls provide the following functionality:

·     play/pause toggle

·     timeline with direct jump to time offset (seeking)

·     timeline displays playback position and buffer progress

·     playback time/duration display

·     volume slider

·     volume on/off button

·     full-screen display button

If the video has no audio track, a crossed-out audio volume controller is displayed. The Firefox audio controls have the same functionality except for the full-screen button.

Firefox has made these controls accessible and easily usable through keyboard control.

Firefox’s controls also provide the following keyboard access:

·     tab: tab onto and off video element.Once on the video element, the following keyboard shortcuts work.

·     space bar: toggles between play and pause.

·     left/right arrow: winds video forward/back by 15 seconds.

·     Command/CTRL+left/right arrow: winds video forward/back by 1/10 of the media duration.

·     HOME: jumps to beginning of video.

·     End: jumps to end of video.

·     up/down arrow: when focused on the volume button, increases/decreases volume.

Full-screen access is not available through a keyboard command.

Macintosh keyboard equivalents are Command for the PC CTRL key and option for the PC Alt key, Function left arrow for the PC Home key, and Function right arrow for the PC End key.

Next up is Safari from Apple. Figure 2-10 shows the Safari player.

9781484204610_Fig02-10

Figure 2-10. The controls on the video elements in Safari

Safari’s controls provide the following functionality:

·     30-second jump back button

·     play/pause toggle

·     playback time display

·     timeline with direct jump to time offset (seeking)

·     timeline displays playback position and buffer progress

·     playback count-down time display

·     volume on/off button with volume slider

·     full-screen button

The audio controls have the same functionality as the normal video controls (when displayed at a minimum width of 264 px), except they don’t have a full-screen button.

Safari doesn’t yet seem to provide keyboard controls for the media elements.

Next, let’s take a look at Google Chrome’s controls as shown in Figure 2-11.

9781484204610_Fig02-11

Figure 2-11. The controls on the video and audio elements in Google Chrome

Google Chrome’s controls provide the following functionality:

·     play/pause toggle

·     timeline with direct jump to time offset (seeking)

·     timeline displays playback position and buffer progress

·     file duration display upon load, playback time display during playback

·     volume on/off button

·     volume slider

·     full-screen button

If the video element has no audio track, the volume button is grayed out. Again, the audio controls are the same as the video controls except they lack the full-screen button.

Google Chrome doesn’t yet seem to provide keyboard controls for the media elements.

Next in line is Opera which, as shown in Figure 2-12, closely resembles those from Chrome.

9781484204610_Fig02-12

Figure 2-12. The Opera controls

Opera’s controls provide the following functionality:

·     play/pause toggle

·     timeline with direct jump to time offset (seeking)

·     timeline displays playback position and buffer progress

·     file duration display upon load, playback time display during playback

·     volume on/off button

·     volume slider

·     full-screen button

Opera also makes these controls accessible and easily usable through keyboard control.

Opera’s controls provide the following keyboard access:

·     Tab: tab onto play button to transport bar and volume control.

·     When on play button: space bar toggles between play and pause.

·     When on volume control: space bar toggles between volume on and off.

·     When on transport bar: left/right moves to start/end respectively.

·     When on transport bar: CTRL-left/right moves 1/10 of the video duration backwards/forwards.

·     When on volume control: up/down arrow increases/decreases volume.

Our last browser is Internet Explore 10’s controls as shown in Figure 2-13.

9781484204610_Fig02-13

Figure 2-13. The controls on the video and audio elements in IE 10

IE’s controls provide the following functionality:

·     play/pause toggle

·     timeline with direct jump to time offset (seeking)

·     timeline displays playback position and buffer progress

·     playback time and duration display

·     volume slider

·     volume on/off button

·     full-screen button

IE also makes some of these controls accessible and usable through keyboard control.

·     tab: tab onto and away from the video

·     space bar: toggles between play and pause

·     up/down arrow: increases/decreases volume

·     left/right arrow: winds video to the beginning/end

The IE controls are semi-transparent. Firefox’s controls are also semi-transparent. All the other browsers have solid-colored controls.

Context Menus

Context menus provide users with shortcuts to common operations. Context menus are visible when the user right-clicks or Control-click’s, on a Mac, the video or audio element. Most functionality is the same for audio and video.

Firefox’s context menu, Figure 2-14, contains the following items:

·     Toggle Play/Pause

·     Toggle Mute/Unmute

·     Select Play Speed—useful for longer videos

·     Toggle Show/Hide controls

·     View Video (opens the video by itself in a new tab, similar to View Image).

·     Copy Video Location (this is the video URL).

·     Save Video As.... Saves the video file locally.

·     Save Snapshot As... Saves the current video frame as an image locally.

·     Email video... Composes an email with the video URL in the body.

9781484204610_Fig02-14

Figure 2-14. The Firefox context menus

In the case of Safari, the context menu, shown in Figure 2-15, is a bit different if not minimalist. You have menu items that allow you to Hide the Controls or select a video-only full-screen rendering instead of clicking the Enter Full Screen button on the controller.

9781484204610_Fig02-15

Figure 2-15. The context menu on the video element in Safari

Google Chrome’s context menu, shown in Figure 2-16, contains the following functionality:

·     Toggle Play/Pause

·     Toggle Mute/Unmute (when sound is available)

·     Toggle Loop on/off

·     Toggle Show/Hide controls

·     Save Video locally under a name

·     Copy Video location

·     Open Video in New Tab

·     Inspect the <video> element (this feature is for developers who want to inspect the HTML code)

9781484204610_Fig02-16

Figure 2-16. The context menu on the video element in Google Chrome

Opera’s context menu, shown in Figure 2-17, is pretty similar to the others.

9781484204610_Fig02-17

Figure 2-17. The context menu on the video element in Opera

The last one is Internet Explorer 10’s context menu, shown in Figure 2-18, which offers the usual choices along with an opportunity to Bookmark the page.

9781484204610_Fig02-18

Figure 2-18. The context menu on the video element in IE

Publishing

To this point in the chapter we have introduced how to write web pages with HTML5 video elements. In the previous chapter we reviewed how to encode the media files in order to have them supported by HTML5 video capable browsers. Now we close the circle by looking at how to actually publish the videos and their web pages by understanding how these files move from “here”—the server—to “there”—the browser. After this, we have all the tools to make HTML5 web pages with video and audio available.

Here we will merely look at how to publish your set of media files via an ordinary web server. This is the most basic means of publishing.

Remember, you have to create at least one version of your video file in MPEG-4 and one in WebM (or Ogg Theora) to allow you to reach all browsers. For audio files, you will need MP3 and Ogg Vorbis versions. Also, if you are targeting video for a high resolution and a lower resolution, you are best off using different video file versions with different dimensions and use media queries to distinguish between them.

Listing 2-26 shows an example markup for video.

Listing 2-26. A Video Element That Supports Multiple Browsers and Video Element Sizes

<video poster="img/BabyVulture.png" controls>
     <source src="video/Vultures.mp4" media="screen and (min-width:800px)"
                   type=’video/mp4’ />
     <source src="video/Vultures-small.mp4" media="screen and (max-width:799px)"
                   type=’video/mp4’ />
     <source src="video/Vultures.webm" media="screen and (min-width:800px)"
                   type=’video/webm’ />
     <source src="video/Vultures-small.webm" media="screen and (max-width:799px)"
                   type=’video/webm’ />
</video>

You should copy the WebM and MP4 files into a directory on your web server where it is appropriate for your web application/web page. A web server is a piece of software that can speak HTTP (the HyperText Transfer Protocol) and deliver web content through computer networks. Several open source web servers exist, the most popular being Apache and Nginx.

Serving HTML5 video over HTTP is the standard way in which the HTML5 video element is being supported by web browsers. When making a choice between which server software to choose, make sure it supports HTTP 1.1 byte range requests. Most common web servers, including Apache and Nginx, will support it, but occasionally you can still find one that doesn’t or doesn’t work properly.

Support for HTTP byte range requests is important because it is the standard way in which browsers receive HTML5 media resources from web servers—this is called progressive download. The importance of byte range requests is twofold: first, it allows transfer of a media file in small chunks, which gives the browser the ability to start playback without waiting for the full file to download. Second, and more important, it allows getting these chunks from anywhere in the file, rather than waiting for all the previous data to have been received. This in particular allows seeking to random locations of the video and starting playback from there.

How does that work? Well, media file decoding is a complicated matter. A media file contains information required to set up the audio and video decoding pipelines (see Chapter 1). This information is typically stored at the beginning of a file. Audio and video data is provided in a multiplexed manner (i.e., a bit of video, then the related bit of audio, then the next bit of video, etc.). To separate between these bits of data, and to get a mapping to playback time, media files often contain an index table. In MP4 files, that index sits at the end of the file. The browser cannot start decoding and playback without receiving this file. So, without byte range requests, we’d have to wait with playback until the complete file was downloaded.

After setting up the metadata and getting the index table, the browser will typically make only one request for the entire media resource and will start playing it back as the data arrives. For a very long file, when download is very far ahead of the current playback position and the user has stopped watching the video, downloading may be paused by the browser. A condition that allows the browser to pause download is that the web server supports byte range requests so the browser may resume downloading when the playback position is again getting closer to the buffered position. This will save bandwidth use, particularly on videos that are longer than a few minutes.

All of the received data will be cached in the browser. When the user seeks and the seek time is not yet buffered, the browser will stop the download and request a byte range that starts at the given time offset the user seeked to.

What we have just described is performed by the browser in the background. It is something the user will never really notice but is important for you to understand.

When setting up your web server to host HTML5 audio and video, you don’t have to do anything special. For older browsers, which perform limited content sniffing, you need to ensure that the media resources are served with the correct MIME type. It is possible that your web server may require manually adding some of the following media types to a mime.types setup file:

audio/ogg    ogg oga
audio/webm   webm
video/ogg    ogv
video/webm   webm
audio/mp4    mp4a
audio/mpeg   mp3
video/mp4    mp4

To publish HTML audio and video, you only need to copy the files into a directory structure on your web server and make sure the resource location and hyperlinks are all correct and they work by doing a browser test.

The following text block shows an example layout of a directory structure on a web server (/var/www is the typical root directory of a web site hosted with Apache on a Linux server).

/var/www/ - the Web server root directory, exposed e.g. as www.example.net/

/var/www/page.html – the Web page with a video element

/var/www/video/video.webm and
/var/www/video/video.mp4 – the video in two formats

/var/www/thumbs/video.png – the video’s thumbnail

/var/www/audio/audio.ogg and
/var/www/audio/audio.mp3 – the compressed audio in two formats

What this basically says is that the web page at www.example.net/page.html would include a video <source> with video/video.webm and one with video/video.mp4.

To test if your web server supports HTTP 1.1 byte range requests, you can inspect the HTTP header via a browser developer tool or try downloading a video file from the following command line, for example, using a URL loading tool such as curl:

$ curl -I  http://www.example.net/video/video.webm

You are looking for a response header line that contains “Accept-Ranges: bytes.” If that exists, you’re fine. If not, you should use a different web server.

Now that you know how to add media, control it, and serve let’s take a look at the final piece of the equation—how it is presented to the user through CSS3.

CSS and Video

As is so typical of the evolution of web design and development, things have changed over the years. One of the biggest changes is the shift to Responsive Web Design, which acknowledges that we have no control over the screen/viewport being used to view the content. This has resulted in a move to the use of CSS to control the presentation of a video rather than inline styles whereby the content and its presentation are literally embedded into the body of the HTML document.

Our assumption around CSS is that you have a basic understanding of this technology.

In conjunction with the change to a Responsive universe and the change over to HTML5 there is also an important change in CSS that is underway. A new standard—CSS level 3 or CSS3—is in development which provides many new and improved features including improved layout, shapes, extended media queries, and improvements in rendering text with speech synthesis. Many of these can be applied to HTML5 video and, in this section, we will be taking a brief look at specific examples.

Just to keep things simple, we will be using inline styles to demonstrate techniques rather than using an external style sheet, which is regarded as a best practice.

We start with the basic CSS box model and just add a bit of “jazz” to how the <video> element is presented in the browser. To this point it has simply been treated as content on a page. CSS3 lets you make it more noticeable. In this case we have added a drop shadow and a rounded border, added a bit of padding, and resized the video. Listing 2-27 presents the code.

Listing 2-27. CSS to Make Your Video Look Nicer

video{
    width: 80%;
    padding: 15px;
    margin: 0px;
    border: 5px solid black;
    border-radius: 15px;
    box-shadow: 10px 10px 5px gray;
    box-sizing: border-box;
}

When viewed in the browser, Figure 2-19 is what the user sees.

9781484204610_Fig02-19

Figure 2-19. CSS3 styling pulls attention to the video

It isn’t only the styling of the <video> element that can grab the viewer’s attention; it is also its positioning on the page that will make it stand out. CSS lays out all elements on a page through its “box model”—that is, all HTML elements are represented by a rectangular box on the page and rendered according to standard rules, all of which can be overruled by CSS properties.

The following are typical CSS properties that are being used to style boxes:

·     width and height: as with the @width and @height attributes on HTML elements, you can scale boxes relatively (percentage or em) or absolutely (px).

·     padding: space around the content of the box, which shares the background of the content.

·     border: the border around the content, including the padding space.

·     margin: a transparent space around the outside of the box.

CSS3 introduced many new styling methods, such as border-radius for rounded corners on the box, border-image to use images as borders, border-shadow for drop-shadows, and box-sizing to specify what parts of the box the dimension calculation relates to.

CSS, however, doesn’t just do styling. It also takes care of exact positioning of the boxes on the page. This is done first by identifying the box’s default display position, then adjusting its position with the “position” and “float” properties

There are three different basic display box types.

·     Block boxes (display: block): boxes that are placed on a row by themselves—think of paragraphs of text and how they are placed underneath each other.

·     Inline boxes (display: inline): boxes that are placed in the normal flow row by row, usually inside a block box—think of the characters of a sentence of text and how they flow.

·     Invisible boxes (display: none): boxes that are not visible on the page (e.g., audio elements without a @controls attribute).

All other display box types are derivatives of these three basic types. Not only that, but you can influence the box type by changing the “display” property on a box.

By default, audio and video have an inline box type. This means they basically behave like a character in a paragraph and move around in the context of neighboring inline box elements and parent block elements.

If we start with the basic code shown in Listing 2-28 we can see this at work in this example with a 50% wide video.

Listing 2-28. A Video Element Is an Inline Element

<div class=".container">

    <h2>The basic code:</h2>
    A sentence before the video element.
    <video src ="video/Vultures.mp4"  controls></video>
   A sentence after the video element.
</div>

Figure 2-20 shows the result. Note the placement of the sentences above and below the video. This is normal mode.

9781484204610_Fig02-20

Figure 2-20. The video is in its normal inline mode

To make the video turn up on its own paragraph (line), you can put a <div> or <p> around each of the sentences, which leaves the video in its own anonymous block. You could also put a “display:block” rule on the video element, turning it into a “block” element and forcing the sentence before and after each in their own anonymous block.

Listing 2-29 shows this process with the following markup:

Listing 2-29. Make the Video Stand Out in Its Own Anonymous Block

 <p>A sentence before the video element.</p>

  <video poster="img/BabyVulture.jpg" controls>
     <source src="video/Vultures.mp4" type=’video/mp4’/>
     <source src="video/Vultures.webm" type=’video/webm’/>
     <source src="video/Vultures.ogv" type=’video/ogg’/>
  </video>
  <p>A sentence after the video element.</p>

When you browser test, the content moves to the horizontal content alignment typical of an inline box type as shown in Figure 2-21.

9781484204610_Fig02-21

Figure 2-21. The video is in its own (anonymous) block

If we make the video “display:none,” it disappears from rendering. We could, instead, make it merely invisible with “visibility:hidden.” This will keep the inline video box where it was but make it completely transparent. Note that the video will still preload, so you might want to use@preload="none" to avoid this.

Once the default positioning of the elements on the page has been finished—that is, the browser has placed all the elements in their “normal” display position based on their “display” property—the browser will check on the “position” and “float” properties.

One of the more common techniques is to place the video in a block of text and actually have the text flow around it. This is accomplished using the float property, which can be set to left or right as shown in the code presented in Listing 2-30.

Listing 2-30. Float the Video to the Right Within Its Parent Block

video{
    float: right;
    width: 30%;
    padding: 15px;
    margin: 10px;
    border: 5px solid black;
    border-radius: 15px;
    box-shadow: 10px 10px 5px gray;
    box-sizing: border-box;
}

The result, shown in Figure 2-22, is text wrapping around a <video> element.

9781484204610_Fig02-22

Figure 2-22. Text wrap using the float:right property

Now that we understand how CSS can be used to position the <video> element, let’s take a look at how CSS properties can also change its “look” on the page.

CSS Basic Properties

There are a number of CSS properties that can be applied to HTML5 media elements, but, in the interests of space and time, let’s look the following two properties you can apply:

·     Opacity: the video is semi-transparent; and

·     Gradient: add a color effect to a video.

The opacity property has been around for a long time but has only become standardized across the browsers when it made the official CSS3 lineup. Opacity is defined as any value between 0.0 and 1.0 where 0.0 is when the element is fully transparent and 1.0 where it is fully opaque.

Listing 2-31 presents the code that shows 60% opacity applied to a video and the video moved over the top of an image rendered before it using the “position,” “top,” and “left” properties.

Listing 2-31. Display a Semi-transparent Video

video{
    opacity: 0.6;
    width: 50%;
    padding: 15px;
    margin: 0px;
    border: 5px solid black;
    border-radius: 15px;
    box-shadow: 10px 10px 5px gray;
    box-sizing: border-box;
    position: relative;
    top: -440px;
    left: 126px;
    background-color: white;
}
...
    <img src="img/BabyVulture.jpg"/>
    <video poster="img/BabyVulture.jpg" controls>
       <source src="video/Vultures.mp4" type=’video/mp4’/>
       <source src="video/Vultures.webm" type=’video/webm’/>
       <source src="video/Vultures.ogv" type=’video/ogg’/>
    </video>

As you can see in Figure 2-23 the background image under the video is showing through.

9781484204610_Fig02-23

Figure 2-23. An opacity value of .5 is applied to the video

Applying a gradient to a video is a pretty interesting challenge. With images, one would put the image into the background and overlay a gradient on top of it. We want to replicate this for video but cannot use CSS to put the video in the background. Thus, we have to render a gradient<div> on top of the video. The CSS3 spec specifies two functions to create gradients: linear-gradient() and radial-gradient().

In our case (see Listing 2-32) we want to apply a linear gradient for the video. To accomplish this we simply created a <div> to hold a gradient that moves from transparent to solid white. Then we overlay the gradient onto the video element using relative positioning.

Listing 2-32. Display a Gradient on Top of the Video

<style type= "text/css">
video{
    width: 400px;
        height: 225px;
}
#gradient{
    position: relative;
    width: 400px;
    height: 225px;
    top: -225px;
    background: linear-gradient(rgba(255,255,255,0), white);
    pointer-events: none;
}
</style>
...
    <video src ="video/Vultures.mp4" controls></video>
     <div id="gradient"></div>

The key here is the background property. This property requires two values: the start color and the end color. In this case we are moving from a transparent white to an opaque white at the bottom of the gradient. The opacity changes give the effect of the gradient mask over the video. We use the vendor prefixes to ensure the gradient works in all of the browsers. Figure 2-24 shows the end result.

9781484204610_Fig02-24

Figure 2-24. The gradient is placed over the video

A side effect of this feature is that the video, as you can see, is actually hidden underneath the gradient <div>. The implication here is that the controls are hidden from user interaction. You have to set the CSS pointer-event property on the gradient <div> to “none” to get pointer events redirected to the video element. You could also set the video to autoplay and remove the @controls attribute instead, but then users cannot interact with the video. Your last resort would be to create your own JavaScript controls. We will teach how to do this in the next chapter.

Playing Background Video

This technique is becoming quite common. You arrive at a site and there is a full-screen video playing under the content on the page. Though there are a number of rather complicated techniques for accomplishing this, it can easily be done. The trick is in the CSS. One of the CSS properties available is z-position. A positive number puts the <div> with the video in front of everything and a negative number pushes it under everything.

9781484204610_Fig02-25

Figure 2-25. Full-screen background video

As you can see in Figure 2-25, the names of the book authors are displayed over the video playing in the background.

From this you can surmise, as shown in Listing 2-33, that the video was simply “slid” under the content by using the large negative z-index number in the following code block:

Listing 2-33. Using the CSS z-index Property to Position a Video in the Background

#video_bkgrnd{
        position: absolute;
        bottom:0px;
        right:0px;
        min-width:100%;
        min-height:100%;
        width:auto;
        height:auto;
        z-index: -1000;
        overflow:hidden;
}

<video id="video_bkgrnd" poster ="img/BabyVultures.jpg" muted autoplay loop >
  <source src="video/HK_Traffic.mp4" type="video/mp4">
  <source src="video/HK_Traffic.webm" type="video/webm">
  <source src="video/HK_Traffic.ogv" type="video/ogg">
  <p>This browser can’t show HTML5 video.</p>
</video>

Obviously, having controls and audio for this sort of thing is out of the question, and because we don’t have any control over how long the user stays on the page, the video should loop, which is why it has a @loop attribute in Listing 2-33.

Though this technique is in the realm of “cool,” it is one you need to use with a high degree of caution.

First, the physical dimensions of the video pretty well shackle it to desktop use. Even then this technique is best-suited to looping videos that are short—around 10–30 seconds—and whose file size is not large. For example, reducing the video’s color depth to grayscale or 8-bit (256 colors) will have a significant impact on the final file size. Also, the video needs to be compressed as effectively as possible while still being able to scale across devices and screens. As a rule of thumb, try to keep the file size under 5 mb with 500k being the ideal.

Ideally the video should display a placeholder image to accommodate browsers that do not support HTML5. This placeholder image will also be used a background on mobile devices because most phones and tablets do not support autoplay.

CSS 2D Transforms

A really neat feature of the CSS3 specification is the use of transforms which allow you to reposition the video in its CSS box. The following transform functions are available:

·     matrix: describes a linear transformation on the box.

·     rotate, rotateX, rotateY: describes how the box is rotated.

·     scale, scaleX, scaleY: describes how the x and y axes of the box should be rotated.

·     skew, skewX, skewY: describes how the x and y axes should skew based on an angle.

·     translate, translateX, translateY: describes the horizontal or vertical repositioning of the box.

In this example we rotate the video -30 degrees. The code in Listing 2-34 makes this happen.

Listing 2-34. Display a -30 Degree Slanted Video

video {
 width: 50%;
 transform: rotate(-30deg);
 -webkit-transform: rotate(-30deg);
 position: relative;
 left: 100px;
}

<body>
  <video src="media/HK_Traffic_1.mp4" controls/>
</body>

In this example the video is simply rotated 30 degrees to the left using the transform property. We used the –webkit prefix to make this work in Chrome, Safari, and Opera, which are in the process of removing this prefix.

Figure 2-26 shows the end result.

9781484204610_Fig02-26

Figure 2-26. The rotate transform applied to the video

CSS Animations

CSS3 introduces animations that allow you to put objects in motion without writing JavaScript.

Ignoring JavaScript when this specification was being discussed was not without controversy because JavaScript already allows the same effects. However, a key web principle won the day. The principle was one we have stressed throughout this chapter: HTML manages content, CSS handles display, and JavaScript handles interactivity.

Animations are really nothing more than an extension of the CSS transitions. They have a start state and an end state and the object moves from “here” to “there” over a given length of time. You can also add a series of intermediate states to the action through the use of the @keyframesselector.

Listing 2-35 shows the code that moves a video from left to right and back again.

Listing 2-35. The Video Element Moves Twice Between the 0 and the 600 px Mark

@keyframes moveIt {
  0% {transform: translateX(0);}
  100% {transform: translateX(600px);}
}
video{
    width: 200px;
    animation-name: moveIt;
    animation-duration: 2s;
    animation-timing-function: ease-out;
    animation-iteration-count: 4;
    animation-fill-mode: forwards;
    animation-direction: alternate;
}

We have left out the -webkit prefixes, which are necessary for all browsers except IE and Firefox. Figure 2-27 shows the initial and an intermediate state of the animated video element.

9781484204610_Fig02-27

Figure 2-27. Two states of the animated video element

As you can see from the code, there are actually two pieces to the code. The properties start with a name—moveIt—which makes the link to the @keyframes rule. The @keyframes rule defines the animation stepping points. The video moves across the screen over two seconds and the alternate property simply reverses the animation. You should also pay particular attention to animation-time-function property, which applies easing to the motion.

The @keyframes rule sets the limits on the distance of the motion on the X axis.

We have given merely a small introduction to the use of CSS with the video element. CSS3 in particular has many more features that can be applied to CSS boxes. The visuals that you can develop are almost limitless. In a previous book version we have shown spinning 3D cubes with video running on each side.

Summary

As you have discovered throughout this chapter there is a lot more to working with HTML5 video than simply placing it between the <video></video> tags. We covered

·     A thorough review of the <video> element

·     A review of the attributes that can be applied to the <video> element

·     How to use the attributes for the <audio> element

·     A thorough review of how media moves from a server to a web page

·     Where CSS fits into the design and development process including the use of CSS transforms, CSS transitions, and CSS animations

That is a lot of stuff to cover, but it is what you need to know to successfully add audio or video to an HTML5 page. In the next chapter we dig into making media interactive through the use of the JavaScript API. We’ll see you there.