Mobile Movies withHTTP Live Streaming   Chris Adamson • @invalidname    March 9, 2013 • Chicago, IL   Sides and code avail...
So Many Streaming     Apps!
Netflix
Flixter
ABC
PBS
NBC Olympics Live     Extra
DirecTV
UStream
Crunchyroll
TwitchTV
iTunes Festival
Apple Events
Oh, and it can be embedded     in the <video> tag
Sounds good? Howdo I get in on that?
What Youll Learn• What streaming is (and isnt)• Setting up HLS on the server• Using HLS streams in iOS apps• Real-world de...
HLS: What It Is (and      isnt)
Good Ol Broadcast
Broadcast Media• Always live on some channel (a band of EM  spectrum).• Every client tuned to that channel sees the  same ...
Internet• Generally one-to-one (host to host). • Multicast IP is an exception, but is rare on   the public Internet.• Two-...
Impedence Mismatch!
Ye Olde Streaming• Client makes socket connection and keeps it  open for duration of program.• Server sends media at playb...
Streaming Problems• Difficult and expensive to scale.• Special port numbers routinely blocked by  businesses, ISPs, firewa...
What If…• We didnt need an always-on socket  connection?• We could just run over port 80?• We could just adopt industry st...
HTTP Live Streaming• Serves media as a series of short flat files, via  HTTP, usually on port 80.• Any web server will do....
Serving up HLS• Client URL is an .m3u8 playlist file• Playlist points to the media segment files
The HLS playlist#EXTM3U                    Format: .m3u8 format,#EXT-X-TARGETDURATION:10#EXT-X-VERSION:3                  ...
<video src="prog_index.m3u8" width="640" height="480"controls autoplay>No video tag support</video>
How is this better than   a flat .m4v file?• Streams can provide variants for different  bandwidths (as we’ll see…)• Segme...
The “Live” in HLS• A playlist is a live stream if it doesn’t have an  #EXT-X-ENDLIST tag• Live playlist will generally jus...
Serving up HLS
mediafilesegmenter• Splits an A/V file into segment files, creates  the .m3u8 playlist• Source must be .mov or .m4v with H...
Technical Note TN2224The following audio and video formats are supported:• Video: H.264 Baseline Profile Level 3.0 (iPhone...
Yuna:HTTP Live Streaming tests cadamson$ mediafilesegmenter-f basic source/IMG_0251.MOVJun 24 2012 10:01:24.203: Using flo...
Demo
Variant Playlists• One bitrate does not fit all: Mac on Ethernet  versus iPhone on Edge.• Solution: encode your video at m...
variantplaylistcreator• Creates a playlist that itself points to playlists  created with mediafilesegmenter.  • Each entry...
Encoding the variants
TN2224Recommendations
Creating variantsYuna:HTTP Live Streaming tests cadamson$mediafilesegmenter -I -f variants/broadband/ source/IMG_0254_Broa...
Creating variant playlistYuna:HTTP Live Streaming tests cadamson$variantplaylistcreator -o variants/variants.m3u8 variants...
Demo
Thats Great, but…How do we keep people from stealing our stream?
Encryption• HLS encrypts files, not transport.• Easy to scale: still serving flat files, but now  theyre useless without d...
Encrypting a playlistYuna:HTTP Live Streaming tests cadamson$ mediafilesegmenter -I -k keys -f encrypted/cellular source/I...
Encrypted .ts files
Protect those keys/www/protected                 /www/hls
Demo
Captions• HLS supports CEA-608 closed captions in the  MPEG-2 Transport Stream • If using file segmenter, add a closed-   ...
Client-side HLS
Opening an HLS        stream• Provide the .m3u8 URL to  MPMoviePlayerController or AVPlayer• Add the movie view or layer t...
Create an   MPMoviePlayerController// create new movie playerself.moviePlayer = [[MPMoviePlayerController alloc]        in...
Add it to your UI[self.moviePlayer.view setFrame:       self.movieContainerView.bounds];[self.movieContainerView addSubvie...
MPMovieScalingMode    AspectFit
MPMovieScalingMode    AspectFill
MPMovieScalingMode        Fill
Accessing Encrypted     Streams• Media Player and AV Foundation can use  NSURLCredentials that youve provided• Place crede...
Setting credentials (1)NSURLProtectionSpace *protectionSpace =  [[NSURLProtectionSpace alloc]  initWithHost:host  port:por...
Setting credentials (2)NSURLCredential *credential = [NSURLCredential credentialWithUser:username                 password...
Live Streaming
Live Streaming• mediastreamsegmenter mostly works like the  file version, but takes its input from UDP  stream or a Unix p...
You Dont• None of Apples tools create the required  stream • This is a "third party opportunity" • Which begs the question...
Streaming in the Real       World     Its not all about iPhones…
Streaming Clients• Mobile Devices: iPhone, iPad, iPod Touch…  plus Android, Windows Mobile, etc.• Mac and Windows PCs• Ove...
Lets Get Practical• What devices do you have to be on?• What devices will you get for free? • HLS is the preferred format ...
HLS Alternatives• Flash still rules on the desktop/browser  space, thanks in part to Mozillas obstinance  about H.264 in <...
MPEG-DASH• Attempt at a standardized approach to HTTP  adaptive-bitrate streaming. ISO/IEC 23009-1.           http://xkcd....
Emerging Consensus• Flash for PCs• HTTP Live Streaming for iOS• Plus whatever other devices you need to  support
Real-World HLS• Can you competently encode all your media  at all the variant bitrates you need?• Do you have a way to QC ...
Build or Buy: Services• Provide hosting, transcoding, bandwidth • All-in-one: UStream, LiveStream, Justin.tv /   Twitch.tv...
invalidstreamhttp://www.ustream.tv/channel/invalidstream
Production Demohttp://www.telestream.net/wirecast/
Content Delivery            Networkshttp://en.wikipedia.org/wiki/Content_delivery_network
Content Delivery        Networks• CDNs host your media on edge servers that  are closer to your clients. Less strain on yo...
Buy or Build: Encoders
Buy or Build:          Bandwidth• Each HLS client will consume up to 1GB /  hour, depending on variant bitrates, client  b...
Bandwidth CostsPrices from ScaleEngine, UStream, and LiveStream as of                    October 2012
Self-Hosted CostsPrices from MacMiniColo and Amazon EC2 as of March 2013
Wrap Up
Takeaways• HLS is a very practical streaming solution• Only part of the picture if youre multi-platform• Encoding and serv...
Q&ASlides and code will be available on my blog:        http://www.subfurther.com/blog    http://www.slideshare.net/invali...
Mobile Movies with HTTP Live Streaming (CocoaConf Chicago, March 2013)
Mobile Movies with HTTP Live Streaming (CocoaConf Chicago, March 2013)
Upcoming SlideShare
Loading in...5
×

Mobile Movies with HTTP Live Streaming (CocoaConf Chicago, March 2013)

1,000

Published on

If your iOS app streams video, then you're going to be using HTTP Live Streaming. Between the serious support for it in iOS, and App Store rules mandating its use in some cases, there realistically is no other choice. But where do you get started and what do you have to do? In this session, we'll take a holistic look at how to use HLS. We'll cover how to encode media for HLS and how to get the best results for all the clients and bitrates you might need to support, how to serve that media (and whether it makes sense to let someone else do it for you), and how to integrate the HLS stream into your app.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,000
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Mobile Movies with HTTP Live Streaming (CocoaConf Chicago, March 2013)"

  1. 1. Mobile Movies withHTTP Live Streaming Chris Adamson • @invalidname March 9, 2013 • Chicago, IL Sides and code available on my blog: http://www.subfurther.com/blog
  2. 2. So Many Streaming Apps!
  3. 3. Netflix
  4. 4. Flixter
  5. 5. ABC
  6. 6. PBS
  7. 7. NBC Olympics Live Extra
  8. 8. DirecTV
  9. 9. UStream
  10. 10. Crunchyroll
  11. 11. TwitchTV
  12. 12. iTunes Festival
  13. 13. Apple Events
  14. 14. Oh, and it can be embedded in the <video> tag
  15. 15. Sounds good? Howdo I get in on that?
  16. 16. What Youll Learn• What streaming is (and isnt)• Setting up HLS on the server• Using HLS streams in iOS apps• Real-world deployment
  17. 17. HLS: What It Is (and isnt)
  18. 18. Good Ol Broadcast
  19. 19. Broadcast Media• Always live on some channel (a band of EM spectrum).• Every client tuned to that channel sees the same thing, at the same time.• One-way, one-to-many model.
  20. 20. Internet• Generally one-to-one (host to host). • Multicast IP is an exception, but is rare on the public Internet.• Two-way communication over sockets.• Routing can take many hops, via multiple transport media (wire, wifi, cellular, etc.).
  21. 21. Impedence Mismatch!
  22. 22. Ye Olde Streaming• Client makes socket connection and keeps it open for duration of program.• Server sends media at playback speed (plus buffering). • Shoutcast: MP3 files served slowly over HTTP.• Typically use a special port number and special server software.
  23. 23. Streaming Problems• Difficult and expensive to scale.• Special port numbers routinely blocked by businesses, ISPs, firewalls, etc.• Competing standards: Real Player, Windows Media, QuickTime (all with their own plugins). • No wonder Flash won.• Good luck holding a socket connection on cellular.
  24. 24. What If…• We didnt need an always-on socket connection?• We could just run over port 80?• We could just adopt industry standards like H.264 and AAC instead of cooking custom codecs?
  25. 25. HTTP Live Streaming• Serves media as a series of short flat files, via HTTP, usually on port 80.• Any web server will do.• Client software reassembles the data into a continuous media stream.• Spec does not specify contents, but Apple uses H.264 and AAC, just like all their media apps.
  26. 26. Serving up HLS• Client URL is an .m3u8 playlist file• Playlist points to the media segment files
  27. 27. The HLS playlist#EXTM3U Format: .m3u8 format,#EXT-X-TARGETDURATION:10#EXT-X-VERSION:3 just a list of files to play#EXT-X-MEDIA-SEQUENCE:0#EXT-X-PLAYLIST-TYPE:VOD Metadata tags describe#EXTINF:9.975, fileSequence0.ts the contents#EXTINF:9.975, fileSequence1.ts#EXTINF:9.975, Each segment filefileSequence2.ts#EXTINF:9.9767, preceded by metadatafileSequence3.ts (e.g., duration)#EXTINF:9.975, [...] If no end tag, client#EXT-X-ENDLIST refreshes periodically
  28. 28. <video src="prog_index.m3u8" width="640" height="480"controls autoplay>No video tag support</video>
  29. 29. How is this better than a flat .m4v file?• Streams can provide variants for different bandwidths (as we’ll see…)• Segments make it easier to scrub into the video• Streams can be live video
  30. 30. The “Live” in HLS• A playlist is a live stream if it doesn’t have an #EXT-X-ENDLIST tag• Live playlist will generally just contain the last minute or so of segments• Client will refresh playlist every minute or so, download whatever segments it doesn’t already have, queue them locally• “Live” isn’t really “live” (often a minute behind)
  31. 31. Serving up HLS
  32. 32. mediafilesegmenter• Splits an A/V file into segment files, creates the .m3u8 playlist• Source must be .mov or .m4v with H.264 video, AAC audio• Output segments will be MPEG-2 Transport Stream (.ts) files, or .aac if audio-only• Segment paths are relative, use -b to prepend URL stub
  33. 33. Technical Note TN2224The following audio and video formats are supported:• Video: H.264 Baseline Profile Level 3.0 (iPhone/iPod Touch), Main Profile Level 3.1 (iPad 1,2)• Audio: HE-AAC or AAC-LC up to 48 kHz, stereo audio OR MP3 (MPEG-1 Audio Layer 3) 8 kHz to 48 kHz, stereo audioNote: iPhone 3G supports Baseline Profile Level 3.1. If yourapp runs on older iPhones, however, you should use H.264Baseline Profile 3.0 for compatibility.
  34. 34. Yuna:HTTP Live Streaming tests cadamson$ mediafilesegmenter-f basic source/IMG_0251.MOVJun 24 2012 10:01:24.203: Using floating point is notbackward compatible to iOS 4.1 or earlier devicesJun 24 2012 10:01:24.204: Processing file /Users/cadamson/Documents/HTTP Live Streaming tests/source/IMG_0251.MOVJun 24 2012 10:01:24.338: Finalized /Users/cadamson/Documents/HTTP Live Streaming tests/basic/fileSequence0.tsJun 24 2012 10:01:24.375: segment bitrate 3.78028e+06 isnew maxJun 24 2012 10:01:24.468: Finalized /Users/cadamson/Documents/HTTP Live Streaming tests/basic/fileSequence1.tsJun 24 2012 10:01:24.554: Finalized /Users/cadamson/Documents/HTTP Live Streaming tests/basic/fileSequence2.tsJun 24 2012 10:01:24.631: Finalized /Users/cadamson/Documents/HTTP Live Streaming tests/basic/fileSequence3.tsJun 24 2012 10:01:24.717: Finalized /Users/cadamson/Documents/HTTP Live Streaming tests/basic/fileSequence4.ts
  35. 35. Demo
  36. 36. Variant Playlists• One bitrate does not fit all: Mac on Ethernet versus iPhone on Edge.• Solution: encode your video at multiple bitrates, offer metadata in playlist about whats available, let client figure out which to use.• HLS clients automatically switch to best variant for current network conditions, switch on the fly.
  37. 37. variantplaylistcreator• Creates a playlist that itself points to playlists created with mediafilesegmenter. • Each entry contains metadata describing the bitrate and encoding of the variant.• Tool takes argument pairs: file or URL of a variant .m3u8, and metadata .plist created with mediafilesegmenter -I flag• First entry in variant playlist is default; client will try this one first
  38. 38. Encoding the variants
  39. 39. TN2224Recommendations
  40. 40. Creating variantsYuna:HTTP Live Streaming tests cadamson$mediafilesegmenter -I -f variants/broadband/ source/IMG_0254_Broadband.m4vYuna:HTTP Live Streaming tests cadamson$mediafilesegmenter -I -f variants/wifisource/IMG_0254_WiFi.m4vYuna:HTTP Live Streaming tests cadamson$mediafilesegmenter -I -f variants/cellular source/IMG_0254_Cellular.m4v
  41. 41. Creating variant playlistYuna:HTTP Live Streaming tests cadamson$variantplaylistcreator -o variants/variants.m3u8 variants/broadband/prog_index.m3u8 source/IMG_0254_Broadband.plist variants/wifi/prog_index.m3u8 source/IMG_0254_WiFi.plist variants/cellular/prog_index.m3u8 source/IMG_0254_Cellular.plist
  42. 42. Demo
  43. 43. Thats Great, but…How do we keep people from stealing our stream?
  44. 44. Encryption• HLS encrypts files, not transport.• Easy to scale: still serving flat files, but now theyre useless without decryption keys.• Serving the keys still needs to be secure.• Necessary, but not sufficient, for DRM.
  45. 45. Encrypting a playlistYuna:HTTP Live Streaming tests cadamson$ mediafilesegmenter -I -k keys -f encrypted/cellular source/IMG_0426_Cellular.m4vJun 24 2012 18:59:47.115: Using new key/iv rotation period;this is not backward compatible to iOS 3.1.* or earlierdevices.  Use the "-encrypt-iv=sequence" option forcompatibility with those devices.Jun 24 2012 18:59:47.115: Using floating point is notbackward compatible to iOS 4.1 or earlier devicesJun 24 2012 18:59:47.115: Processing file /Users/cadamson/Documents/HTTP Live Streaming tests/source/IMG_0426_Cellular.m4vJun 24 2012 18:59:47.152: changing IVJun 24 2012 18:59:47.160: Finalized /Users/cadamson/Documents/HTTP Live Streaming tests/encrypted/cellular/fileSequence0.tsJun 24 2012 18:59:47.160: segment bitrate 271257 is new max
  46. 46. Encrypted .ts files
  47. 47. Protect those keys/www/protected /www/hls
  48. 48. Demo
  49. 49. Captions• HLS supports CEA-608 closed captions in the MPEG-2 Transport Stream • If using file segmenter, add a closed- caption track (type clcp) to your source QuickTime .mov • Or use Compressor and Sonic Scenarist
  50. 50. Client-side HLS
  51. 51. Opening an HLS stream• Provide the .m3u8 URL to MPMoviePlayerController or AVPlayer• Add the movie view or layer to your UI, customizing size or scaling if necessary
  52. 52. Create an MPMoviePlayerController// create new movie playerself.moviePlayer = [[MPMoviePlayerController alloc] initWithContentURL:streamURL];[self.moviePlayer prepareToPlay]; • This is the same as playing a local file or any other URL
  53. 53. Add it to your UI[self.moviePlayer.view setFrame: self.movieContainerView.bounds];[self.movieContainerView addSubview: self.moviePlayer.view];self.moviePlayer.scalingMode = MPMovieScalingModeFill; • Can inspect the moviePlayers naturalSize, though it may change during playback (listen for MPMovieNaturalSizeAvailableNotification), or just setFullscreen:animated:
  54. 54. MPMovieScalingMode AspectFit
  55. 55. MPMovieScalingMode AspectFill
  56. 56. MPMovieScalingMode  Fill
  57. 57. Accessing Encrypted Streams• Media Player and AV Foundation can use NSURLCredentials that youve provided• Place credentials in NSURLCredentialStorage• Server can provide the keys securely(*) with HTTP Basic or Digest authentication, an HTTPS script, etc. * - For various values of "secure"
  58. 58. Setting credentials (1)NSURLProtectionSpace *protectionSpace = [[NSURLProtectionSpace alloc] initWithHost:host port:port protocol:protocol realm:realm authenticationMethod: NSURLAuthenticationMethodDefault];
  59. 59. Setting credentials (2)NSURLCredential *credential = [NSURLCredential credentialWithUser:username         password:password persistence: NSURLCredentialPersistenceForSession];[[NSURLCredentialStorage sharedCredentialStorage] setDefaultCredential:credential forProtectionSpace:protectionSpace];
  60. 60. Live Streaming
  61. 61. Live Streaming• mediastreamsegmenter mostly works like the file version, but takes its input from UDP stream or a Unix pipe • Only difference is that .m3u8 file doesnt have an EXT-X-ENDLIST tag, so client reloads periodically to fetch new segments• How the heck do you create a UDP A/V stream?
  62. 62. You Dont• None of Apples tools create the required stream • This is a "third party opportunity" • Which begs the question… buy or build?
  63. 63. Streaming in the Real World Its not all about iPhones…
  64. 64. Streaming Clients• Mobile Devices: iPhone, iPad, iPod Touch… plus Android, Windows Mobile, etc.• Mac and Windows PCs• Over-the-top (OTT) boxes: Apple TV, Roku• Game consoles
  65. 65. Lets Get Practical• What devices do you have to be on?• What devices will you get for free? • HLS is the preferred format for Roku• How to encode and deliver to the devices you need?
  66. 66. HLS Alternatives• Flash still rules on the desktop/browser space, thanks in part to Mozillas obstinance about H.264 in <video> (irony alert: H.264 is the de facto standard for Flash video)• Adobe Dynamic Streaming and Microsoft Smooth Streaming are highly similar to HLS: bitrate-adaptive streams over HTTP
  67. 67. MPEG-DASH• Attempt at a standardized approach to HTTP adaptive-bitrate streaming. ISO/IEC 23009-1. http://xkcd.com/927/
  68. 68. Emerging Consensus• Flash for PCs• HTTP Live Streaming for iOS• Plus whatever other devices you need to support
  69. 69. Real-World HLS• Can you competently encode all your media at all the variant bitrates you need?• Do you have a way to QC all your streams?• Can you handle the server load?
  70. 70. Build or Buy: Services• Provide hosting, transcoding, bandwidth • All-in-one: UStream, LiveStream, Justin.tv / Twitch.tv (all of which have iOS apps) • May provide broadcast tools (Flash applet, Telestream Wirecast, etc. • Often free with ads; you pay to go ad-free, embed on your site, etc.
  71. 71. invalidstreamhttp://www.ustream.tv/channel/invalidstream
  72. 72. Production Demohttp://www.telestream.net/wirecast/
  73. 73. Content Delivery Networkshttp://en.wikipedia.org/wiki/Content_delivery_network
  74. 74. Content Delivery Networks• CDNs host your media on edge servers that are closer to your clients. Less strain on your servers and the backbones.• Examples: Akamai, Limelight, EdgeCast• Big media companies may have their own CDN• Most already know how to do HLS
  75. 75. Buy or Build: Encoders
  76. 76. Buy or Build: Bandwidth• Each HLS client will consume up to 1GB / hour, depending on variant bitrates, client bandwidth, etc.• Many CDNs charge around $0.20/GB.
  77. 77. Bandwidth CostsPrices from ScaleEngine, UStream, and LiveStream as of October 2012
  78. 78. Self-Hosted CostsPrices from MacMiniColo and Amazon EC2 as of March 2013
  79. 79. Wrap Up
  80. 80. Takeaways• HLS is a very practical streaming solution• Only part of the picture if youre multi-platform• Encoding and serving correctly requires some care and expertise, and a lot of money• Client-side software requirements are fairly simple
  81. 81. Q&ASlides and code will be available on my blog: http://www.subfurther.com/blog http://www.slideshare.net/invalidname @invalidname

×