Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building A Streaming Apple TV App (CocoaConf DC, Sept 2016)


Published on

Apple TV offers a friendly SDK, full of familiar view controllers and Foundation classes, with everything an iOS developer needs to develop their own streaming channel. Except for… you know… the streaming part. In this session, we'll look at how Apple's HTTP Live Streaming video works -- from flat files or live sources -- and how to get it from your computer to a streaming server and then to an Apple TV. We'll also look at common challenges for building streaming channel apps, like serving metadata, protecting content, and supporting single sign-on.

Published in: Software
  • Be the first to like this

Building A Streaming Apple TV App (CocoaConf DC, Sept 2016)

  1. 1. Building a Streaming Apple TV app Chris Adamson (@invalidname) CocoaConf DC • September, 2016 Slides will be available at Code will be available at
  2. 2. “The future of TV is apps.” “The future of apps is TV.”
  3. 3. Video (how do you even?)
  4. 4. Digital Media • Codecs — How we encode (represent) audio or video in a digital form • AAC, Apple Lossless, H.264, ProRes • Containers — How we deliver encoded media • MP3, .aac, Shoutcast, .mov, .mp4
  5. 5. Codecs • From “coder / decoder” • Simplest: PCM (audio), M-JPEG (video) • Modern: AAC, H.264/AVC • Next gen: H.265 (HEVC), VP9
  6. 6. Inter-frame compression
  7. 7. Containers: QuickTime • Dates back to the early 1990s • Originally used Mac dual-fork files • Content agnostic: nearly any media content can be contained in a QuickTime file • MPEG-4 adopted QuickTime as basis of its file format (1998)
  8. 8. QuickTime: Great for Editing
  9. 9. QuickTime: OK for playback • Resolve all references into a single file (flatten) or export (possibly re-encoding) to a file • Dual-fork approach abandoned to support QuickTime for Windows (and later, Mac OS X)
  10. 10. QuickTime: Bad for Streaming • Metadata at end of file meant you needed the whole file • First solution: “Quick-Start”, which put metadata at beginning of file • Later: QuickTime Streaming Server • Also-ran to RealPlayer, Windows ASF, Flash
  11. 11. Streaming • Different from progressive download in that the client does only wants the media for the current playback time • Live sources are possible
  12. 12. Bad old streaming • Real, ASF, Flash, QTSS – socket-based connections • Hard to scale • Bad for mobile • Not on port 80; frequently blocked
  13. 13. Adaptive streaming • Send “stream” as a series of small files over HTTP, expect client to reassemble them • Easy to scale • Good for mobile / inconsistent networks • Runs on port 80; never blocked
  14. 14. HTTP Live Streaming • Apple’s de facto standard for adaptive streaming • Supported on iOS (iPhone, Apple TV) from day one • Also: QuickTime, Safari, Microsoft Edge, Flash, Android, Roku, etc • Standards-based competitor: MPEG-DASH
  15. 15. Demo
  16. 16. Streaming playback @IBAction func handlePlayButtonTapped(_ sender: AnyObject) { guard let demoItem = demoItem else { return } self.playerVC?.player?.pause() let playerVC = AVPlayerViewController() playerVC.showsPlaybackControls = true let player = AVPlayer(url: demoItem.url) playerVC.player = player present(playerVC, animated: true) self.playerVC = playerVC } This is identical to local file playback
  17. 17. HLS, how do you even? • Source media is split into short (~10 second) segments, as MPEG-2 transport stream (.ts) files • iOS 10 / macOS Sierra support “Fragmented MP4”, which is potentially MPEG-DASH compatible • An .m3u8 playlist provides a manifest of files to be played back
  18. 18. #EXTM3U #EXT-X-TARGETDURATION:10 #EXT-X-VERSION:3 #EXT-X-MEDIA-SEQUENCE:0 #EXT-X-PLAYLIST-TYPE:VOD #EXTINF:10.00000, fileSequence0.ts #EXTINF:10.00000, fileSequence1.ts #EXTINF:10.00000, fileSequence2.ts #EXTINF:10.00000, fileSequence3.ts #EXTINF:10.00000, fileSequence4.ts #EXTINF:10.00000, fileSequence5.ts #EXTINF:10.00000, fileSequence6.ts #EXTINF:10.00000, fileSequence7.ts #EXTINF:10.00000, fileSequence8.ts #EXTINF:10.00000, fileSequence9.ts #EXTINF:7.40000, fileSequence10.ts #EXT-X-ENDLIST
  19. 19. Streaming Tools
  20. 20. mediafilesegmenter • Takes a source MP4 file and splits it into segment files and an .m3u8 playlist • Resulting directory can be hosted on any http server to provide stream
  21. 21. Demo
  22. 22. Why is this better than a flat file? • Easier to skip around • Can be encrypted / DRM’ed • Also, about that bitrate…
  23. 23. Variant Playlists • Allow you to encode at multiple bitrates • Client determines at runtime if it is keeping up, and whether to request a different variant for the next 10 sec segment • Instead of “buffering” stalls, viewers just see degraded image/sound quality
  24. 24. variantplaylistcreator #EXTM3U #EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1269452,BANDWIDTH=1304420,CODECS="mp4a.40.2, avc1.4d4016",RESOLUTION=640x360,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE wifi-high/prog_index.m3u8 #EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=5172486,BANDWIDTH=5330176,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=960x540,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE broadband-high/prog_index.m3u8 #EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2599260,BANDWIDTH=2677271,CODECS="mp4a.40.2, avc1.4d4016",RESOLUTION=640x360,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE broadband-low/prog_index.m3u8 #EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=757070,BANDWIDTH=782983,CODECS="mp4a.40.2, avc1.42e016",RESOLUTION=640x360,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE wifi-low/prog_index.m3u8 #EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=474503,BANDWIDTH=507149,CODECS="mp4a.40.2, avc1.42e015",RESOLUTION=428x240,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE cellular-high/prog_index.m3u8 #EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=224956,BANDWIDTH=243197,CODECS="mp4a.40.2, avc1.42e015",RESOLUTION=428x240,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE cellular-low/prog_index.m3u8 #EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=33400,BANDWIDTH=33547,CODECS="mp4a. 40.2",CLOSED-CAPTIONS=NONE audio-only/prog_index.m3u8
  25. 25. Demo
  26. 26. Compressor’s HLS bitrates Table 1 Broadband high 5000 Broadband low 2500 Wi-fi high 1250 Wi-fi low 735 Cellular high 450 Cellular low 225 Audio only 32
  27. 27. TN 2224
  28. 28. TN 2224 Bitrates
  29. 29. Livestreaming!
  30. 30. HLS livestreams • Exactly the same as the VOD streams we’ve seen before • Support variant playlists and encryption • Difference: the playlist doesn’t have a #EXT-X- ENDLIST • Client knows to refresh the playlist periodically to fetch new segment files
  31. 31. mediastreamsegmenter • Works just like mediafilesegmenter to create the segment files and a playlist • Takes as input an MPEG-2 transport stream • Nothing in Apple’s developer toolchain can actually create such a stream
  32. 32. Streaming servers • Services that create the stream files from an input stream you provide • Can also transcode to lower bitrates and provide a variant playlist • Can also transmux to non-HLS formats • Generally you send them RTMP
  33. 33. Open Broadcast Software
  34. 34. Demo
  35. 35. Wowza
  36. 36. Wowza • Top streaming service provider • Wowza Streaming Engine — runs on your server (real or virtual) • Wowza Streaming Cloud — streaming as a service
  37. 37. Wirecast
  38. 38. Wirecast • Proprietary streaming production software • Mixes shots, captures from cameras, iOS devices, or screen, highly customizable • Prices start at $500 • Watch for a 1/3 off sale on Black Friday
  39. 39. Demo
  40. 40. Securing Streams • With -k option, mediafilesegmenter and mediastreamsegmenter will AES-128 encrypt segments, using keys from the specified location • FairPlay Streaming for HLS provides severe DRM. • See WWDC 2015 session “Content Protection for HTTP Live Streaming”
  41. 41. Takeaways • Video streaming is the best thing the Apple TV does • HLS streams are just flat files on a web server • Create them with mediafilesegmenter or a streaming server • For lifestreams RTMP to the streaming server with OBS, Wirecast, etc.
  42. 42. Building a Streaming Apple TV app Chris Adamson (@invalidname) CocoaConf DC • September, 2016 Slides will be available at Code will be available at