This document discusses building a streaming video app for Apple TV. It covers topics like codecs, containers, livestreaming, adaptive streaming using HTTP Live Streaming (HLS), creating HLS streams with tools like mediafilesegmenter, and securing streams. HLS breaks video into small file segments delivered over HTTP, making streaming scalable and suitable for mobile. Variant playlists allow encoding at multiple bitrates to adapt to network conditions.

1. Building a Streaming
Apple TV app
Chris Adamson (@invalidname)
CocoaConf San Jose • November, 2016
Slides available at slideshare.net/invalidname
Code available at github.com/invalidstream

6. “The future of TV is apps.”
“The future of apps is TV.”

7. I can haz video app?
Client Backend
Login Authentication
Browse/search CMS
Playback Streaming
Content
Production
Encoding
Monetization

8. Video
(how do you even?)

10. Digital Media
• Codecs — How we encode (represent) audio or
video in a digital form
• AAC, Apple Lossless, H.264, ProRes
• Containers — How we deliver encoded media
• MP3, .aac, Shoutcast, .mov, .mp4

11. Codecs
• From “coder / decoder”
• Simplest: PCM (audio), M-JPEG (video)
• Modern: AAC, H.264/AVC
• Next gen: H.265 (HEVC), VP9

12. Inter-frame compression

13. Containers: QuickTime
• Dates back to the early 1990s
• Originally used Mac dual-fork files
• Content agnostic: nearly any media content can
be contained in a QuickTime file
• MPEG-4 adopted QuickTime as basis of its file
format (1998)

14. QuickTime: Great for Editing

15. QuickTime: OK for playback
• Resolve all references into a single file (flatten)
or export (possibly re-encoding) to a file
• Dual-fork approach abandoned to support
QuickTime for Windows (and later, Mac OS X)

16. QuickTime: Bad for
Streaming
• Metadata at end of file meant you needed the
whole file
• First solution: “Quick-Start”, which put metadata
at beginning of file
• Later: QuickTime Streaming Server
• Also-ran to RealPlayer, Windows ASF, Flash

17. Streaming
• Different from progressive download in that the
client does only wants the media for the current
playback time
• Live sources are possible

18. Livestreaming
• Essential to some content types: breaking news,
sports, one-of-a-kind events
• Chat rooms allow interaction with audience
during the program
• It's just fun
• Twitch.tv often has over 1 million
simultaneous viewers

19. Bad old streaming
• Real, ASF, Flash, QTSS – socket-based
connections
• Hard to scale
• Bad for mobile
• Not on port 80; frequently blocked

20. Adaptive streaming
• Send “stream” as a series of small files over
HTTP, expect client to reassemble them
• Easy to scale
• Good for mobile / inconsistent networks
• Runs on port 80; never blocked

21. HTTP Live Streaming
• Apple’s de facto standard for adaptive streaming
• Supported on iOS (iPhone, Apple TV) from day
one
• Also: QuickTime, Safari, Microsoft Edge, Flash,
Android, Roku, etc
• Standards-based competitor: MPEG-DASH

22. Demo

24. Streaming playback
@IBAction func handlePlayButtonTapped(_ sender: AnyObject) {
guard let demoItem = demoItem else { return }
self.playerVC?.player?.pause()
let playerVC = AVPlayerViewController()
playerVC.showsPlaybackControls = true
let player = AVPlayer(url: demoItem.url)
playerVC.player = player
present(playerVC, animated: true)
player.play()
self.playerVC = playerVC
}
This is identical to local file playback

25. HLS, how do you even?
• Source media is split into short (~10 second)
segments, as MPEG-2 transport stream (.ts) files
• iOS 10 / macOS Sierra support “Fragmented
MP4”, which is potentially MPEG-DASH
compatible
• An .m3u8 playlist provides a manifest of files to
be played back

26. #EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:10.00000,
fileSequence0.ts
#EXTINF:10.00000,
fileSequence1.ts
#EXTINF:10.00000,
fileSequence2.ts
#EXTINF:10.00000,
fileSequence3.ts
#EXTINF:10.00000,
fileSequence4.ts
#EXTINF:10.00000,
fileSequence5.ts
#EXTINF:10.00000,
fileSequence6.ts
#EXTINF:10.00000,
fileSequence7.ts
#EXTINF:10.00000,
fileSequence8.ts
#EXTINF:10.00000,
fileSequence9.ts
#EXTINF:7.40000,
fileSequence10.ts
#EXT-X-ENDLIST

27. Streaming Tools

28. mediafilesegmenter
• Takes a source MP4 file and splits it into
segment files and an .m3u8 playlist
• Resulting directory can be hosted on any http
server to provide stream

29. Demo

31. Why is this better than a flat
file?
• Easier to skip around
• Can be encrypted / DRM’ed
• Also, about that bitrate…

32. Variant Playlists
• Allow you to encode at multiple bitrates
• Client determines at runtime if it is keeping up,
and whether to request a different variant for the
next 10 sec segment
• Instead of “buffering” stalls, viewers just see
degraded image/sound quality

34. variantplaylistcreator
#EXTM3U
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1269452,BANDWIDTH=1304420,CODECS="mp4a.40.2,
avc1.4d4016",RESOLUTION=640x360,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE
wifi-high/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=5172486,BANDWIDTH=5330176,CODECS="mp4a.40.2,
avc1.4d401f",RESOLUTION=960x540,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE
broadband-high/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2599260,BANDWIDTH=2677271,CODECS="mp4a.40.2,
avc1.4d4016",RESOLUTION=640x360,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE
broadband-low/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=757070,BANDWIDTH=782983,CODECS="mp4a.40.2,
avc1.42e016",RESOLUTION=640x360,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE
wifi-low/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=474503,BANDWIDTH=507149,CODECS="mp4a.40.2,
avc1.42e015",RESOLUTION=428x240,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE
cellular-high/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=224956,BANDWIDTH=243197,CODECS="mp4a.40.2,
avc1.42e015",RESOLUTION=428x240,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE
cellular-low/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=33400,BANDWIDTH=33547,CODECS="mp4a.
40.2",CLOSED-CAPTIONS=NONE
audio-only/prog_index.m3u8

35. Demo

38. Compressor’s HLS bitrates
Table 1
Broadband high 5000
Broadband low 2500
Wi-fi high 1250
Wi-fi low 735
Cellular high 450
Cellular low 225
Audio only 32

39. TN 2224
https://developer.apple.com/library/ios/technotes/tn2224/_index.html

40. TN 2224 Bitrates

41. Livestreaming!

44. HLS livestreams
• Exactly the same as the VOD streams we’ve
seen before
• Support variant playlists and encryption
• Difference: the playlist doesn’t have a #EXT-X-
ENDLIST
• Client knows to refresh the playlist periodically
to fetch new segment files

45. mediastreamsegmenter
• Works just like mediafilesegmenter to create the
segment files and a playlist
• Takes as input an MPEG-2 transport stream
• Nothing in Apple’s developer toolchain can
actually create such a stream

46. Streaming servers
• Services that create the stream files from an
input stream you provide
• Can also transcode to lower bitrates and
provide a variant playlist
• Can also transmux to non-HLS formats
• Generally you send them RTMP

47. Open Broadcast Software
https://obsproject.com

48. Demo

49. Wowza
http://wowza.com

50. Wowza
• Top streaming service provider
• Wowza Streaming Engine — runs on your
server (real or virtual)
• Wowza Streaming Cloud — streaming as a
service

54. Wirecast
http://telestream.net/wirecast/

55. Wirecast
• Proprietary streaming production software
• Mixes shots, captures from cameras, iOS
devices, or screen, highly customizable
• Prices start at $500
• Watch for a 1/3 off sale on Black Friday

56. Demo

57. Securing Streams
• With -k option, mediafilesegmenter and
mediastreamsegmenter will AES-128 encrypt
segments, using keys from the specified location
• FairPlay Streaming for HLS provides severe
DRM.
• See WWDC 2015 session “Content Protection
for HTTP Live Streaming”

58. Takeaways
• Video streaming is the best thing the Apple TV
does
• HLS streams are just flat files on a web server
• Create them with mediafilesegmenter or a
streaming server
• For lifestreams RTMP to the streaming server
with OBS, Wirecast, etc.

59. Building a Streaming
Apple TV app
Chris Adamson (@invalidname)
CocoaConf San Jose • November, 2016
Slides are available at slideshare.net/invalidname
Code is available at github.com/invalidstream