This document discusses building a streaming video app for Apple TV. It covers topics like codecs, containers, livestreaming, adaptive streaming using HTTP Live Streaming (HLS), creating HLS streams with tools like mediafilesegmenter, and securing streams. HLS breaks video into small file segments delivered over HTTP, making streaming scalable and suitable for mobile. Variant playlists allow encoding at multiple bitrates to adapt to network conditions.
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Building A Streaming Apple TV App (CocoaConf San Jose, Nov 2016)
1. Building a Streaming
Apple TV app
Chris Adamson (@invalidname)
CocoaConf San Jose • November, 2016
Slides available at slideshare.net/invalidname
Code available at github.com/invalidstream
10. Digital Media
• Codecs — How we encode (represent) audio or
video in a digital form
• AAC, Apple Lossless, H.264, ProRes
• Containers — How we deliver encoded media
• MP3, .aac, Shoutcast, .mov, .mp4
13. Containers: QuickTime
• Dates back to the early 1990s
• Originally used Mac dual-fork files
• Content agnostic: nearly any media content can
be contained in a QuickTime file
• MPEG-4 adopted QuickTime as basis of its file
format (1998)
15. QuickTime: OK for playback
• Resolve all references into a single file (flatten)
or export (possibly re-encoding) to a file
• Dual-fork approach abandoned to support
QuickTime for Windows (and later, Mac OS X)
16. QuickTime: Bad for
Streaming
• Metadata at end of file meant you needed the
whole file
• First solution: “Quick-Start”, which put metadata
at beginning of file
• Later: QuickTime Streaming Server
• Also-ran to RealPlayer, Windows ASF, Flash
17. Streaming
• Different from progressive download in that the
client does only wants the media for the current
playback time
• Live sources are possible
18. Livestreaming
• Essential to some content types: breaking news,
sports, one-of-a-kind events
• Chat rooms allow interaction with audience
during the program
• It's just fun
• Twitch.tv often has over 1 million
simultaneous viewers
19. Bad old streaming
• Real, ASF, Flash, QTSS – socket-based
connections
• Hard to scale
• Bad for mobile
• Not on port 80; frequently blocked
20. Adaptive streaming
• Send “stream” as a series of small files over
HTTP, expect client to reassemble them
• Easy to scale
• Good for mobile / inconsistent networks
• Runs on port 80; never blocked
21. HTTP Live Streaming
• Apple’s de facto standard for adaptive streaming
• Supported on iOS (iPhone, Apple TV) from day
one
• Also: QuickTime, Safari, Microsoft Edge, Flash,
Android, Roku, etc
• Standards-based competitor: MPEG-DASH
24. Streaming playback
@IBAction func handlePlayButtonTapped(_ sender: AnyObject) {
guard let demoItem = demoItem else { return }
self.playerVC?.player?.pause()
let playerVC = AVPlayerViewController()
playerVC.showsPlaybackControls = true
let player = AVPlayer(url: demoItem.url)
playerVC.player = player
present(playerVC, animated: true)
player.play()
self.playerVC = playerVC
}
This is identical to local file playback
25. HLS, how do you even?
• Source media is split into short (~10 second)
segments, as MPEG-2 transport stream (.ts) files
• iOS 10 / macOS Sierra support “Fragmented
MP4”, which is potentially MPEG-DASH
compatible
• An .m3u8 playlist provides a manifest of files to
be played back
28. mediafilesegmenter
• Takes a source MP4 file and splits it into
segment files and an .m3u8 playlist
• Resulting directory can be hosted on any http
server to provide stream
31. Why is this better than a flat
file?
• Easier to skip around
• Can be encrypted / DRM’ed
• Also, about that bitrate…
32. Variant Playlists
• Allow you to encode at multiple bitrates
• Client determines at runtime if it is keeping up,
and whether to request a different variant for the
next 10 sec segment
• Instead of “buffering” stalls, viewers just see
degraded image/sound quality
44. HLS livestreams
• Exactly the same as the VOD streams we’ve
seen before
• Support variant playlists and encryption
• Difference: the playlist doesn’t have a #EXT-X-
ENDLIST
• Client knows to refresh the playlist periodically
to fetch new segment files
45. mediastreamsegmenter
• Works just like mediafilesegmenter to create the
segment files and a playlist
• Takes as input an MPEG-2 transport stream
• Nothing in Apple’s developer toolchain can
actually create such a stream
46. Streaming servers
• Services that create the stream files from an
input stream you provide
• Can also transcode to lower bitrates and
provide a variant playlist
• Can also transmux to non-HLS formats
• Generally you send them RTMP
50. Wowza
• Top streaming service provider
• Wowza Streaming Engine — runs on your
server (real or virtual)
• Wowza Streaming Cloud — streaming as a
service
55. Wirecast
• Proprietary streaming production software
• Mixes shots, captures from cameras, iOS
devices, or screen, highly customizable
• Prices start at $500
• Watch for a 1/3 off sale on Black Friday
57. Securing Streams
• With -k option, mediafilesegmenter and
mediastreamsegmenter will AES-128 encrypt
segments, using keys from the specified location
• FairPlay Streaming for HLS provides severe
DRM.
• See WWDC 2015 session “Content Protection
for HTTP Live Streaming”
58. Takeaways
• Video streaming is the best thing the Apple TV
does
• HLS streams are just flat files on a web server
• Create them with mediafilesegmenter or a
streaming server
• For lifestreams RTMP to the streaming server
with OBS, Wirecast, etc.
59. Building a Streaming
Apple TV app
Chris Adamson (@invalidname)
CocoaConf San Jose • November, 2016
Slides are available at slideshare.net/invalidname
Code is available at github.com/invalidstream