Lars-Erik from Movi presented the current state of video players, how they're constrained by legacy and how we wrote a video player mobile first, video experience first. https://www.meetup.com/Singapore-Video-Technology/

2. 1. Opening
2. Pandu Kumar
Content Aware Encoding
3. Lars-Erik Ravn
Why Rewrite the Player?
4. Networking

3. Lars-Erik Ravn
Founder
Current mobile players and
Why write a new from scratch?

5. Agenda:
1. Current state of players
2. Why did we rewrite from scratch?

6. Current state
1. Experience with current players
2. Observations within the video industry

7. Why did we rewrite from scratch?
- Better end-user experience (there is a new generation of video consumers
and content - which needs more)
- Giving product owners more creative tools
- Simplify app developer experience making video products
- Dealing with legacy and politics

8. Online video technology industry
- Ingest
- Encode
- Storage
- Distribution
- Playback
- Insights
- Monetizing
- Payments
- Ads

9. Results of your work
- Netflix grade video back-end and CDN’s at the click of a button
- Encoders
- Parallelised encoding
- Both software and hardware
- Dynamic packaging
- Multiple manifests for same assets (HLS, MPEG-Dash)
- On demand transcoding / muxing
- Low latency streaming (LHLS, RT*)

10. More results of your work
- Real time synchronization of live streams
- Better compression
- Lower storage cost
- Smaller data stream
- CDN / Multi CDN

11. What about video players?

12. State of (mobile) video players
- Few options for app developers
- Have become a platform / OS level component
- Mainly tech problem, hard to handle low level graphics and a media pipeline
- Platform manufacturers have their own agenda
- Mainly developed to comply, not to innovate
- Apple, and anyone else developing OS / Browser etc. lagging on H256 because of not
investing in stuff they themselves does not use or see need to invest in right now.
- You will be on their mercy
- Developers are at the mercy of platform support
- Available options are mainly wrappers on the same stack
- This to allow for SLA and support, which you might have encountered is very limited or not
existing with platform offered players

13. Pain points

14. Fragmented / limited capabilities
- Streaming formats (HLS, MPEG-Dash, Smooth Streaming aso.)
- Containers (MP4, MKV, TS aso.)
- Codecs (H264, H265, QV1, VP9, VP10 aso.)
- DRM (Widevine, Fairplay, PlayReady)
- Subtitles (VTT, SSA, SRT, SUB, ASS aso.)
- Functionality (Forced quality on Apple i.e.)

15. Video Player API
- New video player API for each platform you want to target
- Quirks, pitfalls, hacks and workarounds
- Inconsistent reporting of events and data (Metrics and measuring)
- Seemingly simple tasks get complicated to handle
This is why third party wrapped players exists, though that’s kind of expensive and
does make everything lowest common denominator.

16. Seeking in AVPlayer
import AVFoundation
//--------------------------------------
// Option 1: Standard seek using CMTime
//--------------------------------------
let player = AVPlayer()
// I want to seek 10 seconds into the video
// But what is a timescale?!
let time = CMTime( seconds: 10, preferredTimescale: 1 )
// But this never ends up exactly 10 seconds into the
video..
player.seek( to: time )

17. Seeking in AVPlayer
import AVFoundation
//--------------------------------------
// Option 2: Seeking using tolerance
//--------------------------------------
// Zero tolerance! Why isn't this
default?
player.seek( to: time,
toleranceBefore: CMTime.zero,
toleranceAfter: CMTime.zero )

18. Seeking in AVPlayer and Movi Player
import AVFoundation
//--------------------------------------
// Option 2: Seeking using tolerance
//--------------------------------------
// Zero tolerance! Why isn't this
default?
player.seek( to: time,
toleranceBefore: CMTime.zero,
toleranceAfter: CMTime.zero )
import MoviPlayerSDK
let player = MoviPlayer()
player.position = 10.0

19. Looping in AVPlayer
import AVFoundation
//--------------------------------------
// Option 1: Use KVO to seek to the start
//--------------------------------------
let player = AVPlayer()
NotificationCenter.default.addObserver(
forName:
NSNotification.Name.AVPlayerItemDidPlayToEndTime,
object: nil,
queue: nil
){notification in
player.seek( to: kCMTimeZero )
player.play()
}

20. Looping in AVPlayer
import AVFoundation
//-----------------------------------------------
// Option 2: Use AVPlayerLooper
//-----------------------------------------------
var playerLooper: AVPlayerLooper?
// But then I have to use AVQueuePlayer instead of
AVPlayer
let queuePlayer = AVQueuePlayer( url: url )
// and keep a reference to the player looper..
videoLooper = AVPlayerLooper( player: queuePlayer,
templateItem: item )

21. Looping in AVPlayer and Movi Player
import MoviPlayerSDK
let player = MoviPlayer()
player.isLooping = true
import AVFoundation
//-----------------------------------------------
// Option 2: Use AVPlayerLooper
//-----------------------------------------------
var playerLooper: AVPlayerLooper?
// But then I have to use AVQueuePlayer instead of
AVPlayer
let queuePlayer = AVQueuePlayer( url: url )
// and keep a reference to the player looper..
videoLooper = AVPlayerLooper( player: queuePlayer,
templateItem: item )

22. Stopping

23. Stopping in AVPlayer
import AVFoundation
//--------------------------------------
// Option 1: Pause
//--------------------------------------
let player = AVPlayer()
// But this isn't the same as stopping..?
player.pause()

24. Stopping in AVPlayer
import AVFoundation
// Option 2: Release player
//-----------------------------------------------
// But this means my player has to be optional and
mutable..
// and that I have to recreate the player with all its
// settings later..
var player: AVPlayer? = AVPlayer()
player = nil

25. Stopping in AVPlayer
import AVFoundation
// Option 2: Release player
//-----------------------------------------------
// But this means my player has to be optional and mutable..
// and that I have to recreate the player with all its
// settings later..
var player: AVPlayer? = AVPlayer()
player = nil
// Option 3: Set nil player item
// ¯_(ツ)_/¯
player.replaceCurrentItem( with: nil )

26. Stopping in AVPlayer
import AVFoundation
// Option 2: Release player
//-----------------------------------------------
// But this means my player has to be optional and mutable..
// and that I have to recreate the player with all its
// settings later..
var player: AVPlayer? = AVPlayer()
player = nil
import MoviPlayerSDK
let player = MoviPlayer()
player.stop()
// Option 3: Set nil player item
// ¯_(ツ)_/¯
player.replaceCurrentItem( with: nil )

27. Targeting multiple platforms
- Difficult to manage
- Often results in one team per platform
- Multiple streaming endpoints, formats, manifests often needed

28. Experience

33. New generation of consumers and content
Expecting more than a passive and static presentation

34. User experience
- Previous generation human computer interface principle
- Designed to play video at 1x forward (fixed rate)
- Controls resemble DVD player
- Seeking feels like teleporting
- Skip 10 sec, 30 sec etc
- “Don’t touch it - it will break”

35. Innovation is hard
- Developers are boxed in to certain video experiences
- Product owners / UX / UI designers are held back by this
- Currently innovation is happening outside the video player
- Fancy buttons, overlays, chat, social etc.

36. So why is it like this?
- Legacy in what video is and how the medium works
- Comes from advertising way back
- A set storyline and experience pre-composed from A to Z
- Tech has been fueling this concept
- Encoding (Entropy to make transportable on limited bandwidth)
- Storage / formats (Optimised for 1x linear A-Z delivery and playback)
- Streaming / end-points (Manifests are made for 1x A-Z)

37. The spec is deterministic playback
By design - the pipelines are deterministic and static - and all made for a
low-memory footprint - which in all reality is not a bottleneck on any
modern device.
Download Parser Demuxer Decoder Renderer

38. Download Parser Demuxer Decoder Renderer
GOP

39. Our innovation to change players for good!
What drives us when building Movi Player is
- End-user should be able to experience video with creative control
- User experience should leverage capabilities in modern devices
- Development of video experiences should be dead simple
- Developers should be able to choose any tool and framework
- Developers should be able to consume any streaming end-point

40. Modular Cross platform systems
- Extendable to any input and output system
- Extendable to consume any media format (audio, video, subs, rich media)
- C++ makes SDK portable, we can adopt new platforms quickly
- Unity, React, (WebAssembly)

41. Video Player API
- Convenient, unified API for current modern languages and frameworks
- AVPlayer API is written in Objective C and those concepts.
- Not to useful for swift / modern video app developers
- Consistent behavior across all devices
- Functionality, reporting etc
https://movi.ai/docs

42. Overview Movi Player
C API
Movi Player Core C++
Cross Platform
Native
Wrapped Native
C# API
MoviPlayer
macOS
MoviPlayer
tvOS
MoviPlayer
iOS
MoviPlayer
Android
MoviPlayer
Android TV
MoviPlayer
Unity3D
Kotlin / Java APISwift / Obj-C API
JS / Typescript
MoviPlayer
NativeScript
JS / Typescript
MoviPlayer
ReactNative
C#
MoviPlayer
Xamarin
Dart
MoviPlayer
Flutter

43. Leverage platform strengths
- Conform to platform / framework patterns and practices
- Not poor ported libraries
- Rendering (OpenGL, Vulkan, Metal, DirectX)
- Hardware capabilities (Decoders, encoders, input interfaces)
- Native UI controls
- Make it feel and perform native to platform - no bogus wrappers, webviews or poor ports

44. Major breakthroughs from rewrite from scratch
- Interactive VoD and Live streaming experience
- Super fast startup time - 100ms
- HLS, DASH and fMP4 on all platforms
- Unified cross platform features and capabilities
- Unified cross platform API
- Hardware acceleration wherever possible
- Real time viewer attention insights

47. By rewriting from scratch video
feels mobile!
Finally =)

48. Demo

49. Help shape the future of video by joining
Movi Pioneer Program
movi.ai/pioneer