Master Video with AV Foundation


Published on

Slides for my Master Video session at Renaissance 2014. This session provided a high-level overview of some of AV Foundation's video playback and editing capabilities.

The demo app for this talk can be found at:

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Master Video with AV Foundation

  1. 1. Renaissance 2014: Master Video Bob McCune @bobmccune
  2. 2. Vatican Square Anything look different?
  3. 3. Vatican Square Anything look different?
  4. 4. Vatican Square Anything look different?
  5. 5. Vatican Square Anything look different?
  6. 6. AV Foundation Overview ‣ Objective-C framework for advanced media processing ‣ ‣ ‣ High performance, asynchronous processing Hardware accelerated handling of AV media Available in its current form since iOS 4 ‣ ‣ Significant additions and enhancements iOS 6 and 7 Apple’s focus for media apps on both iOS and Mac
  7. 7. iOS Media Environment An Embarrassment of Riches AssetsLibrary UIKit MediaPlayer AV  Founda6on CoreAudio CoreMedia CoreVideo CoreAnima6on
  8. 8. Media Assets Understanding Assets AVAsset provides abstract representation of media resource ‣ Abstracts away the format and location AVAssetTrack models the individual media streams within an asset ‣ Tracks are of a uniform type (video, audio, etc.) AVAssetTrack (Video) AVAssetTrack (Audio)
  9. 9. Media Playback
  10. 10. AVPlayer Playback Controller ‣ AVPlayer is a controller for managing playback - pause - ‣ play seekToTime: Use KVO to observe playback readiness and state - ‣ status Timed Observations - addPeriodicTimeObserverForInterval:queue:usingBlock - addBoundaryTimeObserverForInterval:queue:usingBlock
  11. 11. Playing Media Static vs Dynamic Models ‣ AV Foundation distinguishes between static and dynamic aspects of media AVAsset AVAsset AVAssetTrack AVAsset S ta t i c AVPlayerItemTrack AVPlayerItemTrack AVPlayerItem AVPlayerItemTrack Dy n a m ic
  12. 12. Video Playback Playback In Action AVAsset AVAsset AVAsset AVAssetTrack AVPlayerItem AVPlayerItemTrack AVPlayerItemTrack AVPlayerItemTrack AVPlayer
  13. 13. Video Playback Playback In Action AVPlayerItem AVPlayerItemTrack AVPlayerItemTrack AVPlayerItemTrack AVPlayer AVPlayerLayer
  14. 14. Demo
  15. 15. Composing Media
  16. 16. Composing Assets AVComposition ‣ Concrete extension of AVAsset ‣ Composes asset segments on a timeline
  17. 17. Composing Assets Tracks and Segments AVMutableComposition *composition = [AVMutableComposition composition]; AVComposition
  18. 18. Composing Assets Tracks and Segments AVMutableCompositionTrack *videoTrack = [composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:0]; AVMutableCompositionTrack *audioTrack = [composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:1]; AVComposition AVCompositionTrack (Video) AVCompositionTrack (Audio)
  19. 19. Composing Assets Tracks and Segments AVAssetTrack *srcVideoTrack1 = // source video track 1 [videoTrack insertTimeRange:timeRange ofTrack:srcVideoTrack1 atTime:startTime error:&error]; AVAssetTrack *srcVideoTrack2 = // source video track 2 [videoTrack insertTimeRange:timeRange ofTrack:srcVideoTrack2 atTime:startTime error:&error]; AVAssetTrack *srcAudioTrack = // source audio track [audioTrack insertTimeRange:timeRange ofTrack:srcAudioTrack atTime:startTime error:&error]; AVComposition AVCompositionTrack (Video) AVCompositionTrackSegment AVCompositionTrackSegment Seconds 10-30 of “goldengate.m4v” Seconds 20-60 of “wharf.m4v” AVCompositionTrack (Audio) AVCompositionTrackSegment Seconds 0-60 of “soundtrack.m4a”
  20. 20. Demo
  21. 21. Mixing Audio
  22. 22. Audio Mixing AVAudioMix ‣ ‣ Composition tracks play at their natural volume AVAudioMix applies track-level volume adjustments ‣ Composed of AVAudioMixInputParameters ‣ Parameters control individual track volume over time Time Time Range
  23. 23. Demo
  24. 24. Video Transitions
  25. 25. Video Transitions AVVideoComposition AVVideoComposition AVVideoCompositionInstruction AVVideoCompositionInstruction AVVideoCompositionInstruction Defines how two or more video tracks are composited together Configured through collection of composition instructions describing compositing behavior
  26. 26. Video Transitions AVVideoCompositionInstruction AVVideoComposition AVVideoCompositionInstruction AVVideoCompositionInstruction AVVideoCompositionInstruction AVAsset AVAsset AVVideoCompositionLayerInstruction Defines the time range of compositing behavior Composed of layer instructions describing compositing behavior
  27. 27. Video Transitions AVVideoCompositionLayerInstruction AVVideoComposition AVVideoCompositionInstruction AVVideoCompositionInstruction AVVideoCompositionInstruction AVAsset AVAsset AVVideoCompositionLayerInstruction Defines the transform and opacity ramps of input layers Transform and opacity changes modified over given time range
  28. 28. Demo
  29. 29. Layering Content
  30. 30. Layering Content Core Animation Core Animation a natural choice ‣ ‣ High performance, inherently time-based CALayer subclasses used for all video rendering CALayer: used to layer images and text CAAnimation: used to animate layered content CABasicAnimation CAKeyframeAnimation
  31. 31. Animation Timing AVSynchronizedLayer ‣ Core Animation operates on host time ‣ ‣ Starts at boot, marches towards infinity AVSynchronizedLayer Timeline animations need to use movie time ‣ Starts at time zero and runs to duration ‣ ‣ AVPlayerItem Can be started, stopped, rewound, etc. Timing CATextLayer Use AVSynchronizedLayer to use movie time ‣ Confers player item timing on to its sublayer tree CABasicAnimation
  32. 32. Core Animation Timeline vs Realtime Animations ‣ Exactly the same, almost... ‣ Animations with zero beginTime won’t be seen ‣ ‣ Set beginTime = AVCoreAnimationBeginTimeZero Animations removed by default ‣ Set removedOnCompletion = NO ‣ Unable to use CAAnimationGroup?
  33. 33. Demo
  34. 34. Summary Master iOS Video with AV Foundation! ‣ Powerful tools for audio and video playback ‣ ‣ AVPlayer, AVPlayerItem, AVPlayerLayer Powerful tools for composing/editing media: ‣ ‣ AVAudioMix ‣ AVVideoComposition ‣ ‣ AVComposition AVSynchronizedLayer Relatively steep learning curve, but worth the investment!
  35. 35. Resources Presentation Materials Learning AV Foundation Contact Info @bobmccune
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.