SlideShare a Scribd company logo
1 of 43
Introduction to AV
     Foundation
          Chris Adamson
CocoaHeads Ann Arbor — Aug. 12, 2010
Media on iPhone / iOS
iPhone 2 Media
  Frameworks

  Core Audio     Low-level audio streaming




 Media Player     Full-screen video player




                 Obj-C wrapper for audio file
 AV Foundation
                     playback (2.2 only)
iPhone 3 Media
  Frameworks

  Core Audio      Low-level audio streaming




 Media Player    iPod library search/playback




                 Obj-C wrapper for audio file
 AV Foundation
                     playback, recording
iOS 4 Media Frameworks

       Core Audio      Low-level audio streaming


      Media Player   iPod library search/playback


                     Audio/video capture, editing,
     AV Foundation
                         playback, export…

                       Quartz effects on moving
       Core Video
                                images

                     Objects for representing media
      Core Media
                         times, formats, buffers
AV Foundation in iPhone
         2.2
AV Foundation in iPhone
         3.2
AV Foundation in iOS 4
Size is relative


              AV        android.            QuickTime
                                   QT Kit
           Foundation    media               for Java



Classes       56          40        24        576


Methods      460         280       360      >10,000
AV Foundation Classes


• Capture
• Assets and compositions
 • Playback, editing, and export
• Legacy classes
Media Capture
Capture: Old and Busted
• UIImagePickerController
 • Takes user out of your UI
 • Low configurability
 • No capture-time data access
• AVAudioRecorder
 • Audio only, to file only
Capture: New Hotness


• AV Foundation capture classes
 • Highly configurable
 • Live callbacks with capture data
 • Image/video preview to a CALayer
Capture Classes Seem
          Familiar?
       QT Kit                         AV Foundation
QTCaptureAudioPreviewOutput        AVCaptureAudioDataOutput
QTCaptureConnection                AVCaptureConnection
QTCaptureDecompressedAudioOutput   AVCaptureDevice
QTCaptureDecompressedVideoOutput   AVCaptureFileOutput
QTCaptureDevice                    AVCaptureInput
QTCaptureDeviceInput               AVCaptureMovieFileOutput
QTCaptureFileOutput                AVCaptureOutput
QTCaptureInput                     AVCaptureSession
QTCaptureLayer                     AVCaptureStillImageOutput
QTCaptureMovieFileOutput           AVCaptureVideoDataOutput
QTCaptureOutput                    AVCaptureVideoPreviewLayer
QTCaptureSession
QTCaptureVideoPreviewOutput
QTCaptureView
AVCaptureDevice
• Represents an input (camera,
  microphone) or output (speakers)
  device

  • Discover with +[devices], +
    [devicesWithMediaType], +
    [defaultDeviceWithMediaType:], …
• Flash, white balance, exposure, focus
  settings for camera devices
AVCaptureOutput

• A destination for captured data
  • Files: AVCaptureFileOutput,
    AVCaptureMovieFileOutput
  • Images: AVCaptureStillImageOutput
  • Live data:
    AVCaptureAudioDataOutput,
    AVCaptureVideoDataOutput
AVCaptureSession

• Coordinates the activity of audio and
  video capture devices
• Allows you to connect/disconnect
  inputs and outputs
• startRunning/stopRunning
AVCaptureVideoPreview
        Layer


• Subclass of CALayer
• +[layerWithSession:]
Data Output Callbacks

• Audio and video data outputs provide
  -[setSampleBufferDelegate:queue:]

• Delegates get -[captureOutput:​
  didOutputSampleBuffer:​
  fromConnection];
• Sample buffer is a
  CMSampleBufferRef
Core Media

• New in iOS 4
• Core Foundation opaque types for
  wrapping sample buffers, format
  descriptions, time structures

• Functions convert video samples to
  CVImageBuffer, audio to Core Audio
  AudioBufferList
Core Media Time

• CMTime: value, timescale, flags, epoch
• Timescale is n-ths of a second
  • Set timescale to a resolution
    appropriate to your media (e.g.,
    44100 for CD audio). QT convention
    is 600 for video (ask Chris why!)

• CMTimeConvertScale()
WWDC 2010 Session 409

 Using the Camera
 with AV Foundation
 Overview and best practices




 Brad Ford
 iPhone Engineering




                               2
Assets and
Composition
“iMovie is built entirely on exactly the same public API in
AV Foundation that we’re presenting to you in iPhone 4.”
“Boom Box” APIs


• Simple API for playback,
  sometimes recording

• Little or no support for
  editing, mixing, metadata,
  etc.

• Example: HTML 5 <audio> tag
“Streaming” APIs


• Use “stream of audio”
  metaphor

• Strong support for mixing,
  effects, other real-time
  operations

• Example: Core Audio
“Document” APIs

• Use “media document”
  metaphor

• Strong support for editing
• Mixing may be a special case
  of editing

• Example: QuickTime
  and AV Foundation
Assets and Movies


• AVAsset: Collection of tracks
  representing timed media data
• QTMovie: Collection of tracks
  representing timed media data
QuickTime movies
            Movie




    Track           Track




    Media           Media
AVFoundation assets
              Movie




      Track           Track
AVAsset

• Superclass of all “movie”-like
  structures in AVFoundation
• Represents traits of all tracks taken
  together: size, duration
• Build your own with AVURLAsset
AVComposition

• Subclass of AVAsset representing a
  combination of multiple file-based
  assets

• Tracks are AVCompositionTracks
• For editing, AVMutableComposition
  and AVMutableCompositionTracks
Effects 1

• AVAudioMix, AVMutableAudioMix: set
  volumes or audio ramps at specific
  times

• AVVideoCompositionInstructions:
  provide a set of layer-based
  instructions for performing time-
  based opacity or affine transform
  ramps
Playback


• AVPlayer: Playback controller
 • play, pause, seekToTime, etc.
• AVPlayerLayer: CALayer for
  presenting video from an AVLayer
Effects 2


• AVSynchronizedLayer: CALayer that
  synchronizes with a AVPlayerItem’s
  playback timing

• Use for overlays, titles, rendered
  images, Ken Burns effects, etc.
Export
• AVAssetExportSession
  • Must be created with a canned
    preset
• -[exportAsynchronouslyWith​
  CompletionHandler:]
  • Takes a block!
• Exporting CA effects is tricky…
Other stuff

• AVAssetStillImageGenerator: used for
  generating thumbnails

 • Not suitable for getting individual
    frames (but no GetMediaSample()
    equivalent either!)
• NSCoder, NSValue additions for
  wrapping CMTimes, CMTimeRanges
WWDC 2010 Session 407


 Editing Media with AV Foundation
 Overview and best practices




 Eric Lee
 iPhone Engineering




                                    2
Demo
What’s Next?
What’s Next Next?

AVAssetExportSession.h:
extern NSString *const AVAssetExportPreset1280x720
                __OSX_AVAILABLE_STARTING(__MAC_10_7,__IPHONE_4_0);




CMTime.h:
CM_EXPORT const CFStringRef kCMTimeScaleKey
               __OSX_AVAILABLE_STARTING(__MAC_10_7,__IPHONE_4_0);
Contact Info


• http://www.subfurther.com/blog
• @invalidname
• invalidname [at] gmail [dot] com

More Related Content

What's hot

Stupid Video Tricks, CocoaConf Las Vegas
Stupid Video Tricks, CocoaConf Las VegasStupid Video Tricks, CocoaConf Las Vegas
Stupid Video Tricks, CocoaConf Las VegasChris Adamson
 
Utilizing AVFoundation at dubsmash
Utilizing AVFoundation at dubsmashUtilizing AVFoundation at dubsmash
Utilizing AVFoundation at dubsmashGašper Kolenc
 
Video Killed the Rolex Star (CocoaConf Columbus, July 2015)
Video Killed the Rolex Star (CocoaConf Columbus, July 2015)Video Killed the Rolex Star (CocoaConf Columbus, July 2015)
Video Killed the Rolex Star (CocoaConf Columbus, July 2015)Chris Adamson
 
Introduction to the Roku SDK
Introduction to the Roku SDKIntroduction to the Roku SDK
Introduction to the Roku SDKChris Adamson
 
Get On The Audiobus (CocoaConf Atlanta, November 2013)
Get On The Audiobus (CocoaConf Atlanta, November 2013)Get On The Audiobus (CocoaConf Atlanta, November 2013)
Get On The Audiobus (CocoaConf Atlanta, November 2013)Chris Adamson
 
Building A Streaming Apple TV App (CocoaConf San Jose, Nov 2016)
Building A Streaming Apple TV App (CocoaConf San Jose, Nov 2016)Building A Streaming Apple TV App (CocoaConf San Jose, Nov 2016)
Building A Streaming Apple TV App (CocoaConf San Jose, Nov 2016)Chris Adamson
 
iOS Media APIs (MobiDevDay Detroit, May 2013)
iOS Media APIs (MobiDevDay Detroit, May 2013)iOS Media APIs (MobiDevDay Detroit, May 2013)
iOS Media APIs (MobiDevDay Detroit, May 2013)Chris Adamson
 
Get On The Audiobus (CocoaConf Boston, October 2013)
Get On The Audiobus (CocoaConf Boston, October 2013)Get On The Audiobus (CocoaConf Boston, October 2013)
Get On The Audiobus (CocoaConf Boston, October 2013)Chris Adamson
 
Introduction to AV Foundation (CocoaConf, Aug '11)
Introduction to AV Foundation (CocoaConf, Aug '11)Introduction to AV Foundation (CocoaConf, Aug '11)
Introduction to AV Foundation (CocoaConf, Aug '11)Chris Adamson
 
Azure Backup component matrix
Azure Backup component matrixAzure Backup component matrix
Azure Backup component matrixAnthony Clendenen
 
Building A Streaming Apple TV App (CocoaConf DC, Sept 2016)
Building A Streaming Apple TV App (CocoaConf DC, Sept 2016)Building A Streaming Apple TV App (CocoaConf DC, Sept 2016)
Building A Streaming Apple TV App (CocoaConf DC, Sept 2016)Chris Adamson
 
[@NaukriEngineering] Video handlings on apple platforms
[@NaukriEngineering] Video handlings on apple platforms[@NaukriEngineering] Video handlings on apple platforms
[@NaukriEngineering] Video handlings on apple platformsNaukri.com
 
Video Streaming: Broadcast quality on a shoe string budget.
Video Streaming: Broadcast quality on a shoe string budget.  Video Streaming: Broadcast quality on a shoe string budget.
Video Streaming: Broadcast quality on a shoe string budget. netc2012
 
Where Does VMware Integration Occur?
Where Does VMware Integration Occur?Where Does VMware Integration Occur?
Where Does VMware Integration Occur?Scott Lowe
 
Awesomebox tv brief introduction
Awesomebox tv brief introduction Awesomebox tv brief introduction
Awesomebox tv brief introduction 현민 이
 
MediaPlayer Playing Flow
MediaPlayer Playing FlowMediaPlayer Playing Flow
MediaPlayer Playing FlowJavid Hsu
 

What's hot (20)

Stupid Video Tricks, CocoaConf Las Vegas
Stupid Video Tricks, CocoaConf Las VegasStupid Video Tricks, CocoaConf Las Vegas
Stupid Video Tricks, CocoaConf Las Vegas
 
Stupid Video Tricks
Stupid Video TricksStupid Video Tricks
Stupid Video Tricks
 
Utilizing AVFoundation at dubsmash
Utilizing AVFoundation at dubsmashUtilizing AVFoundation at dubsmash
Utilizing AVFoundation at dubsmash
 
Video Killed the Rolex Star (CocoaConf Columbus, July 2015)
Video Killed the Rolex Star (CocoaConf Columbus, July 2015)Video Killed the Rolex Star (CocoaConf Columbus, July 2015)
Video Killed the Rolex Star (CocoaConf Columbus, July 2015)
 
Introduction to the Roku SDK
Introduction to the Roku SDKIntroduction to the Roku SDK
Introduction to the Roku SDK
 
Get On The Audiobus (CocoaConf Atlanta, November 2013)
Get On The Audiobus (CocoaConf Atlanta, November 2013)Get On The Audiobus (CocoaConf Atlanta, November 2013)
Get On The Audiobus (CocoaConf Atlanta, November 2013)
 
Building A Streaming Apple TV App (CocoaConf San Jose, Nov 2016)
Building A Streaming Apple TV App (CocoaConf San Jose, Nov 2016)Building A Streaming Apple TV App (CocoaConf San Jose, Nov 2016)
Building A Streaming Apple TV App (CocoaConf San Jose, Nov 2016)
 
iOS Media APIs (MobiDevDay Detroit, May 2013)
iOS Media APIs (MobiDevDay Detroit, May 2013)iOS Media APIs (MobiDevDay Detroit, May 2013)
iOS Media APIs (MobiDevDay Detroit, May 2013)
 
Get On The Audiobus (CocoaConf Boston, October 2013)
Get On The Audiobus (CocoaConf Boston, October 2013)Get On The Audiobus (CocoaConf Boston, October 2013)
Get On The Audiobus (CocoaConf Boston, October 2013)
 
Introduction to AV Foundation (CocoaConf, Aug '11)
Introduction to AV Foundation (CocoaConf, Aug '11)Introduction to AV Foundation (CocoaConf, Aug '11)
Introduction to AV Foundation (CocoaConf, Aug '11)
 
HTML5 Audio & Video
HTML5 Audio & VideoHTML5 Audio & Video
HTML5 Audio & Video
 
Azure Backup component matrix
Azure Backup component matrixAzure Backup component matrix
Azure Backup component matrix
 
Building A Streaming Apple TV App (CocoaConf DC, Sept 2016)
Building A Streaming Apple TV App (CocoaConf DC, Sept 2016)Building A Streaming Apple TV App (CocoaConf DC, Sept 2016)
Building A Streaming Apple TV App (CocoaConf DC, Sept 2016)
 
[@NaukriEngineering] Video handlings on apple platforms
[@NaukriEngineering] Video handlings on apple platforms[@NaukriEngineering] Video handlings on apple platforms
[@NaukriEngineering] Video handlings on apple platforms
 
English14
English14English14
English14
 
Video Streaming: Broadcast quality on a shoe string budget.
Video Streaming: Broadcast quality on a shoe string budget.  Video Streaming: Broadcast quality on a shoe string budget.
Video Streaming: Broadcast quality on a shoe string budget.
 
Where Does VMware Integration Occur?
Where Does VMware Integration Occur?Where Does VMware Integration Occur?
Where Does VMware Integration Occur?
 
Awesomebox tv brief introduction
Awesomebox tv brief introduction Awesomebox tv brief introduction
Awesomebox tv brief introduction
 
How to Hack Edison
How to Hack EdisonHow to Hack Edison
How to Hack Edison
 
MediaPlayer Playing Flow
MediaPlayer Playing FlowMediaPlayer Playing Flow
MediaPlayer Playing Flow
 

Viewers also liked

Differentiated Instruction Made Simple
Differentiated Instruction Made SimpleDifferentiated Instruction Made Simple
Differentiated Instruction Made SimpleJennifer Dorman
 
1 visual comunication pptx
1 visual comunication pptx1 visual comunication pptx
1 visual comunication pptxasun camarasa
 
Drawing with Quartz on iOS
Drawing with Quartz on iOSDrawing with Quartz on iOS
Drawing with Quartz on iOSBob McCune
 
Water hammer in pipe_AMIT
Water hammer in pipe_AMITWater hammer in pipe_AMIT
Water hammer in pipe_AMITAmit Sharma
 
Soil structure interaction amec presentation-final
Soil structure interaction amec presentation-finalSoil structure interaction amec presentation-final
Soil structure interaction amec presentation-finalAhmad Hallak PEng
 
Forward Swift 2017: Media Frameworks and Swift: This Is Fine
Forward Swift 2017: Media Frameworks and Swift: This Is FineForward Swift 2017: Media Frameworks and Swift: This Is Fine
Forward Swift 2017: Media Frameworks and Swift: This Is FineChris Adamson
 
Earthquake Resistant Design Techniques
Earthquake Resistant Design TechniquesEarthquake Resistant Design Techniques
Earthquake Resistant Design TechniquesSahil Kaundal
 
PPT ON TRENCHLESS TECHNOLOGY
PPT ON TRENCHLESS TECHNOLOGY PPT ON TRENCHLESS TECHNOLOGY
PPT ON TRENCHLESS TECHNOLOGY MD AMANULLAH
 
Chapter 06 boolean algebra
Chapter 06 boolean algebraChapter 06 boolean algebra
Chapter 06 boolean algebraHareem Aslam
 
Ch4 Boolean Algebra And Logic Simplication1
Ch4 Boolean Algebra And Logic Simplication1Ch4 Boolean Algebra And Logic Simplication1
Ch4 Boolean Algebra And Logic Simplication1Qundeel
 
Digital logic gates and Boolean algebra
Digital logic gates and Boolean algebraDigital logic gates and Boolean algebra
Digital logic gates and Boolean algebraSARITHA REDDY
 
multimedia technologies Introduction
multimedia technologies Introductionmultimedia technologies Introduction
multimedia technologies IntroductionMohammed Fareed
 
Boolean algebra
Boolean algebraBoolean algebra
Boolean algebraGagan Deep
 
Introduction to remote sensing and gis
Introduction to remote sensing and gisIntroduction to remote sensing and gis
Introduction to remote sensing and gisMohsin Siddique
 

Viewers also liked (20)

Es ppt
Es pptEs ppt
Es ppt
 
Differentiated Instruction Made Simple
Differentiated Instruction Made SimpleDifferentiated Instruction Made Simple
Differentiated Instruction Made Simple
 
1 visual comunication pptx
1 visual comunication pptx1 visual comunication pptx
1 visual comunication pptx
 
Drawing with Quartz on iOS
Drawing with Quartz on iOSDrawing with Quartz on iOS
Drawing with Quartz on iOS
 
Water hammer in pipe_AMIT
Water hammer in pipe_AMITWater hammer in pipe_AMIT
Water hammer in pipe_AMIT
 
Rtsp
RtspRtsp
Rtsp
 
TRENCHLESS TECHNOLOGY
TRENCHLESS TECHNOLOGYTRENCHLESS TECHNOLOGY
TRENCHLESS TECHNOLOGY
 
Soil structure interaction amec presentation-final
Soil structure interaction amec presentation-finalSoil structure interaction amec presentation-final
Soil structure interaction amec presentation-final
 
Forward Swift 2017: Media Frameworks and Swift: This Is Fine
Forward Swift 2017: Media Frameworks and Swift: This Is FineForward Swift 2017: Media Frameworks and Swift: This Is Fine
Forward Swift 2017: Media Frameworks and Swift: This Is Fine
 
Earthquake Resistant Design Techniques
Earthquake Resistant Design TechniquesEarthquake Resistant Design Techniques
Earthquake Resistant Design Techniques
 
Remote sensing
Remote sensing   Remote sensing
Remote sensing
 
GIS in Health
GIS in HealthGIS in Health
GIS in Health
 
PPT ON TRENCHLESS TECHNOLOGY
PPT ON TRENCHLESS TECHNOLOGY PPT ON TRENCHLESS TECHNOLOGY
PPT ON TRENCHLESS TECHNOLOGY
 
Chapter 06 boolean algebra
Chapter 06 boolean algebraChapter 06 boolean algebra
Chapter 06 boolean algebra
 
Ch4 Boolean Algebra And Logic Simplication1
Ch4 Boolean Algebra And Logic Simplication1Ch4 Boolean Algebra And Logic Simplication1
Ch4 Boolean Algebra And Logic Simplication1
 
Digital logic gates and Boolean algebra
Digital logic gates and Boolean algebraDigital logic gates and Boolean algebra
Digital logic gates and Boolean algebra
 
multimedia technologies Introduction
multimedia technologies Introductionmultimedia technologies Introduction
multimedia technologies Introduction
 
Boolean algebra
Boolean algebraBoolean algebra
Boolean algebra
 
Introduction to remote sensing and gis
Introduction to remote sensing and gisIntroduction to remote sensing and gis
Introduction to remote sensing and gis
 
Boolean algebra
Boolean algebraBoolean algebra
Boolean algebra
 

Similar to Introduction to AV Foundation

Glitch-Free A/V Encoding (CocoaConf Boston, October 2013)
Glitch-Free A/V Encoding (CocoaConf Boston, October 2013)Glitch-Free A/V Encoding (CocoaConf Boston, October 2013)
Glitch-Free A/V Encoding (CocoaConf Boston, October 2013)Chris Adamson
 
How to make a video - Part 2: Post-Production Basics
How to make a video - Part 2: Post-Production BasicsHow to make a video - Part 2: Post-Production Basics
How to make a video - Part 2: Post-Production BasicsKris Brewer
 
[AWS Media Symposium 2019] AWS Media Services Innovation - Christer Whitehorn...
[AWS Media Symposium 2019] AWS Media Services Innovation - Christer Whitehorn...[AWS Media Symposium 2019] AWS Media Services Innovation - Christer Whitehorn...
[AWS Media Symposium 2019] AWS Media Services Innovation - Christer Whitehorn...Amazon Web Services Korea
 
what_is_a_codec_2010
what_is_a_codec_2010what_is_a_codec_2010
what_is_a_codec_2010Justin Giles
 
Abekas Mira Production Server
Abekas Mira Production ServerAbekas Mira Production Server
Abekas Mira Production ServerAbekasVideo
 
dat-Post-Producer-final
dat-Post-Producer-finaldat-Post-Producer-final
dat-Post-Producer-finalScott Matics
 
AWS powered online classes platform
AWS powered online classes platformAWS powered online classes platform
AWS powered online classes platformUjjavalVerma4
 
20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert
20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert
20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvertAmazon Web Services Japan
 
Janus Workshop @ ClueCon 2020
Janus Workshop @ ClueCon 2020Janus Workshop @ ClueCon 2020
Janus Workshop @ ClueCon 2020Lorenzo Miniero
 
Audio and video streaming
Audio and video streamingAudio and video streaming
Audio and video streamingRohan Bhatkar
 
Building a Video Encoding Pipeline at The New York Times
Building a Video Encoding Pipeline at The New York TimesBuilding a Video Encoding Pipeline at The New York Times
Building a Video Encoding Pipeline at The New York TimesFlávio Ribeiro
 
yapi.js introduction (mopcon 2016 version)
yapi.js introduction (mopcon 2016 version)yapi.js introduction (mopcon 2016 version)
yapi.js introduction (mopcon 2016 version)Jesse (Chien Chen) Chen
 
Streaming Media over the Internet
Streaming Media over the InternetStreaming Media over the Internet
Streaming Media over the InternetVideoguy
 
Continuous Automated Deployment with Apache ACE
Continuous Automated Deployment with Apache ACEContinuous Automated Deployment with Apache ACE
Continuous Automated Deployment with Apache ACEJan Willem Janssen
 
(SPOT209) Raising the Bar on Video Streaming Quality Using AWS
(SPOT209) Raising the Bar on Video Streaming Quality Using AWS(SPOT209) Raising the Bar on Video Streaming Quality Using AWS
(SPOT209) Raising the Bar on Video Streaming Quality Using AWSAmazon Web Services
 
KKBOX WWDC17 Airplay 2 - Dolphin
KKBOX WWDC17 Airplay 2 - DolphinKKBOX WWDC17 Airplay 2 - Dolphin
KKBOX WWDC17 Airplay 2 - DolphinLiyao Chen
 

Similar to Introduction to AV Foundation (20)

Apan media encoding
Apan media encodingApan media encoding
Apan media encoding
 
Video Meets Documentation
Video Meets DocumentationVideo Meets Documentation
Video Meets Documentation
 
Glitch-Free A/V Encoding (CocoaConf Boston, October 2013)
Glitch-Free A/V Encoding (CocoaConf Boston, October 2013)Glitch-Free A/V Encoding (CocoaConf Boston, October 2013)
Glitch-Free A/V Encoding (CocoaConf Boston, October 2013)
 
How to make a video - Part 2: Post-Production Basics
How to make a video - Part 2: Post-Production BasicsHow to make a video - Part 2: Post-Production Basics
How to make a video - Part 2: Post-Production Basics
 
[AWS Media Symposium 2019] AWS Media Services Innovation - Christer Whitehorn...
[AWS Media Symposium 2019] AWS Media Services Innovation - Christer Whitehorn...[AWS Media Symposium 2019] AWS Media Services Innovation - Christer Whitehorn...
[AWS Media Symposium 2019] AWS Media Services Innovation - Christer Whitehorn...
 
what_is_a_codec_2010
what_is_a_codec_2010what_is_a_codec_2010
what_is_a_codec_2010
 
Abekas Mira Production Server
Abekas Mira Production ServerAbekas Mira Production Server
Abekas Mira Production Server
 
dat-Post-Producer-final
dat-Post-Producer-finaldat-Post-Producer-final
dat-Post-Producer-final
 
Cinemarkup
CinemarkupCinemarkup
Cinemarkup
 
AWS powered online classes platform
AWS powered online classes platformAWS powered online classes platform
AWS powered online classes platform
 
20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert
20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert
20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert
 
Janus Workshop @ ClueCon 2020
Janus Workshop @ ClueCon 2020Janus Workshop @ ClueCon 2020
Janus Workshop @ ClueCon 2020
 
Audio and video streaming
Audio and video streamingAudio and video streaming
Audio and video streaming
 
Building a Video Encoding Pipeline at The New York Times
Building a Video Encoding Pipeline at The New York TimesBuilding a Video Encoding Pipeline at The New York Times
Building a Video Encoding Pipeline at The New York Times
 
yapi.js introduction (mopcon 2016 version)
yapi.js introduction (mopcon 2016 version)yapi.js introduction (mopcon 2016 version)
yapi.js introduction (mopcon 2016 version)
 
Chapter 3.pptx
Chapter 3.pptxChapter 3.pptx
Chapter 3.pptx
 
Streaming Media over the Internet
Streaming Media over the InternetStreaming Media over the Internet
Streaming Media over the Internet
 
Continuous Automated Deployment with Apache ACE
Continuous Automated Deployment with Apache ACEContinuous Automated Deployment with Apache ACE
Continuous Automated Deployment with Apache ACE
 
(SPOT209) Raising the Bar on Video Streaming Quality Using AWS
(SPOT209) Raising the Bar on Video Streaming Quality Using AWS(SPOT209) Raising the Bar on Video Streaming Quality Using AWS
(SPOT209) Raising the Bar on Video Streaming Quality Using AWS
 
KKBOX WWDC17 Airplay 2 - Dolphin
KKBOX WWDC17 Airplay 2 - DolphinKKBOX WWDC17 Airplay 2 - Dolphin
KKBOX WWDC17 Airplay 2 - Dolphin
 

More from Chris Adamson

Whatever Happened to Visual Novel Anime? (AWA/Youmacon 2018)
Whatever Happened to Visual Novel Anime? (AWA/Youmacon 2018)Whatever Happened to Visual Novel Anime? (AWA/Youmacon 2018)
Whatever Happened to Visual Novel Anime? (AWA/Youmacon 2018)Chris Adamson
 
Whatever Happened to Visual Novel Anime? (JAFAX 2018)
Whatever Happened to Visual Novel Anime? (JAFAX 2018)Whatever Happened to Visual Novel Anime? (JAFAX 2018)
Whatever Happened to Visual Novel Anime? (JAFAX 2018)Chris Adamson
 
Media Frameworks Versus Swift (Swift by Northwest, October 2017)
Media Frameworks Versus Swift (Swift by Northwest, October 2017)Media Frameworks Versus Swift (Swift by Northwest, October 2017)
Media Frameworks Versus Swift (Swift by Northwest, October 2017)Chris Adamson
 
Fall Premieres: Media Frameworks in iOS 11, macOS 10.13, and tvOS 11 (CocoaCo...
Fall Premieres: Media Frameworks in iOS 11, macOS 10.13, and tvOS 11 (CocoaCo...Fall Premieres: Media Frameworks in iOS 11, macOS 10.13, and tvOS 11 (CocoaCo...
Fall Premieres: Media Frameworks in iOS 11, macOS 10.13, and tvOS 11 (CocoaCo...Chris Adamson
 
CocoaConf Chicago 2017: Media Frameworks and Swift: This Is Fine
CocoaConf Chicago 2017: Media Frameworks and Swift: This Is FineCocoaConf Chicago 2017: Media Frameworks and Swift: This Is Fine
CocoaConf Chicago 2017: Media Frameworks and Swift: This Is FineChris Adamson
 
Firebase: Totally Not Parse All Over Again (Unless It Is) (CocoaConf San Jose...
Firebase: Totally Not Parse All Over Again (Unless It Is) (CocoaConf San Jose...Firebase: Totally Not Parse All Over Again (Unless It Is) (CocoaConf San Jose...
Firebase: Totally Not Parse All Over Again (Unless It Is) (CocoaConf San Jose...Chris Adamson
 
Firebase: Totally Not Parse All Over Again (Unless It Is)
Firebase: Totally Not Parse All Over Again (Unless It Is)Firebase: Totally Not Parse All Over Again (Unless It Is)
Firebase: Totally Not Parse All Over Again (Unless It Is)Chris Adamson
 
Revenge of the 80s: Cut/Copy/Paste, Undo/Redo, and More Big Hits (CocoaConf C...
Revenge of the 80s: Cut/Copy/Paste, Undo/Redo, and More Big Hits (CocoaConf C...Revenge of the 80s: Cut/Copy/Paste, Undo/Redo, and More Big Hits (CocoaConf C...
Revenge of the 80s: Cut/Copy/Paste, Undo/Redo, and More Big Hits (CocoaConf C...Chris Adamson
 
Core Image: The Most Fun API You're Not Using, CocoaConf Atlanta, December 2014
Core Image: The Most Fun API You're Not Using, CocoaConf Atlanta, December 2014Core Image: The Most Fun API You're Not Using, CocoaConf Atlanta, December 2014
Core Image: The Most Fun API You're Not Using, CocoaConf Atlanta, December 2014Chris Adamson
 
Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)
Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)
Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)Chris Adamson
 
Core Audio in iOS 6 (CocoaConf San Jose, April 2013)
Core Audio in iOS 6 (CocoaConf San Jose, April 2013) Core Audio in iOS 6 (CocoaConf San Jose, April 2013)
Core Audio in iOS 6 (CocoaConf San Jose, April 2013) Chris Adamson
 
Core Audio in iOS 6 (CocoaConf DC, March 2013)
Core Audio in iOS 6 (CocoaConf DC, March 2013)Core Audio in iOS 6 (CocoaConf DC, March 2013)
Core Audio in iOS 6 (CocoaConf DC, March 2013)Chris Adamson
 
Mobile Movies with HTTP Live Streaming (CocoaConf DC, March 2013)
Mobile Movies with HTTP Live Streaming (CocoaConf DC, March 2013)Mobile Movies with HTTP Live Streaming (CocoaConf DC, March 2013)
Mobile Movies with HTTP Live Streaming (CocoaConf DC, March 2013)Chris Adamson
 
Core Audio in iOS 6 (CocoaConf Chicago, March 2013)
Core Audio in iOS 6 (CocoaConf Chicago, March 2013)Core Audio in iOS 6 (CocoaConf Chicago, March 2013)
Core Audio in iOS 6 (CocoaConf Chicago, March 2013)Chris Adamson
 
Core Audio Intro (Detroit Mobile City 2013)
Core Audio Intro (Detroit Mobile City 2013)Core Audio Intro (Detroit Mobile City 2013)
Core Audio Intro (Detroit Mobile City 2013)Chris Adamson
 
Objective-C Is Not Java
Objective-C Is Not JavaObjective-C Is Not Java
Objective-C Is Not JavaChris Adamson
 

More from Chris Adamson (16)

Whatever Happened to Visual Novel Anime? (AWA/Youmacon 2018)
Whatever Happened to Visual Novel Anime? (AWA/Youmacon 2018)Whatever Happened to Visual Novel Anime? (AWA/Youmacon 2018)
Whatever Happened to Visual Novel Anime? (AWA/Youmacon 2018)
 
Whatever Happened to Visual Novel Anime? (JAFAX 2018)
Whatever Happened to Visual Novel Anime? (JAFAX 2018)Whatever Happened to Visual Novel Anime? (JAFAX 2018)
Whatever Happened to Visual Novel Anime? (JAFAX 2018)
 
Media Frameworks Versus Swift (Swift by Northwest, October 2017)
Media Frameworks Versus Swift (Swift by Northwest, October 2017)Media Frameworks Versus Swift (Swift by Northwest, October 2017)
Media Frameworks Versus Swift (Swift by Northwest, October 2017)
 
Fall Premieres: Media Frameworks in iOS 11, macOS 10.13, and tvOS 11 (CocoaCo...
Fall Premieres: Media Frameworks in iOS 11, macOS 10.13, and tvOS 11 (CocoaCo...Fall Premieres: Media Frameworks in iOS 11, macOS 10.13, and tvOS 11 (CocoaCo...
Fall Premieres: Media Frameworks in iOS 11, macOS 10.13, and tvOS 11 (CocoaCo...
 
CocoaConf Chicago 2017: Media Frameworks and Swift: This Is Fine
CocoaConf Chicago 2017: Media Frameworks and Swift: This Is FineCocoaConf Chicago 2017: Media Frameworks and Swift: This Is Fine
CocoaConf Chicago 2017: Media Frameworks and Swift: This Is Fine
 
Firebase: Totally Not Parse All Over Again (Unless It Is) (CocoaConf San Jose...
Firebase: Totally Not Parse All Over Again (Unless It Is) (CocoaConf San Jose...Firebase: Totally Not Parse All Over Again (Unless It Is) (CocoaConf San Jose...
Firebase: Totally Not Parse All Over Again (Unless It Is) (CocoaConf San Jose...
 
Firebase: Totally Not Parse All Over Again (Unless It Is)
Firebase: Totally Not Parse All Over Again (Unless It Is)Firebase: Totally Not Parse All Over Again (Unless It Is)
Firebase: Totally Not Parse All Over Again (Unless It Is)
 
Revenge of the 80s: Cut/Copy/Paste, Undo/Redo, and More Big Hits (CocoaConf C...
Revenge of the 80s: Cut/Copy/Paste, Undo/Redo, and More Big Hits (CocoaConf C...Revenge of the 80s: Cut/Copy/Paste, Undo/Redo, and More Big Hits (CocoaConf C...
Revenge of the 80s: Cut/Copy/Paste, Undo/Redo, and More Big Hits (CocoaConf C...
 
Core Image: The Most Fun API You're Not Using, CocoaConf Atlanta, December 2014
Core Image: The Most Fun API You're Not Using, CocoaConf Atlanta, December 2014Core Image: The Most Fun API You're Not Using, CocoaConf Atlanta, December 2014
Core Image: The Most Fun API You're Not Using, CocoaConf Atlanta, December 2014
 
Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)
Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)
Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)
 
Core Audio in iOS 6 (CocoaConf San Jose, April 2013)
Core Audio in iOS 6 (CocoaConf San Jose, April 2013) Core Audio in iOS 6 (CocoaConf San Jose, April 2013)
Core Audio in iOS 6 (CocoaConf San Jose, April 2013)
 
Core Audio in iOS 6 (CocoaConf DC, March 2013)
Core Audio in iOS 6 (CocoaConf DC, March 2013)Core Audio in iOS 6 (CocoaConf DC, March 2013)
Core Audio in iOS 6 (CocoaConf DC, March 2013)
 
Mobile Movies with HTTP Live Streaming (CocoaConf DC, March 2013)
Mobile Movies with HTTP Live Streaming (CocoaConf DC, March 2013)Mobile Movies with HTTP Live Streaming (CocoaConf DC, March 2013)
Mobile Movies with HTTP Live Streaming (CocoaConf DC, March 2013)
 
Core Audio in iOS 6 (CocoaConf Chicago, March 2013)
Core Audio in iOS 6 (CocoaConf Chicago, March 2013)Core Audio in iOS 6 (CocoaConf Chicago, March 2013)
Core Audio in iOS 6 (CocoaConf Chicago, March 2013)
 
Core Audio Intro (Detroit Mobile City 2013)
Core Audio Intro (Detroit Mobile City 2013)Core Audio Intro (Detroit Mobile City 2013)
Core Audio Intro (Detroit Mobile City 2013)
 
Objective-C Is Not Java
Objective-C Is Not JavaObjective-C Is Not Java
Objective-C Is Not Java
 

Recently uploaded

Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...amber724300
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
Dynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationDynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationBuild Intuit
 
A PowerPoint Presentation on Vikram Lander pptx
A PowerPoint Presentation on Vikram Lander pptxA PowerPoint Presentation on Vikram Lander pptx
A PowerPoint Presentation on Vikram Lander pptxatharvdev2010
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...BookNet Canada
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentMahmoud Rabie
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Jeffrey Haguewood
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024BookNet Canada
 
Women in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationWomen in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationDianaGray10
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...Karmanjay Verma
 
Automation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions managementAutomation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions managementDianaGray10
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 

Recently uploaded (20)

Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
Dynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationDynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientation
 
A PowerPoint Presentation on Vikram Lander pptx
A PowerPoint Presentation on Vikram Lander pptxA PowerPoint Presentation on Vikram Lander pptx
A PowerPoint Presentation on Vikram Lander pptx
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career Development
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
 
Women in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationWomen in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automation
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
 
Automation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions managementAutomation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions management
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 

Introduction to AV Foundation

  • 1. Introduction to AV Foundation Chris Adamson CocoaHeads Ann Arbor — Aug. 12, 2010
  • 3. iPhone 2 Media Frameworks Core Audio Low-level audio streaming Media Player Full-screen video player Obj-C wrapper for audio file AV Foundation playback (2.2 only)
  • 4. iPhone 3 Media Frameworks Core Audio Low-level audio streaming Media Player iPod library search/playback Obj-C wrapper for audio file AV Foundation playback, recording
  • 5. iOS 4 Media Frameworks Core Audio Low-level audio streaming Media Player iPod library search/playback Audio/video capture, editing, AV Foundation playback, export… Quartz effects on moving Core Video images Objects for representing media Core Media times, formats, buffers
  • 6. AV Foundation in iPhone 2.2
  • 7. AV Foundation in iPhone 3.2
  • 9. Size is relative AV android. QuickTime QT Kit Foundation media for Java Classes 56 40 24 576 Methods 460 280 360 >10,000
  • 10. AV Foundation Classes • Capture • Assets and compositions • Playback, editing, and export • Legacy classes
  • 12. Capture: Old and Busted • UIImagePickerController • Takes user out of your UI • Low configurability • No capture-time data access • AVAudioRecorder • Audio only, to file only
  • 13. Capture: New Hotness • AV Foundation capture classes • Highly configurable • Live callbacks with capture data • Image/video preview to a CALayer
  • 14. Capture Classes Seem Familiar? QT Kit AV Foundation QTCaptureAudioPreviewOutput AVCaptureAudioDataOutput QTCaptureConnection AVCaptureConnection QTCaptureDecompressedAudioOutput AVCaptureDevice QTCaptureDecompressedVideoOutput AVCaptureFileOutput QTCaptureDevice AVCaptureInput QTCaptureDeviceInput AVCaptureMovieFileOutput QTCaptureFileOutput AVCaptureOutput QTCaptureInput AVCaptureSession QTCaptureLayer AVCaptureStillImageOutput QTCaptureMovieFileOutput AVCaptureVideoDataOutput QTCaptureOutput AVCaptureVideoPreviewLayer QTCaptureSession QTCaptureVideoPreviewOutput QTCaptureView
  • 15. AVCaptureDevice • Represents an input (camera, microphone) or output (speakers) device • Discover with +[devices], + [devicesWithMediaType], + [defaultDeviceWithMediaType:], … • Flash, white balance, exposure, focus settings for camera devices
  • 16. AVCaptureOutput • A destination for captured data • Files: AVCaptureFileOutput, AVCaptureMovieFileOutput • Images: AVCaptureStillImageOutput • Live data: AVCaptureAudioDataOutput, AVCaptureVideoDataOutput
  • 17. AVCaptureSession • Coordinates the activity of audio and video capture devices • Allows you to connect/disconnect inputs and outputs • startRunning/stopRunning
  • 18. AVCaptureVideoPreview Layer • Subclass of CALayer • +[layerWithSession:]
  • 19. Data Output Callbacks • Audio and video data outputs provide -[setSampleBufferDelegate:queue:] • Delegates get -[captureOutput:​ didOutputSampleBuffer:​ fromConnection]; • Sample buffer is a CMSampleBufferRef
  • 20. Core Media • New in iOS 4 • Core Foundation opaque types for wrapping sample buffers, format descriptions, time structures • Functions convert video samples to CVImageBuffer, audio to Core Audio AudioBufferList
  • 21. Core Media Time • CMTime: value, timescale, flags, epoch • Timescale is n-ths of a second • Set timescale to a resolution appropriate to your media (e.g., 44100 for CD audio). QT convention is 600 for video (ask Chris why!) • CMTimeConvertScale()
  • 22. WWDC 2010 Session 409 Using the Camera with AV Foundation Overview and best practices Brad Ford iPhone Engineering 2
  • 24. “iMovie is built entirely on exactly the same public API in AV Foundation that we’re presenting to you in iPhone 4.”
  • 25.
  • 26. “Boom Box” APIs • Simple API for playback, sometimes recording • Little or no support for editing, mixing, metadata, etc. • Example: HTML 5 <audio> tag
  • 27. “Streaming” APIs • Use “stream of audio” metaphor • Strong support for mixing, effects, other real-time operations • Example: Core Audio
  • 28. “Document” APIs • Use “media document” metaphor • Strong support for editing • Mixing may be a special case of editing • Example: QuickTime and AV Foundation
  • 29. Assets and Movies • AVAsset: Collection of tracks representing timed media data • QTMovie: Collection of tracks representing timed media data
  • 30. QuickTime movies Movie Track Track Media Media
  • 31. AVFoundation assets Movie Track Track
  • 32. AVAsset • Superclass of all “movie”-like structures in AVFoundation • Represents traits of all tracks taken together: size, duration • Build your own with AVURLAsset
  • 33. AVComposition • Subclass of AVAsset representing a combination of multiple file-based assets • Tracks are AVCompositionTracks • For editing, AVMutableComposition and AVMutableCompositionTracks
  • 34. Effects 1 • AVAudioMix, AVMutableAudioMix: set volumes or audio ramps at specific times • AVVideoCompositionInstructions: provide a set of layer-based instructions for performing time- based opacity or affine transform ramps
  • 35. Playback • AVPlayer: Playback controller • play, pause, seekToTime, etc. • AVPlayerLayer: CALayer for presenting video from an AVLayer
  • 36. Effects 2 • AVSynchronizedLayer: CALayer that synchronizes with a AVPlayerItem’s playback timing • Use for overlays, titles, rendered images, Ken Burns effects, etc.
  • 37. Export • AVAssetExportSession • Must be created with a canned preset • -[exportAsynchronouslyWith​ CompletionHandler:] • Takes a block! • Exporting CA effects is tricky…
  • 38. Other stuff • AVAssetStillImageGenerator: used for generating thumbnails • Not suitable for getting individual frames (but no GetMediaSample() equivalent either!) • NSCoder, NSValue additions for wrapping CMTimes, CMTimeRanges
  • 39. WWDC 2010 Session 407 Editing Media with AV Foundation Overview and best practices Eric Lee iPhone Engineering 2
  • 40. Demo
  • 42. What’s Next Next? AVAssetExportSession.h: extern NSString *const AVAssetExportPreset1280x720 __OSX_AVAILABLE_STARTING(__MAC_10_7,__IPHONE_4_0); CMTime.h: CM_EXPORT const CFStringRef kCMTimeScaleKey __OSX_AVAILABLE_STARTING(__MAC_10_7,__IPHONE_4_0);
  • 43. Contact Info • http://www.subfurther.com/blog • @invalidname • invalidname [at] gmail [dot] com