Core Audio
Stephen Cussen

Dec 6th, 2013
for the
Cocoa Conspiracy team at Ideas and Coffee
What is the connection?
Get Lucky (2013)

I Feel Love (1977)
Highest level is best
•

most of the time if you need audio services you will be using
AVFoundation/AVFoundation.h or
Audi...
‘High level’ - another example:
•

SpriteKit allows you to play music (or sound effects)
as an action

•

an example of a ...
From AVFoundation’s
perspective:
Core Audio - some things you
need to dive deeper for
•

Say… you wanted to build an application like Final Cut Pro,
Sound ...
Core Audio is built on
AudioUnit components
•

Audio Units are arranged in the form of a graph (AUGraph), in
a pipeline si...
Audio Units
•

Effects

•

Music Effects

•

Instruments

•

Generators

•

Panners

•

Converters

•

Mixers

•

Offline ...
Core Audio - a real example
with some code
An iOS example to:
- load multiple instruments from a sound font file
- perform...
AU Graph for Synth Mixer Demo
Synth Mixer Demo
•demo
•so

- a simulator demo then a look at the Synth Mixer source

how did I find out how to adjust the...
Synth Mixer Demo cont…
•

So how did I find out how to load a Sound Font?

•

Apple Technical Note TN2283 AUSampler - Load...
MIDI
•I’m

using MusicDeviceMIDIEvent (declared in
MusicDevice.h) that is used to send MIDI events to an
Audio Unit
// som...
Sound Scape
•

My Core Audio example code on Github (scussen)
Documentation & Resources
What’s missing from the Apple Documentation?
- The ‘Unified’ CoreAudio Overview Document :-)
Wel...
Resources cont…
•

AU Lab

•

Panaiotis - our very own world class music
professor, composer and synth expert!
What is the link?
•

Daft Punk

•

Donna Summer

•

Giorgio Moroder

•

Moog Synthesizer
Upcoming SlideShare
Loading in...5
×

Core audio

869
-1

Published on

iOS Core Audio Introduction

Stephen Cussen's presentation for the Cocoa Conspiracy team at Ideas and Coffee at the South West Cyber Port

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
869
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Core audio

  1. 1. Core Audio Stephen Cussen Dec 6th, 2013 for the Cocoa Conspiracy team at Ideas and Coffee
  2. 2. What is the connection? Get Lucky (2013) I Feel Love (1977)
  3. 3. Highest level is best • most of the time if you need audio services you will be using AVFoundation/AVFoundation.h or AudioToolbox/AudioServices.h • an example using AudioToolbox - loading a sound from a mp3 file: . . . • @property (assign, nonatomic) SystemSoundID noteE1; NSURL *url = [[NSBundle mainBundle] URLForResource:@"guitarE1" withExtension:@"mp3"]; AudioServicesCreateSystemSoundID((__bridge CFURLRef)url, &noteE1); …then play the sound AudioServicesPlaySystemSound(self.noteE1); • demo - quick look at TrackTouch source and simulator demo
  4. 4. ‘High level’ - another example: • SpriteKit allows you to play music (or sound effects) as an action • an example of a ‘play sound’ action: SKAction *playMusic = [SKAction playSoundFileNamed:@"flight.caf" waitForCompletion:NO]; • demo - quick look at the Seed2 source for MyScene and a simulator demo
  5. 5. From AVFoundation’s perspective:
  6. 6. Core Audio - some things you need to dive deeper for • Say… you wanted to build an application like Final Cut Pro, Sound Track Pro, Garage Band, your own Synth App for the iOS or indeed the ‘Pano Composer’ :-) • Core Audio is the engine underlying all audio services on iOS and OSX. Programming it directly has a cost - it’s complicated and quite a lot of the ‘documentation’ is reading the header files and trying things out • First, make sure that you really need to be at this level. For example you could build a simple DJ mixer in AVFoundation • However, if you wanted to build the real full featured DJ Mixer with complex fades, effects and playing things backwards(!) you’d be at CoreAudio level
  7. 7. Core Audio is built on AudioUnit components • Audio Units are arranged in the form of a graph (AUGraph), in a pipeline similar to the way Core Image works with filters • You end up with a single output from an AUGraph that results in the final output sound stream • Audio Units are based on a plugin architecture with Apple supplying an extensive set of Audio Units that cover just about anything you could need. The standard CoreAudio bundle is in ‘/System/Library/Components’ and there is a search path for components so you can include Audio Component Plugins • You could build your own Audio Units and package them as plugins - for an overview see WWDC 2011 Video #411
  8. 8. Audio Units • Effects • Music Effects • Instruments • Generators • Panners • Converters • Mixers • Offline Effects • Output Units
  9. 9. Core Audio - a real example with some code An iOS example to: - load multiple instruments from a sound font file - perform real-time pitch adjustment on a single instrument - mix the output (play multiple instruments simultaneously) We will call it the Synth Mixer Demo…
  10. 10. AU Graph for Synth Mixer Demo
  11. 11. Synth Mixer Demo •demo •so - a simulator demo then a look at the Synth Mixer source how did I find out how to adjust the pitch with the effects unit? •documentation - from AudioUnitParameters.h - from AudioUnitParameters.h // Parameters for AUNewTimePitch enum { // Global, rate, 1/32 -> 32.0, 1.0 kNewTimePitchParam_Rate // Global, Cents, -2400 -> 2400, 1.0 kNewTimePitchParam_Pitch // Global, generic, 3.0 -> 32.0, 8.0 kNewTimePitchParam_Overlap // Global, Boolean, 0->1, 1 kNewTimePitchParam_EnablePeakLocking }; - from AudioUnitParameters.h enum { kAudioUnitScope_Global kAudioUnitScope_Input kAudioUnitScope_Output kAudioUnitScope_Group kAudioUnitScope_Part kAudioUnitScope_Note kAudioUnitScope_Layer kAudioUnitScope_LayerItem }; = 0, = 1, = 2, = 3, = 6, = 4, = 5, =7 = 0, = 1, = 4, =6
  12. 12. Synth Mixer Demo cont… • So how did I find out how to load a Sound Font? • Apple Technical Note TN2283 AUSampler - Loading Instruments
  13. 13. MIDI •I’m using MusicDeviceMIDIEvent (declared in MusicDevice.h) that is used to send MIDI events to an Audio Unit // some MIDI constants: enum { kMIDIMessage_NoteOn = 0x9, kMIDIMessage_NoteOff = 0x8, }; UInt32 noteNum = 60; UInt32 onVelocity = 127; UInt32 noteCommand = kMIDIMessage_NoteOn << 4 | 0; MusicDeviceMIDIEvent (self.samplerUnit, noteCommand, noteNum, onVelocity, 0);
  14. 14. Sound Scape • My Core Audio example code on Github (scussen)
  15. 15. Documentation & Resources What’s missing from the Apple Documentation? - The ‘Unified’ CoreAudio Overview Document :-) Well there is a Core Audio Overview document in the Developer Library but it was last updated in 2008… Documentation & Resources available: Your number one place to go is the iOS Developer Library (now much better but not complete) The videos: WWDC 2013: What’s New In Core Audio for iOS - Video #602 WWDC 2011: AudioSession and MultiRoute Audio - Video #505 WWDC 2011: Audio Session Management For iOS - Video #413 WWDC 2011: Music in iOS and Lion - Video #411 The documents: https://developer.apple.com/library/ios/navigation/#section=Frameworks&topic=AVFoundation https://developer.apple.com/library/ios/navigation/#section=Frameworks&topic=AudioToolbox https://developer.apple.com/library/ios/navigation/#section=Frameworks&topic=AudioUnit https://developer.apple.com/library/ios/navigation/#section=Frameworks&topic=CoreAudio Tech notes: - iOS Developer Library - Technical Note TN2283 ‘AUSampler - Loading Instruments’ - Example of the level of detail available: Audio Queue - Looping Compressed Audio The header files: This is where you will find most (and sometimes all the) detail ‘documentation’ :-)
  16. 16. Resources cont… • AU Lab • Panaiotis - our very own world class music professor, composer and synth expert!
  17. 17. What is the link? • Daft Punk • Donna Summer • Giorgio Moroder • Moog Synthesizer
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×