Dynamic Audio
For Apps and Games
360 iDev tackmobile.com
About Me
• @thillerson, +thillerson (if you’re into that)
• Developer at Tack Mobile (tackmobile.c...
360 iDev tackmobile.com
Rawk Star
About Me
360 iDev tackmobile.com
About Me
• Bass
• Electronic Music
• Some day I hope to
play this thing
--------->
360 iDev tackmobile.com
Sound Design Work: Start
• Alarm App
• Alarm Sounds
• App Store: “Start by Tack”
360 iDev tackmobile.com
Sound Design Work: Adrift
• Puzzle Game - “Adrift by Tack” in App Store
• Background Music
• Sound...
360 iDev tackmobile.com
Programming Sound
• In Beta Soon*
(*this week)
• http://pragprog.com/
• What we’re talking
about t...
360 iDev tackmobile.com
What We’re Going To Do Today
• Answer: What Is Sound Design?
• Discuss UX Principles of Sound Desi...
360 iDev tackmobile.com
When We Are Done, You Will
• Understand Sound Design
• Understand Procedural Audio
• Know what Pd ...
360 iDev tackmobile.com
What is Sound Design?
360 iDev tackmobile.com
A Sound Designer
Ben Burtt
360 iDev tackmobile.com
A Sound Designer...
• Conceptualizes
• Captures or Designs
• Records
• Engineers
• Produces
• ... ...
360 iDev tackmobile.com
Sound Designer Skills
• Understand Sound
• Intuitive vs. Theoretical
• Music vs. Sound
• Audio Eng...
360 iDev tackmobile.com
Sound Design in Mobile Apps
• Developers/Designers wear lots of hats
• Control
• Efficiency
• Asse...
360 iDev tackmobile.com
Sound Design Principles
360 iDev tackmobile.com
A Sound Designer’s Goals
• Audible Feedback
• Fulfill Expectations
• Immersion
• Emotional Response
360 iDev tackmobile.com
Sound Design and Good UX
• Your app doesn’t need sound
• Expect to be turned off
• Don’t make me l...
360 iDev tackmobile.com
Always
• Test with phone calls
• Test while playing music
• Have fallbacks - e.g. No music, just e...
360 iDev tackmobile.com
A Good Example: Clear
• Realmac Software
• Josh Mobley
• Meaningful Sound
• demo
360 iDev tackmobile.com
Sound
360 iDev tackmobile.com
Longitudinal Waves
360 iDev tackmobile.com
Of Pressure
360 iDev tackmobile.com
Through A Medium
360 iDev tackmobile.com
Transverse Wave
Longitudinal Wave
360 iDev tackmobile.com
• Longitudinal Waves
• Of Pressure
• Through a Medium
Sound
360 iDev tackmobile.com
Properties of Waves
• A - Wavelength (Distance), Period (Time)
• B - Amplitude
• C - Frequency - 1...
360 iDev tackmobile.com
Fourier Series
• Based on his work ------->
• Periodic signals composed
of simple oscillating
func...
360 iDev tackmobile.com
Oscillators
• Something that moves according to a periodic
function
• A physical body
• An electro...
360 iDev tackmobile.com
Basic Electronic Sound
oscillator
voltage
magnet
pressure
waves
speaker
360 iDev tackmobile.com
Digital Audio
• Stream of Numbers
• Signal
• Frequency and Amplitude
• DAC - Digital to Analog Con...
360 iDev tackmobile.com
Dynamic Audio
360 iDev tackmobile.com
Procedural Audio
• Functions
• Objects
• Controls and Components
• Produce a Signal
• “Building So...
360 iDev tackmobile.com
Pure Data
• Procedural Audio
• Visual Programming Environment
• Free and Open Source
• pure-data.i...
360 iDev tackmobile.com
Pure Data 101
360 iDev tackmobile.com
Signals and Values
• Signals are streams of numbers, usually audio
data
• Values are numbers or sy...
360 iDev tackmobile.com
• Pd’s class
• or, kind of like a command line tool
Objects
osc~ 440
signal inlet inlet
signal out...
360 iDev tackmobile.com
Hot vs Cold Inlets
• Hot inlets make objects immediately act
• Cold inlets usually store a value
•...
360 iDev tackmobile.com
• Signal connections are thick
• Others are thin
Connections
osc~ 440
*~
dac~
1
360 iDev tackmobile.com
Messages
• Used for sending values without a connection
• [send] and [receive] objects
• Message b...
360 iDev tackmobile.com
libpd
• http://libpd.cc
• Peter Brinkmann - Google Engineer
• Enables you to embed an instance of ...
360 iDev tackmobile.com
Examples
360 iDev tackmobile.com
Source Code
• github.com/thillerson/dynamic-sound-ios
• Examples of capabilities, not necessarily ...
360 iDev tackmobile.com
Example 1: A Simple Keyboard
• Sine wave oscillator
• Single voice
• Note on/off
360 iDev tackmobile.com
Example 1: A Simple Keyboard Patch
osc~
dac~
receive midinote
mtof
receive gate
line~
*~
;
midinot...
360 iDev tackmobile.com
Example 2: A Game
• Top down perspective space shooter
• Code by Lucas Jordan
from iOS 5 Game Dev ...
360 iDev tackmobile.com
Example 2: A Game Patch
load a square wave
at startup
Thruster
noise~
*~
samphold~
dac~
hip~ 200
p...
360 iDev tackmobile.com
Further Reading
• puredata.info && libpd.cc
• youtube.com/user/cheetomoskeeto
• http://designingso...
360 iDev tackmobile.com
Questions
Thank you!
Dynamic Audio • Tony Hillerson
• We’re Hiring!
• careers@tackmobile.com
• Excellent Team
• Awesome Projects
• G...
Upcoming SlideShare
Loading in...5
×

Dynamic Sound for iOS Apps and Games

1,900

Published on

Slides for my http://360idev.com/ topic, covering sound design, Pure Data, libpd and examples integrating Pure Data into iOS.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,900
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Dynamic Sound for iOS Apps and Games

  1. 1. Dynamic Audio For Apps and Games
  2. 2. 360 iDev tackmobile.com About Me • @thillerson, +thillerson (if you’re into that) • Developer at Tack Mobile (tackmobile.com) • iOS, Android, and Mobile Web • Rails, Node, maybe Elixir (one day)
  3. 3. 360 iDev tackmobile.com Rawk Star About Me
  4. 4. 360 iDev tackmobile.com About Me • Bass • Electronic Music • Some day I hope to play this thing --------->
  5. 5. 360 iDev tackmobile.com Sound Design Work: Start • Alarm App • Alarm Sounds • App Store: “Start by Tack”
  6. 6. 360 iDev tackmobile.com Sound Design Work: Adrift • Puzzle Game - “Adrift by Tack” in App Store • Background Music • Sound Effects
  7. 7. 360 iDev tackmobile.com Programming Sound • In Beta Soon* (*this week) • http://pragprog.com/ • What we’re talking about today... • ... and much more
  8. 8. 360 iDev tackmobile.com What We’re Going To Do Today • Answer: What Is Sound Design? • Discuss UX Principles of Sound Design • Introduce Pure Data • Introduce libpd • Tour Two iOS Apps Embedding Pd
  9. 9. 360 iDev tackmobile.com When We Are Done, You Will • Understand Sound Design • Understand Procedural Audio • Know what Pd is, and how libpd makes it awesome • Be excited about Dynamic Audio in your apps and games
  10. 10. 360 iDev tackmobile.com What is Sound Design?
  11. 11. 360 iDev tackmobile.com A Sound Designer Ben Burtt
  12. 12. 360 iDev tackmobile.com A Sound Designer... • Conceptualizes • Captures or Designs • Records • Engineers • Produces • ... the sound for an experience
  13. 13. 360 iDev tackmobile.com Sound Designer Skills • Understand Sound • Intuitive vs. Theoretical • Music vs. Sound • Audio Engineering
  14. 14. 360 iDev tackmobile.com Sound Design in Mobile Apps • Developers/Designers wear lots of hats • Control • Efficiency • Asset size
  15. 15. 360 iDev tackmobile.com Sound Design Principles
  16. 16. 360 iDev tackmobile.com A Sound Designer’s Goals • Audible Feedback • Fulfill Expectations • Immersion • Emotional Response
  17. 17. 360 iDev tackmobile.com Sound Design and Good UX • Your app doesn’t need sound • Expect to be turned off • Don’t make me listen to you • Listen on lots of devices/environments • Engineer for small speakers • Make your sound mean something!
  18. 18. 360 iDev tackmobile.com Always • Test with phone calls • Test while playing music • Have fallbacks - e.g. No music, just effects • Provide control to the user
  19. 19. 360 iDev tackmobile.com A Good Example: Clear • Realmac Software • Josh Mobley • Meaningful Sound • demo
  20. 20. 360 iDev tackmobile.com Sound
  21. 21. 360 iDev tackmobile.com Longitudinal Waves
  22. 22. 360 iDev tackmobile.com Of Pressure
  23. 23. 360 iDev tackmobile.com Through A Medium
  24. 24. 360 iDev tackmobile.com Transverse Wave Longitudinal Wave
  25. 25. 360 iDev tackmobile.com • Longitudinal Waves • Of Pressure • Through a Medium Sound
  26. 26. 360 iDev tackmobile.com Properties of Waves • A - Wavelength (Distance), Period (Time) • B - Amplitude • C - Frequency - 1/Period (measured in Hertz) A B B C 1 2 3
  27. 27. 360 iDev tackmobile.com Fourier Series • Based on his work -------> • Periodic signals composed of simple oscillating functions • Sine/Cosine waves are fundamental
  28. 28. 360 iDev tackmobile.com Oscillators • Something that moves according to a periodic function • A physical body • An electronic device
  29. 29. 360 iDev tackmobile.com Basic Electronic Sound oscillator voltage magnet pressure waves speaker
  30. 30. 360 iDev tackmobile.com Digital Audio • Stream of Numbers • Signal • Frequency and Amplitude • DAC - Digital to Analog Converter
  31. 31. 360 iDev tackmobile.com Dynamic Audio
  32. 32. 360 iDev tackmobile.com Procedural Audio • Functions • Objects • Controls and Components • Produce a Signal • “Building Sound from First Principles”
  33. 33. 360 iDev tackmobile.com Pure Data • Procedural Audio • Visual Programming Environment • Free and Open Source • pure-data.info
  34. 34. 360 iDev tackmobile.com Pure Data 101
  35. 35. 360 iDev tackmobile.com Signals and Values • Signals are streams of numbers, usually audio data • Values are numbers or symbols
  36. 36. 360 iDev tackmobile.com • Pd’s class • or, kind of like a command line tool Objects osc~ 440 signal inlet inlet signal outlet class name arguments
  37. 37. 360 iDev tackmobile.com Hot vs Cold Inlets • Hot inlets make objects immediately act • Cold inlets usually store a value • Execution is right to left
  38. 38. 360 iDev tackmobile.com • Signal connections are thick • Others are thin Connections osc~ 440 *~ dac~ 1
  39. 39. 360 iDev tackmobile.com Messages • Used for sending values without a connection • [send] and [receive] objects • Message boxes
  40. 40. 360 iDev tackmobile.com libpd • http://libpd.cc • Peter Brinkmann - Google Engineer • Enables you to embed an instance of Pd in your app • iOS and Android
  41. 41. 360 iDev tackmobile.com Examples
  42. 42. 360 iDev tackmobile.com Source Code • github.com/thillerson/dynamic-sound-ios • Examples of capabilities, not necessarily the best architecture • Goal: understand capabilities, have a place to start
  43. 43. 360 iDev tackmobile.com Example 1: A Simple Keyboard • Sine wave oscillator • Single voice • Note on/off
  44. 44. 360 iDev tackmobile.com Example 1: A Simple Keyboard Patch osc~ dac~ receive midinote mtof receive gate line~ *~ ; midinote 60; gate 1; ; gate 0$1 1 print
  45. 45. 360 iDev tackmobile.com Example 2: A Game • Top down perspective space shooter • Code by Lucas Jordan from iOS 5 Game Dev -> • Now with sound!
  46. 46. 360 iDev tackmobile.com Example 2: A Game Patch load a square wave at startup Thruster noise~ *~ samphold~ dac~ hip~ 200 phasor~ 800 line~ $1 200 *~ 0.1 receive thruster BulletShip_HitAsteroid_Hit samphold~ tabosc4~ square dac~ line~ phasor~ 8000 tabosc4~ square line~ unpack f f 0 $1 tabosc4~ square line~ unpack f f 0 $1 hip~ 1000 unpack f f 0 $1 *~ 0.5 receive asteroid receive ship receive bullet 8-bit-ify these sounds cut low frequencies comment comment Test_Messages ; bullet 25 200; ; thruster 0 ; thruster 1 pd wavesetup ; ship 1000 50; ; asteroid 200 100;
  47. 47. 360 iDev tackmobile.com Further Reading • puredata.info && libpd.cc • youtube.com/user/cheetomoskeeto • http://designingsound.org • mitpress.mit.edu/books/designing-sound
  48. 48. 360 iDev tackmobile.com Questions
  49. 49. Thank you! Dynamic Audio • Tony Hillerson • We’re Hiring! • careers@tackmobile.com • Excellent Team • Awesome Projects • Great Office
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×