Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Dynamic Sound for iOS Apps and Games

2,604 views

Published on

Slides for my http://360idev.com/ topic, covering sound design, Pure Data, libpd and examples integrating Pure Data into iOS.

  • Be the first to comment

  • Be the first to like this

Dynamic Sound for iOS Apps and Games

  1. 1. Dynamic Audio For Apps and Games
  2. 2. 360 iDev tackmobile.com About Me • @thillerson, +thillerson (if you’re into that) • Developer at Tack Mobile (tackmobile.com) • iOS, Android, and Mobile Web • Rails, Node, maybe Elixir (one day)
  3. 3. 360 iDev tackmobile.com Rawk Star About Me
  4. 4. 360 iDev tackmobile.com About Me • Bass • Electronic Music • Some day I hope to play this thing --------->
  5. 5. 360 iDev tackmobile.com Sound Design Work: Start • Alarm App • Alarm Sounds • App Store: “Start by Tack”
  6. 6. 360 iDev tackmobile.com Sound Design Work: Adrift • Puzzle Game - “Adrift by Tack” in App Store • Background Music • Sound Effects
  7. 7. 360 iDev tackmobile.com Programming Sound • In Beta Soon* (*this week) • http://pragprog.com/ • What we’re talking about today... • ... and much more
  8. 8. 360 iDev tackmobile.com What We’re Going To Do Today • Answer: What Is Sound Design? • Discuss UX Principles of Sound Design • Introduce Pure Data • Introduce libpd • Tour Two iOS Apps Embedding Pd
  9. 9. 360 iDev tackmobile.com When We Are Done, You Will • Understand Sound Design • Understand Procedural Audio • Know what Pd is, and how libpd makes it awesome • Be excited about Dynamic Audio in your apps and games
  10. 10. 360 iDev tackmobile.com What is Sound Design?
  11. 11. 360 iDev tackmobile.com A Sound Designer Ben Burtt
  12. 12. 360 iDev tackmobile.com A Sound Designer... • Conceptualizes • Captures or Designs • Records • Engineers • Produces • ... the sound for an experience
  13. 13. 360 iDev tackmobile.com Sound Designer Skills • Understand Sound • Intuitive vs. Theoretical • Music vs. Sound • Audio Engineering
  14. 14. 360 iDev tackmobile.com Sound Design in Mobile Apps • Developers/Designers wear lots of hats • Control • Efficiency • Asset size
  15. 15. 360 iDev tackmobile.com Sound Design Principles
  16. 16. 360 iDev tackmobile.com A Sound Designer’s Goals • Audible Feedback • Fulfill Expectations • Immersion • Emotional Response
  17. 17. 360 iDev tackmobile.com Sound Design and Good UX • Your app doesn’t need sound • Expect to be turned off • Don’t make me listen to you • Listen on lots of devices/environments • Engineer for small speakers • Make your sound mean something!
  18. 18. 360 iDev tackmobile.com Always • Test with phone calls • Test while playing music • Have fallbacks - e.g. No music, just effects • Provide control to the user
  19. 19. 360 iDev tackmobile.com A Good Example: Clear • Realmac Software • Josh Mobley • Meaningful Sound • demo
  20. 20. 360 iDev tackmobile.com Sound
  21. 21. 360 iDev tackmobile.com Longitudinal Waves
  22. 22. 360 iDev tackmobile.com Of Pressure
  23. 23. 360 iDev tackmobile.com Through A Medium
  24. 24. 360 iDev tackmobile.com Transverse Wave Longitudinal Wave
  25. 25. 360 iDev tackmobile.com • Longitudinal Waves • Of Pressure • Through a Medium Sound
  26. 26. 360 iDev tackmobile.com Properties of Waves • A - Wavelength (Distance), Period (Time) • B - Amplitude • C - Frequency - 1/Period (measured in Hertz) A B B C 1 2 3
  27. 27. 360 iDev tackmobile.com Fourier Series • Based on his work -------> • Periodic signals composed of simple oscillating functions • Sine/Cosine waves are fundamental
  28. 28. 360 iDev tackmobile.com Oscillators • Something that moves according to a periodic function • A physical body • An electronic device
  29. 29. 360 iDev tackmobile.com Basic Electronic Sound oscillator voltage magnet pressure waves speaker
  30. 30. 360 iDev tackmobile.com Digital Audio • Stream of Numbers • Signal • Frequency and Amplitude • DAC - Digital to Analog Converter
  31. 31. 360 iDev tackmobile.com Dynamic Audio
  32. 32. 360 iDev tackmobile.com Procedural Audio • Functions • Objects • Controls and Components • Produce a Signal • “Building Sound from First Principles”
  33. 33. 360 iDev tackmobile.com Pure Data • Procedural Audio • Visual Programming Environment • Free and Open Source • pure-data.info
  34. 34. 360 iDev tackmobile.com Pure Data 101
  35. 35. 360 iDev tackmobile.com Signals and Values • Signals are streams of numbers, usually audio data • Values are numbers or symbols
  36. 36. 360 iDev tackmobile.com • Pd’s class • or, kind of like a command line tool Objects osc~ 440 signal inlet inlet signal outlet class name arguments
  37. 37. 360 iDev tackmobile.com Hot vs Cold Inlets • Hot inlets make objects immediately act • Cold inlets usually store a value • Execution is right to left
  38. 38. 360 iDev tackmobile.com • Signal connections are thick • Others are thin Connections osc~ 440 *~ dac~ 1
  39. 39. 360 iDev tackmobile.com Messages • Used for sending values without a connection • [send] and [receive] objects • Message boxes
  40. 40. 360 iDev tackmobile.com libpd • http://libpd.cc • Peter Brinkmann - Google Engineer • Enables you to embed an instance of Pd in your app • iOS and Android
  41. 41. 360 iDev tackmobile.com Examples
  42. 42. 360 iDev tackmobile.com Source Code • github.com/thillerson/dynamic-sound-ios • Examples of capabilities, not necessarily the best architecture • Goal: understand capabilities, have a place to start
  43. 43. 360 iDev tackmobile.com Example 1: A Simple Keyboard • Sine wave oscillator • Single voice • Note on/off
  44. 44. 360 iDev tackmobile.com Example 1: A Simple Keyboard Patch osc~ dac~ receive midinote mtof receive gate line~ *~ ; midinote 60; gate 1; ; gate 0$1 1 print
  45. 45. 360 iDev tackmobile.com Example 2: A Game • Top down perspective space shooter • Code by Lucas Jordan from iOS 5 Game Dev -> • Now with sound!
  46. 46. 360 iDev tackmobile.com Example 2: A Game Patch load a square wave at startup Thruster noise~ *~ samphold~ dac~ hip~ 200 phasor~ 800 line~ $1 200 *~ 0.1 receive thruster BulletShip_HitAsteroid_Hit samphold~ tabosc4~ square dac~ line~ phasor~ 8000 tabosc4~ square line~ unpack f f 0 $1 tabosc4~ square line~ unpack f f 0 $1 hip~ 1000 unpack f f 0 $1 *~ 0.5 receive asteroid receive ship receive bullet 8-bit-ify these sounds cut low frequencies comment comment Test_Messages ; bullet 25 200; ; thruster 0 ; thruster 1 pd wavesetup ; ship 1000 50; ; asteroid 200 100;
  47. 47. 360 iDev tackmobile.com Further Reading • puredata.info && libpd.cc • youtube.com/user/cheetomoskeeto • http://designingsound.org • mitpress.mit.edu/books/designing-sound
  48. 48. 360 iDev tackmobile.com Questions
  49. 49. Thank you! Dynamic Audio • Tony Hillerson • We’re Hiring! • careers@tackmobile.com • Excellent Team • Awesome Projects • Great Office

×