Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Dynamic Sound for Android

1,686 views

Published on

Slides for my AnDevCon talk from fall 2013, covering Pure Data from an Android app or game developer's perspective

  • Be the first to comment

Dynamic Sound for Android

  1. 1. Dynamic Audio For Apps and Games slides: is.gd/dynamic_sound_android code: github.com/thillerson/dynamic-sound-android
  2. 2. is.gd/ dynamic_sound_android Presentation tackmobile.com
  3. 3. is.gd/ dynamic_sound_android ! github.com/thillerson/ dynamic-sound-android ! Presentation tackmobile.com
  4. 4. About Me • @thillerson, +thillerson • Developer at Tack Mobile
 (tackmobile.com), @tackmobile • Android, iOS, and Mobile Web • Rails, Node, maybe Elixir (one day) Presentation tackmobile.com
  5. 5. About Me Rawk Star Presentation tackmobile.com
  6. 6. About Me • Bass • Electronic Music • Some day I hope to play this thing
 ---------> Presentation tackmobile.com
  7. 7. Sound Design Work: Start • Alarm App • Alarm Sounds • Play Store: “tackmobile” Presentation tackmobile.com
  8. 8. Sound Design Work: Adrift • Puzzle Game - “Adrift” in App Store • Background Music 
 soundcloud.com/tackmobile • Sound Effects Presentation tackmobile.com
  9. 9. Programming Sound • • In Beta • What we’re talking about today... • Presentation http://pragprog.com/ ... and much more tackmobile.com
  10. 10. What We’re Going To Do Today • Answer: What Is Sound Design? • Discuss UX Principles of Sound Design • Introduce Pure Data • Introduce libpd • Tour Two iOS Apps Embedding Pd Presentation tackmobile.com
  11. 11. When We Are Done, You Will • Understand Sound Design • Understand Procedural Audio • Know what Pd is, and how libpd makes it awesome • Be excited about Dynamic Audio in your apps and games Presentation tackmobile.com
  12. 12. What is Sound Design? Presentation tackmobile.com
  13. 13. A Sound Designer Ben Burtt Presentation tackmobile.com
  14. 14. A Sound Designer... • Conceptualizes • Captures or Designs • Records • Engineers • Produces • ... the sound for
 an experience Presentation tackmobile.com
  15. 15. Sound Designer Skills • Understand Sound • Intuitive vs. Theoretical • Music vs. Sound • Audio Engineering Presentation tackmobile.com
  16. 16. Sound Design in Mobile Apps • Developers/Designers wear lots of hats • Control • Efficiency • Asset size Presentation tackmobile.com
  17. 17. Sound Design Principles Presentation tackmobile.com
  18. 18. A Sound Designer’s Goals • Audible Feedback • Fulfill Expectations • Immersion • Emotional Response Presentation tackmobile.com
  19. 19. Sound Design and Good UX • Your app doesn’t need sound • Expect to be turned off • Don’t make me listen to you • Listen on lots of devices/environments • Engineer for small speakers • Make your sound mean something! Presentation tackmobile.com
  20. 20. Always • Test with phone calls • Test while playing music • Have fallbacks - e.g. No music, just effects • Provide control to the user Presentation tackmobile.com
  21. 21. A Good Example: Clear • Realmac Software • Josh Mobley • Meaningful Sound • demo Presentation tackmobile.com
  22. 22. Sound Presentation tackmobile.com
  23. 23. Longitudinal Waves Presentation tackmobile.com
  24. 24. Of Pressure Presentation tackmobile.com
  25. 25. Through A Medium Presentation tackmobile.com
  26. 26. Transverse Wave Longitudinal Wave Presentation tackmobile.com
  27. 27. Sound • Longitudinal Waves • Of Pressure • Through a Medium Presentation tackmobile.com
  28. 28. Properties of Waves A B B C 1 2 • A - Wavelength (Distance), Period (Time) • B - Amplitude • 3 C - Frequency - 1/Period (measured in Hertz) Presentation tackmobile.com
  29. 29. Fourier Series • Based on his work -------> • Periodic signals composed
 of simple oscillating
 functions • Sine/Cosine waves are
 fundamental Presentation tackmobile.com
  30. 30. Oscillators • Something that moves according to a periodic function • A physical body • An electronic device Presentation tackmobile.com
  31. 31. Basic Electronic Sound magnet voltage pressure
 waves oscillator speaker Presentation tackmobile.com
  32. 32. Digital Audio • Stream of Numbers • Signal • Frequency and Amplitude • DAC - Digital to Analog Converter Presentation tackmobile.com
  33. 33. Dynamic Audio Presentation tackmobile.com
  34. 34. Procedural Audio • Functions • Objects • Controls and Components • Produce a Signal • “Building Sound from First Principles” Presentation tackmobile.com
  35. 35. Pure Data • Procedural Audio • Visual Programming Environment • Free and Open Source • pure-data.info Presentation tackmobile.com
  36. 36. Pure Data 101 Presentation tackmobile.com
  37. 37. Signals and Values • Signals are streams of numbers, usually audio data • Values are numbers or symbols Presentation tackmobile.com
  38. 38. Objects • Pd’s class • or, kind of like a command line tool signal inlet class name osc~ 440 inlet arguments signal outlet Presentation tackmobile.com
  39. 39. Hot vs Cold Inlets • Hot inlets make objects immediately act • Cold inlets usually store a value • Execution is right to left Presentation tackmobile.com
  40. 40. Connections • Signal connections are thick • Others are thin osc~ 440 1 *~ dac~ Presentation tackmobile.com
  41. 41. Messages • Used for sending values without a connection • [send] and [receive] objects • Message boxes Presentation tackmobile.com
  42. 42. libpd • http://libpd.cc • Peter Brinkmann - Google Engineer • Enables you to embed an instance of Pd in your app • iOS and Android Presentation tackmobile.com
  43. 43. libpd for Android • Patches should be compressed raw resources • Utilities for dealing with patches • Provides interfaces for audio management • PdBase bridges to Pd • Pass messages, floats, and lists • Provides PdService for long running audio Presentation tackmobile.com
  44. 44. Examples Presentation tackmobile.com
  45. 45. Source Code • github.com/thillerson/dynamic-sound-android • Examples of capabilities, not necessarily the best architecture • Goal: understand capabilities, have a place to start Presentation tackmobile.com
  46. 46. Example 1: A Simple Keyboard • Sine wave oscillator • Single voice • Note on/off • Great example of Android screen latency! Presentation tackmobile.com
  47. 47. Example 1: A Simple Keyboard Patch receive midinote mtof osc~ receive gate ; midinote 60; gate 1; $1 1 ; gate 0 line~ print *~ dac~ Presentation tackmobile.com
  48. 48. Example 2: Space Rocks • Side scroller space shooter • Very simple implementation - not production ready game code • With dynamic sound! Presentation tackmobile.com
  49. 49. Example 2: A Game Patch pd wavesetup load a square wave at startup Test_Messages Thruster ; asteroid 200 100; ; ship 1000 50; ; thruster 1 noise~ Bullet receive asteroid receive ship receive bullet unpack f f 0 $1 receive thruster Ship_Hit unpack f f ; bullet 25 200; Asteroid_Hit 0 $1 unpack f f $1 200 0 $1 line~ *~ samphold~ phasor~ 800 line~ line~ tabosc4~ square tabosc4~ square line~ tabosc4~ square *~ 0.1 ; thruster 0 *~ 0.5 hip~ 200 comment comment dac~ phasor~ 8000 samphold~ 8-bit-ify these sounds hip~ 1000 cut low frequencies dac~ Presentation tackmobile.com
  50. 50. Programming Sound Presentation • http://pragprog.com/ book/thsound tackmobile.com
  51. 51. Further Reading • puredata.info && libpd.cc • youtube.com/user/cheetomoskeeto • http://designingsound.org • mitpress.mit.edu/books/designing-sound Presentation tackmobile.com
  52. 52. Questions Presentation tackmobile.com
  53. 53. Thank you! Dynamic Audio • Tony Hillerson • We’re Hiring! • careers@tackmobile.com • Excellent Team • Awesome Projects • Great Office

×