Dynamic Sound for Android

924
-1

Published on

Slides for my AnDevCon talk from fall 2013, covering Pure Data from an Android app or game developer's perspective

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
924
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Dynamic Sound for Android

  1. 1. Dynamic Audio For Apps and Games slides: is.gd/dynamic_sound_android code: github.com/thillerson/dynamic-sound-android
  2. 2. is.gd/ dynamic_sound_android Presentation tackmobile.com
  3. 3. is.gd/ dynamic_sound_android ! github.com/thillerson/ dynamic-sound-android ! Presentation tackmobile.com
  4. 4. About Me • @thillerson, +thillerson • Developer at Tack Mobile
 (tackmobile.com), @tackmobile • Android, iOS, and Mobile Web • Rails, Node, maybe Elixir (one day) Presentation tackmobile.com
  5. 5. About Me Rawk Star Presentation tackmobile.com
  6. 6. About Me • Bass • Electronic Music • Some day I hope to play this thing
 ---------> Presentation tackmobile.com
  7. 7. Sound Design Work: Start • Alarm App • Alarm Sounds • Play Store: “tackmobile” Presentation tackmobile.com
  8. 8. Sound Design Work: Adrift • Puzzle Game - “Adrift” in App Store • Background Music 
 soundcloud.com/tackmobile • Sound Effects Presentation tackmobile.com
  9. 9. Programming Sound • • In Beta • What we’re talking about today... • Presentation http://pragprog.com/ ... and much more tackmobile.com
  10. 10. What We’re Going To Do Today • Answer: What Is Sound Design? • Discuss UX Principles of Sound Design • Introduce Pure Data • Introduce libpd • Tour Two iOS Apps Embedding Pd Presentation tackmobile.com
  11. 11. When We Are Done, You Will • Understand Sound Design • Understand Procedural Audio • Know what Pd is, and how libpd makes it awesome • Be excited about Dynamic Audio in your apps and games Presentation tackmobile.com
  12. 12. What is Sound Design? Presentation tackmobile.com
  13. 13. A Sound Designer Ben Burtt Presentation tackmobile.com
  14. 14. A Sound Designer... • Conceptualizes • Captures or Designs • Records • Engineers • Produces • ... the sound for
 an experience Presentation tackmobile.com
  15. 15. Sound Designer Skills • Understand Sound • Intuitive vs. Theoretical • Music vs. Sound • Audio Engineering Presentation tackmobile.com
  16. 16. Sound Design in Mobile Apps • Developers/Designers wear lots of hats • Control • Efficiency • Asset size Presentation tackmobile.com
  17. 17. Sound Design Principles Presentation tackmobile.com
  18. 18. A Sound Designer’s Goals • Audible Feedback • Fulfill Expectations • Immersion • Emotional Response Presentation tackmobile.com
  19. 19. Sound Design and Good UX • Your app doesn’t need sound • Expect to be turned off • Don’t make me listen to you • Listen on lots of devices/environments • Engineer for small speakers • Make your sound mean something! Presentation tackmobile.com
  20. 20. Always • Test with phone calls • Test while playing music • Have fallbacks - e.g. No music, just effects • Provide control to the user Presentation tackmobile.com
  21. 21. A Good Example: Clear • Realmac Software • Josh Mobley • Meaningful Sound • demo Presentation tackmobile.com
  22. 22. Sound Presentation tackmobile.com
  23. 23. Longitudinal Waves Presentation tackmobile.com
  24. 24. Of Pressure Presentation tackmobile.com
  25. 25. Through A Medium Presentation tackmobile.com
  26. 26. Transverse Wave Longitudinal Wave Presentation tackmobile.com
  27. 27. Sound • Longitudinal Waves • Of Pressure • Through a Medium Presentation tackmobile.com
  28. 28. Properties of Waves A B B C 1 2 • A - Wavelength (Distance), Period (Time) • B - Amplitude • 3 C - Frequency - 1/Period (measured in Hertz) Presentation tackmobile.com
  29. 29. Fourier Series • Based on his work -------> • Periodic signals composed
 of simple oscillating
 functions • Sine/Cosine waves are
 fundamental Presentation tackmobile.com
  30. 30. Oscillators • Something that moves according to a periodic function • A physical body • An electronic device Presentation tackmobile.com
  31. 31. Basic Electronic Sound magnet voltage pressure
 waves oscillator speaker Presentation tackmobile.com
  32. 32. Digital Audio • Stream of Numbers • Signal • Frequency and Amplitude • DAC - Digital to Analog Converter Presentation tackmobile.com
  33. 33. Dynamic Audio Presentation tackmobile.com
  34. 34. Procedural Audio • Functions • Objects • Controls and Components • Produce a Signal • “Building Sound from First Principles” Presentation tackmobile.com
  35. 35. Pure Data • Procedural Audio • Visual Programming Environment • Free and Open Source • pure-data.info Presentation tackmobile.com
  36. 36. Pure Data 101 Presentation tackmobile.com
  37. 37. Signals and Values • Signals are streams of numbers, usually audio data • Values are numbers or symbols Presentation tackmobile.com
  38. 38. Objects • Pd’s class • or, kind of like a command line tool signal inlet class name osc~ 440 inlet arguments signal outlet Presentation tackmobile.com
  39. 39. Hot vs Cold Inlets • Hot inlets make objects immediately act • Cold inlets usually store a value • Execution is right to left Presentation tackmobile.com
  40. 40. Connections • Signal connections are thick • Others are thin osc~ 440 1 *~ dac~ Presentation tackmobile.com
  41. 41. Messages • Used for sending values without a connection • [send] and [receive] objects • Message boxes Presentation tackmobile.com
  42. 42. libpd • http://libpd.cc • Peter Brinkmann - Google Engineer • Enables you to embed an instance of Pd in your app • iOS and Android Presentation tackmobile.com
  43. 43. libpd for Android • Patches should be compressed raw resources • Utilities for dealing with patches • Provides interfaces for audio management • PdBase bridges to Pd • Pass messages, floats, and lists • Provides PdService for long running audio Presentation tackmobile.com
  44. 44. Examples Presentation tackmobile.com
  45. 45. Source Code • github.com/thillerson/dynamic-sound-android • Examples of capabilities, not necessarily the best architecture • Goal: understand capabilities, have a place to start Presentation tackmobile.com
  46. 46. Example 1: A Simple Keyboard • Sine wave oscillator • Single voice • Note on/off • Great example of Android screen latency! Presentation tackmobile.com
  47. 47. Example 1: A Simple Keyboard Patch receive midinote mtof osc~ receive gate ; midinote 60; gate 1; $1 1 ; gate 0 line~ print *~ dac~ Presentation tackmobile.com
  48. 48. Example 2: Space Rocks • Side scroller space shooter • Very simple implementation - not production ready game code • With dynamic sound! Presentation tackmobile.com
  49. 49. Example 2: A Game Patch pd wavesetup load a square wave at startup Test_Messages Thruster ; asteroid 200 100; ; ship 1000 50; ; thruster 1 noise~ Bullet receive asteroid receive ship receive bullet unpack f f 0 $1 receive thruster Ship_Hit unpack f f ; bullet 25 200; Asteroid_Hit 0 $1 unpack f f $1 200 0 $1 line~ *~ samphold~ phasor~ 800 line~ line~ tabosc4~ square tabosc4~ square line~ tabosc4~ square *~ 0.1 ; thruster 0 *~ 0.5 hip~ 200 comment comment dac~ phasor~ 8000 samphold~ 8-bit-ify these sounds hip~ 1000 cut low frequencies dac~ Presentation tackmobile.com
  50. 50. Programming Sound Presentation • http://pragprog.com/ book/thsound tackmobile.com
  51. 51. Further Reading • puredata.info && libpd.cc • youtube.com/user/cheetomoskeeto • http://designingsound.org • mitpress.mit.edu/books/designing-sound Presentation tackmobile.com
  52. 52. Questions Presentation tackmobile.com
  53. 53. Thank you! Dynamic Audio • Tony Hillerson • We’re Hiring! • careers@tackmobile.com • Excellent Team • Awesome Projects • Great Office
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×