0
Dynamic Audio
For Apps and Games
slides: is.gd/dynamic_sound_android
code: github.com/thillerson/dynamic-sound-android
is.gd/
dynamic_sound_android

Presentation

tackmobile.com
is.gd/
dynamic_sound_android
!

github.com/thillerson/
dynamic-sound-android
!

Presentation

tackmobile.com
About Me
•

@thillerson, +thillerson

•

Developer at Tack Mobile

(tackmobile.com), @tackmobile

•

Android, iOS, and Mob...
About Me
Rawk Star

Presentation

tackmobile.com
About Me
•

Bass

•

Electronic Music

•

Some day I hope to
play this thing

--------->

Presentation

tackmobile.com
Sound Design Work: Start
•

Alarm App

•

Alarm Sounds

•

Play Store: “tackmobile”

Presentation

tackmobile.com
Sound Design Work: Adrift
•

Puzzle Game - “Adrift” in App Store

•

Background Music 

soundcloud.com/tackmobile

•

Soun...
Programming Sound
•
•

In Beta

•

What we’re talking
about today...

•

Presentation

http://pragprog.com/

... and much ...
What We’re Going To Do Today
•

Answer: What Is Sound Design?

•

Discuss UX Principles of Sound Design

•

Introduce Pure...
When We Are Done, You Will
•

Understand Sound Design

•

Understand Procedural Audio

•

Know what Pd is, and how libpd m...
What is Sound Design?

Presentation

tackmobile.com
A Sound Designer

Ben Burtt
Presentation

tackmobile.com
A Sound Designer...
•

Conceptualizes

•

Captures or Designs

•

Records

•

Engineers

•

Produces

•

... the sound for...
Sound Designer Skills
•

Understand Sound

•

Intuitive vs. Theoretical

•

Music vs. Sound

•

Audio Engineering

Present...
Sound Design in Mobile Apps
•

Developers/Designers wear lots of hats

•

Control

•

Efficiency

•

Asset size

Presentati...
Sound Design Principles

Presentation

tackmobile.com
A Sound Designer’s Goals
•

Audible Feedback

•

Fulfill Expectations

•

Immersion

•

Emotional Response

Presentation

t...
Sound Design and Good UX
•

Your app doesn’t need sound

•

Expect to be turned off

•

Don’t make me listen to you

•

Li...
Always
•

Test with phone calls

•

Test while playing music

•

Have fallbacks - e.g. No music, just effects

•

Provide ...
A Good Example: Clear
•

Realmac Software

•

Josh Mobley

•

Meaningful Sound

•

demo

Presentation

tackmobile.com
Sound

Presentation

tackmobile.com
Longitudinal Waves

Presentation

tackmobile.com
Of Pressure

Presentation

tackmobile.com
Through A Medium

Presentation

tackmobile.com
Transverse Wave

Longitudinal Wave
Presentation

tackmobile.com
Sound
•

Longitudinal Waves

•

Of Pressure

•

Through a Medium

Presentation

tackmobile.com
Properties of Waves
A

B
B
C

1

2

•

A - Wavelength (Distance), Period (Time)

•

B - Amplitude

•

3

C - Frequency - 1...
Fourier Series
•

Based on his work ------->

•

Periodic signals composed

of simple oscillating

functions

•

Sine/Cosi...
Oscillators
•

Something that moves according to a periodic
function

•

A physical body

•

An electronic device

Present...
Basic Electronic Sound

magnet
voltage
pressure

waves

oscillator
speaker

Presentation

tackmobile.com
Digital Audio
•

Stream of Numbers

•

Signal

•

Frequency and Amplitude

•

DAC - Digital to Analog Converter

Presentat...
Dynamic Audio

Presentation

tackmobile.com
Procedural Audio
•

Functions

•

Objects

•

Controls and Components

•

Produce a Signal

•

“Building Sound from First ...
Pure Data
•

Procedural Audio

•

Visual Programming Environment

•

Free and Open Source

•

pure-data.info

Presentation...
Pure Data 101

Presentation

tackmobile.com
Signals and Values
•

Signals are streams of numbers, usually audio
data

•

Values are numbers or symbols

Presentation

...
Objects
•

Pd’s class

•

or, kind of like a command line tool
signal inlet
class name

osc~ 440

inlet
arguments

signal ...
Hot vs Cold Inlets
•

Hot inlets make objects immediately act

•

Cold inlets usually store a value

•

Execution is right...
Connections
•

Signal connections are thick

•

Others are thin

osc~ 440
1
*~

dac~
Presentation

tackmobile.com
Messages
•

Used for sending values without a connection

•

[send] and [receive] objects

•

Message boxes

Presentation
...
libpd
•

http://libpd.cc

•

Peter Brinkmann - Google Engineer

•

Enables you to embed an instance of Pd in your
app

•

...
libpd for Android
•

Patches should be compressed raw resources

•

Utilities for dealing with patches

•

Provides interf...
Examples

Presentation

tackmobile.com
Source Code
•

github.com/thillerson/dynamic-sound-android

•

Examples of capabilities, not necessarily the
best architec...
Example 1: A Simple Keyboard
•

Sine wave oscillator

•

Single voice

•

Note on/off

•

Great example of Android
screen ...
Example 1: A Simple Keyboard Patch
receive midinote
mtof
osc~

receive gate

;
midinote 60;
gate 1;

$1 1

;
gate 0

line~...
Example 2: Space Rocks
•

Side scroller space
shooter

•

Very simple
implementation - not
production ready
game code

•

...
Example 2: A Game Patch
pd wavesetup load a square wave
at startup

Test_Messages

Thruster

;
asteroid 200 100;
;
ship 10...
Programming Sound

Presentation

•

http://pragprog.com/
book/thsound

tackmobile.com
Further Reading
•

puredata.info && libpd.cc

•

youtube.com/user/cheetomoskeeto

•

http://designingsound.org

•

mitpres...
Questions

Presentation

tackmobile.com
Thank you!
Dynamic Audio • Tony Hillerson

•

We’re Hiring!

•

careers@tackmobile.com

•

Excellent Team

•

Awesome Proj...
Upcoming SlideShare
Loading in...5
×

Dynamic Sound for Android

807

Published on

Slides for my AnDevCon talk from fall 2013, covering Pure Data from an Android app or game developer's perspective

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
807
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Dynamic Sound for Android"

  1. 1. Dynamic Audio For Apps and Games slides: is.gd/dynamic_sound_android code: github.com/thillerson/dynamic-sound-android
  2. 2. is.gd/ dynamic_sound_android Presentation tackmobile.com
  3. 3. is.gd/ dynamic_sound_android ! github.com/thillerson/ dynamic-sound-android ! Presentation tackmobile.com
  4. 4. About Me • @thillerson, +thillerson • Developer at Tack Mobile
 (tackmobile.com), @tackmobile • Android, iOS, and Mobile Web • Rails, Node, maybe Elixir (one day) Presentation tackmobile.com
  5. 5. About Me Rawk Star Presentation tackmobile.com
  6. 6. About Me • Bass • Electronic Music • Some day I hope to play this thing
 ---------> Presentation tackmobile.com
  7. 7. Sound Design Work: Start • Alarm App • Alarm Sounds • Play Store: “tackmobile” Presentation tackmobile.com
  8. 8. Sound Design Work: Adrift • Puzzle Game - “Adrift” in App Store • Background Music 
 soundcloud.com/tackmobile • Sound Effects Presentation tackmobile.com
  9. 9. Programming Sound • • In Beta • What we’re talking about today... • Presentation http://pragprog.com/ ... and much more tackmobile.com
  10. 10. What We’re Going To Do Today • Answer: What Is Sound Design? • Discuss UX Principles of Sound Design • Introduce Pure Data • Introduce libpd • Tour Two iOS Apps Embedding Pd Presentation tackmobile.com
  11. 11. When We Are Done, You Will • Understand Sound Design • Understand Procedural Audio • Know what Pd is, and how libpd makes it awesome • Be excited about Dynamic Audio in your apps and games Presentation tackmobile.com
  12. 12. What is Sound Design? Presentation tackmobile.com
  13. 13. A Sound Designer Ben Burtt Presentation tackmobile.com
  14. 14. A Sound Designer... • Conceptualizes • Captures or Designs • Records • Engineers • Produces • ... the sound for
 an experience Presentation tackmobile.com
  15. 15. Sound Designer Skills • Understand Sound • Intuitive vs. Theoretical • Music vs. Sound • Audio Engineering Presentation tackmobile.com
  16. 16. Sound Design in Mobile Apps • Developers/Designers wear lots of hats • Control • Efficiency • Asset size Presentation tackmobile.com
  17. 17. Sound Design Principles Presentation tackmobile.com
  18. 18. A Sound Designer’s Goals • Audible Feedback • Fulfill Expectations • Immersion • Emotional Response Presentation tackmobile.com
  19. 19. Sound Design and Good UX • Your app doesn’t need sound • Expect to be turned off • Don’t make me listen to you • Listen on lots of devices/environments • Engineer for small speakers • Make your sound mean something! Presentation tackmobile.com
  20. 20. Always • Test with phone calls • Test while playing music • Have fallbacks - e.g. No music, just effects • Provide control to the user Presentation tackmobile.com
  21. 21. A Good Example: Clear • Realmac Software • Josh Mobley • Meaningful Sound • demo Presentation tackmobile.com
  22. 22. Sound Presentation tackmobile.com
  23. 23. Longitudinal Waves Presentation tackmobile.com
  24. 24. Of Pressure Presentation tackmobile.com
  25. 25. Through A Medium Presentation tackmobile.com
  26. 26. Transverse Wave Longitudinal Wave Presentation tackmobile.com
  27. 27. Sound • Longitudinal Waves • Of Pressure • Through a Medium Presentation tackmobile.com
  28. 28. Properties of Waves A B B C 1 2 • A - Wavelength (Distance), Period (Time) • B - Amplitude • 3 C - Frequency - 1/Period (measured in Hertz) Presentation tackmobile.com
  29. 29. Fourier Series • Based on his work -------> • Periodic signals composed
 of simple oscillating
 functions • Sine/Cosine waves are
 fundamental Presentation tackmobile.com
  30. 30. Oscillators • Something that moves according to a periodic function • A physical body • An electronic device Presentation tackmobile.com
  31. 31. Basic Electronic Sound magnet voltage pressure
 waves oscillator speaker Presentation tackmobile.com
  32. 32. Digital Audio • Stream of Numbers • Signal • Frequency and Amplitude • DAC - Digital to Analog Converter Presentation tackmobile.com
  33. 33. Dynamic Audio Presentation tackmobile.com
  34. 34. Procedural Audio • Functions • Objects • Controls and Components • Produce a Signal • “Building Sound from First Principles” Presentation tackmobile.com
  35. 35. Pure Data • Procedural Audio • Visual Programming Environment • Free and Open Source • pure-data.info Presentation tackmobile.com
  36. 36. Pure Data 101 Presentation tackmobile.com
  37. 37. Signals and Values • Signals are streams of numbers, usually audio data • Values are numbers or symbols Presentation tackmobile.com
  38. 38. Objects • Pd’s class • or, kind of like a command line tool signal inlet class name osc~ 440 inlet arguments signal outlet Presentation tackmobile.com
  39. 39. Hot vs Cold Inlets • Hot inlets make objects immediately act • Cold inlets usually store a value • Execution is right to left Presentation tackmobile.com
  40. 40. Connections • Signal connections are thick • Others are thin osc~ 440 1 *~ dac~ Presentation tackmobile.com
  41. 41. Messages • Used for sending values without a connection • [send] and [receive] objects • Message boxes Presentation tackmobile.com
  42. 42. libpd • http://libpd.cc • Peter Brinkmann - Google Engineer • Enables you to embed an instance of Pd in your app • iOS and Android Presentation tackmobile.com
  43. 43. libpd for Android • Patches should be compressed raw resources • Utilities for dealing with patches • Provides interfaces for audio management • PdBase bridges to Pd • Pass messages, floats, and lists • Provides PdService for long running audio Presentation tackmobile.com
  44. 44. Examples Presentation tackmobile.com
  45. 45. Source Code • github.com/thillerson/dynamic-sound-android • Examples of capabilities, not necessarily the best architecture • Goal: understand capabilities, have a place to start Presentation tackmobile.com
  46. 46. Example 1: A Simple Keyboard • Sine wave oscillator • Single voice • Note on/off • Great example of Android screen latency! Presentation tackmobile.com
  47. 47. Example 1: A Simple Keyboard Patch receive midinote mtof osc~ receive gate ; midinote 60; gate 1; $1 1 ; gate 0 line~ print *~ dac~ Presentation tackmobile.com
  48. 48. Example 2: Space Rocks • Side scroller space shooter • Very simple implementation - not production ready game code • With dynamic sound! Presentation tackmobile.com
  49. 49. Example 2: A Game Patch pd wavesetup load a square wave at startup Test_Messages Thruster ; asteroid 200 100; ; ship 1000 50; ; thruster 1 noise~ Bullet receive asteroid receive ship receive bullet unpack f f 0 $1 receive thruster Ship_Hit unpack f f ; bullet 25 200; Asteroid_Hit 0 $1 unpack f f $1 200 0 $1 line~ *~ samphold~ phasor~ 800 line~ line~ tabosc4~ square tabosc4~ square line~ tabosc4~ square *~ 0.1 ; thruster 0 *~ 0.5 hip~ 200 comment comment dac~ phasor~ 8000 samphold~ 8-bit-ify these sounds hip~ 1000 cut low frequencies dac~ Presentation tackmobile.com
  50. 50. Programming Sound Presentation • http://pragprog.com/ book/thsound tackmobile.com
  51. 51. Further Reading • puredata.info && libpd.cc • youtube.com/user/cheetomoskeeto • http://designingsound.org • mitpress.mit.edu/books/designing-sound Presentation tackmobile.com
  52. 52. Questions Presentation tackmobile.com
  53. 53. Thank you! Dynamic Audio • Tony Hillerson • We’re Hiring! • careers@tackmobile.com • Excellent Team • Awesome Projects • Great Office
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×