Dynamic Sound for iOS Apps and Games

Dynamic Audio
For Apps and Games

360 iDev tackmobile.com
About Me
• @thillerson, +thillerson (if you’re into that)
• Developer at Tack Mobile (tackmobile.com)
• iOS, Android, and Mobile Web
• Rails, Node, maybe Elixir (one day)

Rawk Star
About Me

About Me
• Bass
• Electronic Music
• Some day I hope to
play this thing
--------->

Sound Design Work: Start
• Alarm App
• Alarm Sounds
• App Store: “Start by Tack”

Sound Design Work: Adrift
• Puzzle Game - “Adrift by Tack” in App Store
• Background Music
• Sound Effects

Programming Sound
• In Beta Soon*
(*this week)
• http://pragprog.com/
• What we’re talking
about today...
• ... and much more

What We’re Going To Do Today
• Answer: What Is Sound Design?
• Discuss UX Principles of Sound Design
• Introduce Pure Data
• Introduce libpd
• Tour Two iOS Apps Embedding Pd

When We Are Done, You Will
• Understand Sound Design
• Understand Procedural Audio
• Know what Pd is, and how libpd makes it
awesome
• Be excited about Dynamic Audio in your apps
and games

What is Sound Design?

A Sound Designer
Ben Burtt

A Sound Designer...
• Conceptualizes
• Captures or Designs
• Records
• Engineers
• Produces
• ... the sound for
an experience

Sound Designer Skills
• Understand Sound
• Intuitive vs. Theoretical
• Music vs. Sound
• Audio Engineering

Sound Design in Mobile Apps
• Developers/Designers wear lots of hats
• Control
• Efficiency
• Asset size

Sound Design Principles

A Sound Designer’s Goals
• Audible Feedback
• Fulfill Expectations
• Immersion
• Emotional Response

Sound Design and Good UX
• Your app doesn’t need sound
• Expect to be turned off
• Don’t make me listen to you
• Listen on lots of devices/environments
• Engineer for small speakers
• Make your sound mean something!

Always
• Test with phone calls
• Test while playing music
• Have fallbacks - e.g. No music, just effects
• Provide control to the user

A Good Example: Clear
• Realmac Software
• Josh Mobley
• Meaningful Sound
• demo

Longitudinal Waves

Of Pressure

Through A Medium

Transverse Wave
Longitudinal Wave

• Longitudinal Waves
• Of Pressure
• Through a Medium
Sound

Properties of Waves
• A - Wavelength (Distance), Period (Time)
• B - Amplitude
• C - Frequency - 1/Period (measured in Hertz)
A
B
B
C 1 2 3

Fourier Series
• Based on his work ------->
• Periodic signals composed
of simple oscillating
functions
• Sine/Cosine waves are
fundamental

Oscillators
• Something that moves according to a periodic
function
• A physical body
• An electronic device

Basic Electronic Sound
oscillator
voltage
magnet
pressure
waves
speaker

Digital Audio
• Stream of Numbers
• Signal
• Frequency and Amplitude
• DAC - Digital to Analog Converter

Dynamic Audio

Procedural Audio
• Functions
• Objects
• Controls and Components
• Produce a Signal
• “Building Sound from First Principles”

Pure Data
• Procedural Audio
• Visual Programming Environment
• Free and Open Source
• pure-data.info

Pure Data 101

Signals and Values
• Signals are streams of numbers, usually audio
data
• Values are numbers or symbols

• Pd’s class
• or, kind of like a command line tool
Objects
osc~ 440
signal inlet inlet
signal outlet
class name arguments

Hot vs Cold Inlets
• Hot inlets make objects immediately act
• Cold inlets usually store a value
• Execution is right to left

• Signal connections are thick
• Others are thin
Connections
osc~ 440
*~
dac~
1

Messages
• Used for sending values without a connection
• [send] and [receive] objects
• Message boxes

libpd
• http://libpd.cc
• Peter Brinkmann - Google Engineer
• Enables you to embed an instance of Pd in your
app
• iOS and Android

Examples

Source Code
• github.com/thillerson/dynamic-sound-ios
• Examples of capabilities, not necessarily the
best architecture
• Goal: understand capabilities, have a place to
start

Example 1: A Simple Keyboard
• Sine wave oscillator
• Single voice
• Note on/off

Example 1: A Simple Keyboard Patch
osc~
dac~
receive midinote
mtof
receive gate
line~
*~
;
midinote 60;
gate 1;
;
gate 0$1 1
print

Example 2: A Game
• Top down perspective space shooter
• Code by Lucas Jordan
from iOS 5 Game Dev ->
• Now with sound!

Example 2: A Game Patch
load a square wave
at startup
Thruster
noise~
*~
samphold~
dac~
hip~ 200
phasor~ 800
line~
$1 200
*~ 0.1
receive thruster
BulletShip_HitAsteroid_Hit
samphold~
tabosc4~ square
dac~
line~
phasor~ 8000
tabosc4~ square
line~
unpack f f
0 $1
tabosc4~ square
line~
unpack f f
0 $1
hip~ 1000
unpack f f
0 $1
*~ 0.5
receive asteroid receive ship receive bullet
8-bit-ify
these sounds
cut low frequencies
comment
comment
Test_Messages
;
bullet 25 200;
;
thruster 0
;
thruster 1
pd wavesetup
;
ship 1000 50;
;
asteroid 200 100;

Further Reading
• puredata.info && libpd.cc
• youtube.com/user/cheetomoskeeto
• http://designingsound.org
• mitpress.mit.edu/books/designing-sound

Questions

Thank you!
Dynamic Audio • Tony Hillerson
• We’re Hiring!
• careers@tackmobile.com
• Excellent Team
• Awesome Projects
• Great Office

Dynamic Sound for iOS Apps and Games

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

More from Tony Hillerson

More from Tony Hillerson (9)

Recently uploaded

Recently uploaded (20)

Dynamic Sound for iOS Apps and Games