Sophie introduces building a virtual piano using Javascript and the Web Audio API. She discusses music theory fundamentals like notes, scales, and frequencies. She explains how to generate waveforms and play notes by calculating their frequencies based on A4=440Hz. The presentation describes creating an oscillator, waveforms, and interface to play notes. Sophie also mentions using the Web MIDI API to play notes from an external MIDI device. Overall, the document provides an overview of creating a virtual piano app using web technologies.
USE CHROME!!!
This is a talk to show you the potential of the web
The link in the corner?
tell you how I built it
the musical theory behind it,
show you a bit of the APIs that power it - the Web Audio API and Web MIDI API
First, let’s see what it does
Start with musical theory, science
This theory actually went into building the app
I know you’re thinking “this is a web conf!”- it will all become clear
Sound = vibrations that travel as waves.
The number of vibrations in a set time period = frequency of the sound
frequency is measured in Hertz which is one cycle per second.
Keys on a piano - hammers on strings - vibrate at frequencies
Western music - set of tones with specific frequencies
UK and US has 12 notes - each with a letter, A to G. Division based on frequency
Only 7? 5 accidentals.
similar to the ones either side, so they don’t get their own letter. Instead we give them a symbol and call them sharp or flat
Talk about notes as on keyboard, starting from C
keyboard has 61 and starts from C, piano has 88 notes and starts from A
Repeat in groups from lowest to highest
These groups of repeating letters are called octaves
61 keys - 5 octaves
88 keys - 7 ¼ octaves.
A4 and A5 demo (next)
Weird ability of brain - same note in diff octaves sounds same
- same note but different pitch (high/low)
NEXT
Each octave jump - double freq
440Hz -> 880Hz
- Music tends to use only some notes, not all of them.
- scales are ordered sequence of notes - a set of tones you can use to build music
- like a colour palette - emotions
- music uses notes of particular scale - key
This picture shows two octaves of the C major scale written in musical notation.
Scales follow strict patterns
G major - pattern for major scale. Different notes but same pattern as the one I just showed you
One semitone or half tone = distance between one note and its neighbour
Tone/whole tone = two semitones
Tone, tone, semitone, tone, tone, tone, semitone.
Every major scale
Other scales have different patterns but are fundamentally the same
Okay! Time for some JavaScript. Enough musical theory. Let’s talk about the Web Audio API.
Play audio
Mix
Analyse
Visualise
Synthesize
Recap: sound waves + frequencies
Synthesizers generate these waves electronically to make sound.
We’re basically going to build a little synthesizer.
Here’s what we need…
Circuit that produces alternating waveform -> constant tone
Define the waveform
Makes a particular sound
frequencies of all the notes on the piano
to tell the synthesizer what note to play
And then we need to actually build those piano keys.
initialise an AudioContext.
This is like a big graph that contains lots of audio modules linked together
exists on the window object in the big 4 browsers.
only want one of these in the app: multiple -> lots of sounds at once
createOscillator
Connect to audio context destination (default system output)
Not doing anything yet
Create waveform
The shape of this waveform determines the sound that’s produced.
4 really common waveforms - sine, square, triangle, sawtooth.
a sine wave will make a very different sound to a square wave, for example (play audio).
Sine wave
Square
Triangle
Sawtooth
I chose triangle
Start oscillator
App playing fixed tone @ 440Hz. How do we do different tone?
We can calculate them using a formula§
A4 is an important note
A above middle C on the piano - 4th octave
Standardised as ISO-16 - standard orchestral tuning frequency
Nice round number
Formula to calculate a note’s frequency based on its position relative to another note, which we know the frequency of
Makes sense to use 440Hz as a constant
where 𝑛 is the number of semitones between A4 and the note we’re looking for.
For example, the note C5 is the next C above A4 on the piano. If you count it out, that’s a difference of 3 semitones.
Translated equation to code.
This function takes a letter and number combination -> freq
Get interval between A and our note within the same octave
Get octave interval - octave difference multiplied by semitones
Add together
Translated equation to code.
This function takes a letter and number combination -> freq
Get interval between A and our note within the same octave
Get octave interval - octave difference multiplied by semitones
Add together
Run equation to calculate frequency
For convenience we round it to 1 decimal place.
Once we have the frequency, it’s simple to set the oscillator frequency. So now we can make an oscillator play whatever note we want.
Now can build piano
Did the OG one in React but had problems with things getting out of sync
Vanilla JS has been much easier
Each key is a button, positioned with CSS grid
Each note has ID - index of array of notes
Title attribute
Each key has event listeners for mouse down and mouse up
We either play or stop the note
Pedal - don’t stop on mouse up
Playing a note - new audio channel for each note press
Create a gain node to ramp the sound up and down to make it sound a bit smoother
CLICK
We set the gain to zero to start with and ramp it up over 0.06s - if you’re familiar with music terms this is controlling the attack of the note, the onset
We init our oscillator, and connect it to the gain node
Think of these like a chain of modules, each one has to be connected to the last
Then we set the wave type and the frequency,
When osc stops, disconnect gain node
Garbage collection
When we click a note, init audio - need to respond to user gesture - then find note and init channel, then start osc
Storing all the channels in global var
Stopping a note - on mouse up, ramp down to 0 very quickly then stop oscillator
Can’t repeatedly stop and start oscillators - need a new one for each one
Oscillator disconnects itself when turned off.
Every time I play a note I overwrite the channel for that note so the old one will get GC’d
Automatically play notes of scale
Patterns of scale -> generate notes
Table of intervals for scales
Each number represents a semitone interval from the previous note
Start at 0 - root note
Recap major scale pattern as numbers
Generating scales
Generate octave
Iterate through scale pattern
Cursor currentPosition that keeps track of which note we’re on based on index
Add each interval to currentPosition to find next note
Playing scale: change oscillator frequency at regular intervals
Only want one oscillator
Set timeouts per note - 0.5 sec - multiply by index of note in array so staggered
Timeout elapses- osc frequency changes
Store timeouts in a global variable if we want to stop
For chords: different patterns, fewer notes, same principle.
Get the oscillator to stop playing 2 seconds in
Find the frequencies new oscillator for each note.
We’ve got it to…
This is a midi controller...
Open browser and demo midi controller
Connects to Musical Instrument Digital Interface devices
Chrome, Edge and Opera - not Firefox
When you press a note on the controller it sends a message to the computer with info about what was pressed
The browser is listening for those messages
The data we get from the midi message looks like this
Command, note, velocity
Ignoring velocity
The two commands we care about
Midi standard assigns numbers to each key
Default for my midi controller is octaves 3+4, starting at 48
My keyboard has 0-24
When I get a number, have to subtract 48 to get index
Midi standard assigns numbers to each key
Default for my midi controller is octaves 3+4, starting at 48
My keyboard has 0-24
When I get a number, have to subtract 48 to get index
One more demo for good measure
