Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Music is the Soul 
The Web is the Platform 
@sydlawrence 
@wemakeawesomesh
DISCLAIMER 
There are no GIFs 
@sydlawrence 
@wemakeawesomesh
DISCLAIMER 
There are no GIFs 
@sydlawrence 
@wemakeawesomesh
Audio Playback 
Audio Metadata 
Audio Analysis 
Audio Filters 
Audio Synthesis 
Web MIDI 
@sydlawrence 
@wemakeawesomesh
Audio Playback 
We are literally swimming in music 
@sydlawrence 
@wemakeawesomesh
Alright, not literally
Simple MP3 Playback 
@sydlawrence 
@wemakeawesomesh 
var audio = new Audio(); 
audio.src = 'mysong.mp3'; 
audio.play();
Soundcloud Playback 
SC.stream('/tracks/6049255', function(sound){ 
@sydlawrence 
@wemakeawesomesh 
sound.play(); 
}); 
ht...
rdio Playback 
var player = document.getElementById('rdio'); 
player.rdio_play('http://rd.io/x/QUxMDDdzOP0/'); 
@sydlawren...
Deezer Playback 
@sydlawrence 
@wemakeawesomesh 
http://developers.deezer.com/ 
DZ.player.playTracks([82230030]);
Spotify Playback 
@sydlawrence 
@wemakeawesomesh 
https://developer.spotify.com/ 
Computer says no 
Emeddable widget only
Toma.hk Playback 
@sydlawrence 
http://toma.hk/api.html 
var track = window.tomahkAPI.Track( 
'Never Gonna Give You Up','R...
Audio Metadata 
“Get me all the songs by Rick Astley” 
@sydlawrence 
@wemakeawesomesh
the echonest 
http://developer.echonest.com/api/v4/artist/songs 
?api_key=<YOUR_API_KEY> 
&name=rick+astley 
&format=jsonp...
Music Graph 
http://api.musicgraph.com/api/v2/artist/e41999c6-a6b5-11e0-b446-00251188dd67/tracks 
?api_key=<YOUR_API_KEY> ...
https://cXXXXXXX.web.cddbp.net/webapi/xml/1.0/ 
! 
<QUERIES> 
<LANG>eng</LANG> 
<AUTH> 
<CLIENT>XXXXXXX-XXXXXXXXXXXXXXXXXX...
gracenote 
SUPER 
SIMPLE* 
*sarcasm 
@sydlawrence 
@wemakeawesomesh
@sydlawrence 
@wemakeawesomesh 
Audio Analysis 
When just simply playing 
music isn’t enough
AudioContext 
@sydlawrence 
@wemakeawesomesh 
AudioContext 
Source ? Destination
AudioContext Buffer 
var audioBuffer = null; 
window.AudioContext = window.AudioContext || window.webkitAudioContext; 
var...
AudioContext Playback 
@sydlawrence 
// creates a sound source 
var source = context.createBufferSource(); 
! 
// tell the...
@sydlawrence 
@wemakeawesomesh 
FFT 
A fast Fourier transform (FFT) is an efficient algorithm to compute the 
discrete Fou...
@sydlawrence 
@wemakeawesomesh 
FFT
FFT 
SUPER 
SIMPLE* 
*sarcasm 
@sydlawrence 
@wemakeawesomesh
FFT 
@sydlawrence 
@wemakeawesomesh
@sydlawrence 
@wemakeawesomesh
@sydlawrence 
@wemakeawesomesh 
dancer.js 
var dancer = new Dancer(); 
! 
dancer.load(audio); 
! 
dancer.bind( 'update', f...
@sydlawrence 
Here’s something 
I made earlier 
@wemakeawesomesh 
wmas.it/musicviz/ripple/
@sydlawrence 
@wemakeawesomesh 
Audio Filters 
Let’s pretend we are super star DJS
Setup the audio node 
@sydlawrence 
var audioContext = new webkitAudioContext; 
! 
var gainNode = audioContext.createGain(...
@sydlawrence 
@wemakeawesomesh 
Set up the filter 
// create the filter 
var lowPassFilter = audioContext.createBiquadFilt...
Add the filter to the audio 
gainNode.connect(lowPassFilter); 
@sydlawrence 
@wemakeawesomesh
Audio Filters 
SUPER 
SIMPLE* 
*sarcasm 
@sydlawrence 
@wemakeawesomesh
@sydlawrence 
Here’s something 
I made earlier* 
@wemakeawesomesh 
wmas.it/vineboard 
! 
! 
*I didn’t make it, 
@skattyadz...
@sydlawrence 
@wemakeawesomesh 
Audio Synthesis 
Now we can pretend 
we are musicians
@sydlawrence 
@wemakeawesomesh 
Tone Synthesis 
var context = new webkitAudioContext(), 
oscillator = context.createOscill...
@sydlawrence 
@wemakeawesomesh 
Tone Synthesis 
http://codepen.io/sydlawrence/pen/lruEy
@sydlawrence 
Here’s something 
I made earlier* 
@wemakeawesomesh 
somakeit.github.io/potzy 
! 
! 
*I didn’t make it, 
@Be...
@sydlawrence 
@wemakeawesomesh 
Web MIDI 
Let’s use instruments with the web
@sydlawrence 
@wemakeawesomesh 
What is MIDI? 
! 
MIDI is just a messaging protocol 
You can send MIDI messages and receiv...
@sydlawrence 
It is experimental! 
Behind a flag in chrome 
OR Install a browser plugin 
http://jazz-soft.net/doc/Jazz-Plu...
navigator.requestMIDIAccess().then(success,error); 
! 
var success = function(midi) { 
inputs = midi.inputs(); 
outputs = ...
input.onmidimessage = function(e) { 
var message = e.data; // [ channel, key, velocity ] 
}; 
! 
output.send( [ channel, k...
Web MIDI 
SUPER 
SIMPLE* 
*NO SARCASM!!!!! 
@sydlawrence 
@wemakeawesomesh
@sydlawrence 
Here’s something 
I made earlier 
@wemakeawesomesh 
wmas.it/midi
Thank you for listening 
@sydlawrence 
@wemakeawesomesh
Music is the Soul - The Web is the Platform FOWA London 2014
Upcoming SlideShare
Loading in …5
×

Music is the Soul - The Web is the Platform FOWA London 2014

1,235 views

Published on

The slides accompanying my talk at the Future of Web Apps 2014 in London. A whistle stop tour of the experimental audio apis available in the browser, and some vendor APIs to accompany web based audio / music.

Published in: Technology
  • Be the first to comment

Music is the Soul - The Web is the Platform FOWA London 2014

  1. 1. Music is the Soul The Web is the Platform @sydlawrence @wemakeawesomesh
  2. 2. DISCLAIMER There are no GIFs @sydlawrence @wemakeawesomesh
  3. 3. DISCLAIMER There are no GIFs @sydlawrence @wemakeawesomesh
  4. 4. Audio Playback Audio Metadata Audio Analysis Audio Filters Audio Synthesis Web MIDI @sydlawrence @wemakeawesomesh
  5. 5. Audio Playback We are literally swimming in music @sydlawrence @wemakeawesomesh
  6. 6. Alright, not literally
  7. 7. Simple MP3 Playback @sydlawrence @wemakeawesomesh var audio = new Audio(); audio.src = 'mysong.mp3'; audio.play();
  8. 8. Soundcloud Playback SC.stream('/tracks/6049255', function(sound){ @sydlawrence @wemakeawesomesh sound.play(); }); https://developers.soundcloud.com/
  9. 9. rdio Playback var player = document.getElementById('rdio'); player.rdio_play('http://rd.io/x/QUxMDDdzOP0/'); @sydlawrence @wemakeawesomesh http://www.rdio.com/developers/
  10. 10. Deezer Playback @sydlawrence @wemakeawesomesh http://developers.deezer.com/ DZ.player.playTracks([82230030]);
  11. 11. Spotify Playback @sydlawrence @wemakeawesomesh https://developer.spotify.com/ Computer says no Emeddable widget only
  12. 12. Toma.hk Playback @sydlawrence http://toma.hk/api.html var track = window.tomahkAPI.Track( 'Never Gonna Give You Up','Rick Astley', { @wemakeawesomesh disabledResolvers: ['spotify'] } ); $('body').append(track.render()); track.play();
  13. 13. Audio Metadata “Get me all the songs by Rick Astley” @sydlawrence @wemakeawesomesh
  14. 14. the echonest http://developer.echonest.com/api/v4/artist/songs ?api_key=<YOUR_API_KEY> &name=rick+astley &format=jsonp &callback=JSONP_CALLBACK @sydlawrence @wemakeawesomesh http://developer.echonest.com/
  15. 15. Music Graph http://api.musicgraph.com/api/v2/artist/e41999c6-a6b5-11e0-b446-00251188dd67/tracks ?api_key=<YOUR_API_KEY> @sydlawrence @wemakeawesomesh https://developer.musicgraph.com/
  16. 16. https://cXXXXXXX.web.cddbp.net/webapi/xml/1.0/ ! <QUERIES> <LANG>eng</LANG> <AUTH> <CLIENT>XXXXXXX-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</CLIENT> <USER>XXXXXXXXXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</USER> </AUTH> <QUERY CMD="ALBUM_SEARCH"> <TEXT TYPE=“ARTIST">rick astley</TEXT> </QUERY> </QUERIES> gracenote @sydlawrence @wemakeawesomesh https://developer.gracenote.com/
  17. 17. gracenote SUPER SIMPLE* *sarcasm @sydlawrence @wemakeawesomesh
  18. 18. @sydlawrence @wemakeawesomesh Audio Analysis When just simply playing music isn’t enough
  19. 19. AudioContext @sydlawrence @wemakeawesomesh AudioContext Source ? Destination
  20. 20. AudioContext Buffer var audioBuffer = null; window.AudioContext = window.AudioContext || window.webkitAudioContext; var context = new AudioContext(); ! var request = new XMLHttpRequest(); request.open('GET', 'mysound.mp3', true); request.responseType = 'arraybuffer'; ! request.onload = function() { context.decodeAudioData(request.response, function(buffer) { @sydlawrence @wemakeawesomesh audioBuffer = buffer; }, onError); }; request.send();
  21. 21. AudioContext Playback @sydlawrence // creates a sound source var source = context.createBufferSource(); ! // tell the source which sound to play source.buffer = audioBuffer; ! // connect the source to the context's destination source.connect(context.destination); source.start(0); @wemakeawesomesh
  22. 22. @sydlawrence @wemakeawesomesh FFT A fast Fourier transform (FFT) is an efficient algorithm to compute the discrete Fourier transform (DFT) of an input vector. Efficient means that the FFT computes the DFT of an n-element vector in O(nlog n) operations in contrast to the O(n2) operations required for computing the DFT by definition.
  23. 23. @sydlawrence @wemakeawesomesh FFT
  24. 24. FFT SUPER SIMPLE* *sarcasm @sydlawrence @wemakeawesomesh
  25. 25. FFT @sydlawrence @wemakeawesomesh
  26. 26. @sydlawrence @wemakeawesomesh
  27. 27. @sydlawrence @wemakeawesomesh dancer.js var dancer = new Dancer(); ! dancer.load(audio); ! dancer.bind( 'update', function() { var magnitude = this.getFrequency(startF, endF); }); http://jsantell.github.io/dancer.js/
  28. 28. @sydlawrence Here’s something I made earlier @wemakeawesomesh wmas.it/musicviz/ripple/
  29. 29. @sydlawrence @wemakeawesomesh Audio Filters Let’s pretend we are super star DJS
  30. 30. Setup the audio node @sydlawrence var audioContext = new webkitAudioContext; ! var gainNode = audioContext.createGain(); ! gainNode.connect(audioContext.destination); @wemakeawesomesh
  31. 31. @sydlawrence @wemakeawesomesh Set up the filter // create the filter var lowPassFilter = audioContext.createBiquadFilter(); lowPassFilter.frequency.value = frequency; ! // create the wave shaper var waveShaper = audioContext.createWaveShaper(); ! // connect the bits together var waveShaper.connect(lowPassFilter); lowPassFilter.connect(audioContext.destination); ! // create the curve array and set it to the shaper var wsCurve = new Float32Array(22050); // .. add floats to the curve waveShaper.curve = wsCurve;
  32. 32. Add the filter to the audio gainNode.connect(lowPassFilter); @sydlawrence @wemakeawesomesh
  33. 33. Audio Filters SUPER SIMPLE* *sarcasm @sydlawrence @wemakeawesomesh
  34. 34. @sydlawrence Here’s something I made earlier* @wemakeawesomesh wmas.it/vineboard ! ! *I didn’t make it, @skattyadz & @robhampson did
  35. 35. @sydlawrence @wemakeawesomesh Audio Synthesis Now we can pretend we are musicians
  36. 36. @sydlawrence @wemakeawesomesh Tone Synthesis var context = new webkitAudioContext(), oscillator = context.createOscillator(); ! oscillator.type = 0; // sine wave oscillator.frequency.value = 2000; oscillator.connect(context.destination); oscillator.noteOn && oscillator.noteOn(0);
  37. 37. @sydlawrence @wemakeawesomesh Tone Synthesis http://codepen.io/sydlawrence/pen/lruEy
  38. 38. @sydlawrence Here’s something I made earlier* @wemakeawesomesh somakeit.github.io/potzy ! ! *I didn’t make it, @Benjie & @MichDdev did
  39. 39. @sydlawrence @wemakeawesomesh Web MIDI Let’s use instruments with the web
  40. 40. @sydlawrence @wemakeawesomesh What is MIDI? ! MIDI is just a messaging protocol You can send MIDI messages and receive MIDI messages They consist of channel, key and velocity
  41. 41. @sydlawrence It is experimental! Behind a flag in chrome OR Install a browser plugin http://jazz-soft.net/doc/Jazz-Plugin/ @wemakeawesomesh
  42. 42. navigator.requestMIDIAccess().then(success,error); ! var success = function(midi) { inputs = midi.inputs(); outputs = midi.outputs(); }; ! var error = function() { console.log('oops'); }; @sydlawrence Request MIDI Access @wemakeawesomesh
  43. 43. input.onmidimessage = function(e) { var message = e.data; // [ channel, key, velocity ] }; ! output.send( [ channel, key, velocity ] ); @sydlawrence Send & Receive MIDI @wemakeawesomesh
  44. 44. Web MIDI SUPER SIMPLE* *NO SARCASM!!!!! @sydlawrence @wemakeawesomesh
  45. 45. @sydlawrence Here’s something I made earlier @wemakeawesomesh wmas.it/midi
  46. 46. Thank you for listening @sydlawrence @wemakeawesomesh

×