Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Web & sound

933 views

Published on

A brief about Web Audio and Web MIDI APIs that were added to the open web platform recently

Published in: Technology
  • Be the first to comment

Web & sound

  1. 1. Web & Sound Playing with Sound APIs recently introduced to the Web
  2. 2. Overview + Demo
  3. 3. New Sound APIs ◇ Web Audio (http://webaudio.github.io/web-audio-api/) ■ Real-time audio synthesis ○ cf. SuperCollider, Pure Data, Max/MSP ■ SFX (Pan, Delay, Filter, etc) ◇ Web MIDI (http://webaudio.github.io/web-midi-api/) ■ Real-time communication ○ Web ⇄ Musical Instruments ● Synths, DJ and lighting controllers ◇ WebRTC (http://w3c.github.io/webrtc-pc/) ■ Camera & Microphone
  4. 4. Web Audio Demo ◇ Web Audio Playground (http://webaudioplayground.appspot.com/) ■ Straight-forward GUI wrapper ◇ WebPd (https://github.com/sebpiq/WebPd) ■ Pure Data clone ◇ CoffeeCollider (https://github.com/mohayonao/CoffeeCollider) ■ Coffee Script + SuperCollider ◇ T’SS JS edition (https://github.com/toyoshim/tss) ■ Chiptune sound emulation (demo, tsdsp)
  5. 5. MIDI is old, and new ◇ 30th anniversary ◇ MIDI is not only SMF but a protocol ■ SMF: Standard MIDI File ○ Protocol dump file with timestamps ■ Real-time communication - lighting and more ○ 冨田勲×初音ミク「イーハトーヴ」@ YouTube ◇ De facto for music production tools ■ Software Synth in DAW: Audio Unit, VSTi, DXi ■ Hardware: DIN, USB, FireWire, RTP, Bluetooth(*) ○ (*): supported by Yosemite, Windows 10, Android M
  6. 6. Web MIDI Demo ◇ Physical controller for online sound systems ■ Web Audio Synth v2 (http://aikelab.net/websynthv2/) ■ Web FM Synthesizer (http://www.taktech.org/takm/WebFMSynth/) ■ Online DAW: audiotool (http://www.audiotool.com/) ■ Online VJ: MajVj (GGT 2014 demo) ◇ Demo and Productions by YAMAHA ■ Pocket Miku (Product page & Web App) ■ Web Music Platform (toyoshim’s fork) ■ Reface + Soundmondo (Product page) ○ Social to share your sound and music via Web MIDI
  7. 7. WebRTC ◇ microphone ■ getUserMedia({audio:true}, …) ○ Can connect to Web Audio via createMediaStreamSource() ◇ camera ■ getUserMedia({video: true}... ) ○ Can use it as a texture for WebGL
  8. 8. Web Apps be Secure! ◇ Hardware is hard to use securely ■ What happens on WebGL? ○ Need GPU process and GPU command validator ■ Recent APIs to use hardware ○ Permission prompt (infobar/bubble UI) ○ Secure origin, e.g. https or localhost, is mandatory ■ Real P0 issue happened on Web MIDI ○ Microsoft GS Wavetable SW Synth crashes ● on receiving a special valid sequence ● outside Chrome sandbox, inside the driver
  9. 9. How to learn ◇ Web Audio ■ Getting Started with Web Audio API html5rocks ■ Web Audio API Ryoya Kawai ■ Web Audio API 解説 g200kg ◇ Web MIDI ■ Web MIDI API (x-webmidi編) Ryoya Kawai ■ ブラウザで電子楽器を作ってみよう HTML5 Experts ■ 最高に乱暴なWeb MIDI API利用方法 tadfmac
  10. 10. Lesson: Web Audio 1 ◇ Simple sine wave var audio = new AudioContext(); var osc = audio.createOscillator(); osc.frequency.value = 440; // in Hz osc.connect(audio.destination); osc.start(0);
  11. 11. Lesson: Web Audio 2 ◇ Other nodes ■ Audio source ○ AudioBufferSourceNode (from Audio resources, e.g., mp3, m4a) ○ MediaElementAudioSourceNode (from DOM media object) ○ MediaStreamSource (from microphone via getUserMedia) ○ ScriptProcessorNode (deprecated) => AudioWorkerNodeProcessor (from JS) ○ OscillatorNode (from various kinds of oscillators) ■ Effect ○ GainNode ○ DelayNode ○ (Stereo)PannerNode / AudioListener ○ ConvolverNode (e.g., Vocoder, Reverb, and so on) ○ DynamicsCompressorNode / WaveShaperNode ○ BuquadFilterNode (e.g., LPF, HPF, BPF, and so on) ■ Misc ○ AnalyserNode (FFT) ○ ChannelSplitterNode / ChannelMergerNode (Mixer)
  12. 12. Lesson: Web MIDI 1 ◇ Device Query var promise = navigator.requestMIDIAccess(); promise.then(function(midi) { for (var input of midi.inputs.values()) console.log(input); // MIDIInput for (var output of midi.outputs.values()) console.log(output); // MIDIOutput });
  13. 13. Lesson: Web MIDI 2 ◇ MIDIInput for (var input of midi.inputs.values()) input.onmidimessage = handler; function handler (e) { // Single MIDI message in Array. console.log(e.data); }
  14. 14. Lesson: Web MIDI 3 ◇ MIDIOutput for (var output of midi.outputs.values()) { // Send single MIDI message or multiple // MIDI messages. output.send([0x90, 0x40, 0x7f]); // Note ON }
  15. 15. Lesson: Promise ◇ New replacement for callbacks foo(successCallback, errorCallback); ↓ foo().then(successCallback, errorCallback); foo().then(bar) .then(baz) .catch(qux); ◇ Chaining Promise.all([foo(), bar()]) .then(baz, qux); ◇ Wait multiple async functions
  16. 16. Lesson: Maplike ◇ readonly maplike<DOMString, MIDIInput> inputs ◇ readonly maplike<DOMString, MIDIOutput> outputs ◇ keys() … MIDIPort.id is used as an unique key ◇ values() … MIDIInput / MIDIOutput for (var input of inputs.values()) { // New syntax for ES6 iterable }
  17. 17. Lesson: Web MIDI 4 ◇ Hotplug midi.onstatechange = function (e) { var newport = e.port; if (newport.state == “connected”) console.log(newport.name + “ is attached”; else if (newport.state == “disconnected”) console.log(newport.name + “ is detached”; };
  18. 18. Fine ||
  19. 19. References ◇ Slide template ■ from Slides Carnival ○ http://www.slidescarnival.com/imogen-free- presentation-template/399

×