Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

2,086 views

Published on

Talk by Shwetank.
Using getUserMedia you can now access the camera and microphone using just plain old JavaScript! In this talk, we'll take a look at the various use cases for it and what all possibilities it opens up for web applications in the future.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,086
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

  1. 1. Mic, Camera, Action! Shwetank Dixit Opera Software
  2. 2. about me Web Evangelist, Web Standards Team Member, W3C Mobile Web for Social Development Group Twitter: @shwetank Email: shwetankd@opera.com
  3. 3. The web is awesome!
  4. 4. but Native apps sometimes have an edge But that edge is getting smaller
  5. 5. We’re adding more ways to get input from the user
  6. 6. We’re making existing ways easier Like web forms
  7. 7. We want to Stop Writing boring JS
  8. 8. We want to Start Writing exciting JS
  9. 9. Mic
  10. 10. Mic DEMO
  11. 11. Mic Web Audio API + getUserMedia
  12. 12. getUserMedia if (navigator.getUserMedia){ navigator.getUserMedia({audio: true}, a_success, a_error); }
  13. 13. getUserMedia Mic returns raw audio samples Cannot be played natively by the audio player Need Web Audio API to process it and then play it
  14. 14. getUserMedia Create audio context and media stream source
  15. 15. getUserMedia //Create audio context and media stream source var audioContext = new AudioContext(); // Prefix it for various browsers // Create an AudioNode from the stream. var mediaStreamSource = audioContext.createMediaStreamSource( stream );
  16. 16. getUserMedia //Create audio context and media stream source var audioContext = new AudioContext(); // Prefix it for various browsers // Create an AudioNode from the stream. var mediaStreamSource = audioContext.createMediaStreamSource( stream ); // Connect audio loop to hear your own voice mediaStreamSource.connect(context.destination);
  17. 17. Recorder.js Great library to help you record mic input using getusermedia
  18. 18. Recorder.js buffers the raw audio samples. converts it to format which can be played by an <audio> control
  19. 19. Camera
  20. 20. Use camera with getusermedia var options = {audio: false, video: true};
  21. 21. Use camera with getusermedia var options = {audio: false, video: true}; navigator.getUserMedia(options, v_success, v_error);
  22. 22. Use camera with getusermedia // In the HTML var video = document.querySelector('video'); .... ..... // In the JS Code function v_success(stream){ video.src = window.URL.createObjectURL(stream) || stream; }
  23. 23. Use camera + Video + Canvas for new tricks var button = document.querySelector('#button'); button.addEventListener('click',snapshot, false); ... ... function snapshot(){ var c = document.querySelector('canvas'); var ctx = c.getContext('2d'); var cw = c.clientWidth; var ch = c.clientHeight; ctx.drawImage(video_element, 0, 0, cw, ch); }
  24. 24. Demo Notice you can click on a polaroid and save that image. You can create data URIs of canvases
  25. 25. Manipulating the image from the webcam
  26. 26. Once you get the snapshot of the webcam into canvas you can go crazy with it
  27. 27. DEMO Warholiser
  28. 28. Action!
  29. 29. Tracking user actions Motion detection gesture recognition
  30. 30. headtrackr.js https://github.com/auduno/headtrackr/
  31. 31. headtrackr.js http://auduno.tumblr.com/post/25125149521/head-tracking-with- webrtc
  32. 32. Face Detection X, Y position of face Width, Height of face relative to canvas headtrackr.js
  33. 33. Demo
  34. 34. Voice driven apps Voice modulation apps Native photo capture in web apps Live image manipulation and effects Video conferencing (using WebRTC) Accessibility enhancements to apps Possible use cases
  35. 35. Cheers! More questions? Ask me now or contact me at: shwetankd@opera.com or on twitter - @shwetank

×