Mic, Camera,
Action!
Shwetank Dixit
Opera Software
about me
Web Evangelist, Web
Standards Team
Member, W3C Mobile Web
for Social Development
Group
Twitter: @shwetank
Email:
...
The web is awesome!
but Native apps
sometimes have an edge
But that edge is getting smaller
We’re adding more ways
to get input from the
user
We’re making existing
ways easier
Like web forms
We want to
Stop
Writing boring JS
We want to
Start
Writing exciting JS
Mic
Mic
DEMO
Mic
Web Audio API + getUserMedia
getUserMedia
if (navigator.getUserMedia){
	
 navigator.getUserMedia({audio: true}, a_success, a_error);
}
getUserMedia
Mic returns raw audio samples
Cannot be played natively by the audio player
Need Web Audio API to process it ...
getUserMedia
Create audio context and media stream source
getUserMedia
//Create audio context and media stream source
var audioContext = new AudioContext(); // Prefix it for variou...
getUserMedia
//Create audio context and media stream source
var audioContext = new AudioContext(); // Prefix it for variou...
Recorder.js
Great library to help you record mic
input using getusermedia
Recorder.js
buffers the raw audio samples.
converts it to format which can be
played by an <audio> control
Camera
Use camera with
getusermedia
var options = {audio: false, video: true};
Use camera with
getusermedia
var options = {audio: false, video: true};
navigator.getUserMedia(options, v_success, v_error...
Use camera with
getusermedia
// In the HTML
var video = document.querySelector('video');
....
.....
// In the JS Code
func...
Use camera + Video +
Canvas for new tricks
var button = document.querySelector('#button');
button.addEventListener('click'...
Demo
Notice you can click on a polaroid
and save that image.
You can create data URIs of canvases
Manipulating the image from the webcam
Once you get the
snapshot of the webcam
into canvas
you can go crazy with it
DEMO
Warholiser
Action!
Tracking user actions
Motion detection
gesture recognition
headtrackr.js
https://github.com/auduno/headtrackr/
headtrackr.js
http://auduno.tumblr.com/post/25125149521/head-tracking-with-
webrtc
Face Detection
X, Y position of face
Width, Height of face relative to canvas
headtrackr.js
Demo
Voice driven apps
Voice modulation apps
Native photo capture in web apps
Live image manipulation and effects
Video confere...
Cheers!
More questions? Ask me now
or contact me at:
shwetankd@opera.com
or on twitter - @shwetank
Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action
Upcoming SlideShare
Loading in...5
×

Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

1,445

Published on

Talk by Shwetank.
Using getUserMedia you can now access the camera and microphone using just plain old JavaScript! In this talk, we'll take a look at the various use cases for it and what all possibilities it opens up for web applications in the future.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,445
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

  1. 1. Mic, Camera, Action! Shwetank Dixit Opera Software
  2. 2. about me Web Evangelist, Web Standards Team Member, W3C Mobile Web for Social Development Group Twitter: @shwetank Email: shwetankd@opera.com
  3. 3. The web is awesome!
  4. 4. but Native apps sometimes have an edge But that edge is getting smaller
  5. 5. We’re adding more ways to get input from the user
  6. 6. We’re making existing ways easier Like web forms
  7. 7. We want to Stop Writing boring JS
  8. 8. We want to Start Writing exciting JS
  9. 9. Mic
  10. 10. Mic DEMO
  11. 11. Mic Web Audio API + getUserMedia
  12. 12. getUserMedia if (navigator.getUserMedia){ navigator.getUserMedia({audio: true}, a_success, a_error); }
  13. 13. getUserMedia Mic returns raw audio samples Cannot be played natively by the audio player Need Web Audio API to process it and then play it
  14. 14. getUserMedia Create audio context and media stream source
  15. 15. getUserMedia //Create audio context and media stream source var audioContext = new AudioContext(); // Prefix it for various browsers // Create an AudioNode from the stream. var mediaStreamSource = audioContext.createMediaStreamSource( stream );
  16. 16. getUserMedia //Create audio context and media stream source var audioContext = new AudioContext(); // Prefix it for various browsers // Create an AudioNode from the stream. var mediaStreamSource = audioContext.createMediaStreamSource( stream ); // Connect audio loop to hear your own voice mediaStreamSource.connect(context.destination);
  17. 17. Recorder.js Great library to help you record mic input using getusermedia
  18. 18. Recorder.js buffers the raw audio samples. converts it to format which can be played by an <audio> control
  19. 19. Camera
  20. 20. Use camera with getusermedia var options = {audio: false, video: true};
  21. 21. Use camera with getusermedia var options = {audio: false, video: true}; navigator.getUserMedia(options, v_success, v_error);
  22. 22. Use camera with getusermedia // In the HTML var video = document.querySelector('video'); .... ..... // In the JS Code function v_success(stream){ video.src = window.URL.createObjectURL(stream) || stream; }
  23. 23. Use camera + Video + Canvas for new tricks var button = document.querySelector('#button'); button.addEventListener('click',snapshot, false); ... ... function snapshot(){ var c = document.querySelector('canvas'); var ctx = c.getContext('2d'); var cw = c.clientWidth; var ch = c.clientHeight; ctx.drawImage(video_element, 0, 0, cw, ch); }
  24. 24. Demo Notice you can click on a polaroid and save that image. You can create data URIs of canvases
  25. 25. Manipulating the image from the webcam
  26. 26. Once you get the snapshot of the webcam into canvas you can go crazy with it
  27. 27. DEMO Warholiser
  28. 28. Action!
  29. 29. Tracking user actions Motion detection gesture recognition
  30. 30. headtrackr.js https://github.com/auduno/headtrackr/
  31. 31. headtrackr.js http://auduno.tumblr.com/post/25125149521/head-tracking-with- webrtc
  32. 32. Face Detection X, Y position of face Width, Height of face relative to canvas headtrackr.js
  33. 33. Demo
  34. 34. Voice driven apps Voice modulation apps Native photo capture in web apps Live image manipulation and effects Video conferencing (using WebRTC) Accessibility enhancements to apps Possible use cases
  35. 35. Cheers! More questions? Ask me now or contact me at: shwetankd@opera.com or on twitter - @shwetank
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×