Presented at Google Extended I/O Event at Ahmedabad Java Meetup Group

1. GoogleIO @ Java Meetup Ahmedabad
Build next generation apps with eyes and ears using
Google Chrome
25th
Jun, 2014
Dhruv Gohil
From Ishi systems

2. Treat your users as human!
UX - 101
Tools should fit human not the other way around!

3. Now we have books on UX!
• With computers, we lost 'human touch' in
interaction
• Touch screens brought it back, but frankly its not
the natural way to do 'things'

4. Let's get back to basics..
• I have to tell you something.. I will speak and you
will listen.
• So why it has to be different for computers?!

5. Enter the new era!
• Now browsers can speak to you.
• And browsers can hear you too!
http://updates.html5rocks.com/2013/01/Voice-Driven-Web-Apps-Introduction-to-the-Web-Speech-API

6. Demo time!
https://www.google.com/intl/en/chrome/demos/speech.html

7. Do it yourself!

8. You talk too much...
● It's time to listen..
● Yes! That's all we need!

9. Off-line mode too!
http://syl22-00.github.io/pocketsphinx.js/

10. This is how translation should work!
http://www.moreawesomeweb.com/demos/speech_translate.html

11. Command mode!
● I am married.... ammm.. Can I command my computer to
do things?
https://www.talater.com/annyang/ https://github.com/TalAter/annyang

12. But can you see me?
http://wolframhempel.github.io/photobooth-js/

13. Ok but how its useful?
http://revealjs.herokuapp.com/
Time to demo!, again.! :-)
https://github.com/willy-vvu/reveal.js

14. What was that?
● Magic of 'Motion detection algorithms'
● Over simplified How To:
– Capture video [getUserMedia()]
– Draw frames [ html5 <canvas> ]
– Run Algorithm to detect change between frames [ .js]
● If you are deeply interested
– Read : http://labs.littlemissrobot.com/behind-the-mirror/
http://www.adobe.com/devnet/html5/articles/javascript-motion-detection.html

15. More!
● https://github.com/foo123/HAAR.js#live-examples
– Interactive Face Detection
– Many Faces Detection
– Interactive Mouth Detection
● http://www.soundstep.com/blog/experiments/jstracking/
– Motion tracking
● http://www.soundstep.com/blog/experiments/jsdetection/
– The Magical Xylophone

16. But Why chrome?

17. Its time to...

18. WebSpeech API fundametals
● Audio capture
● Request authorization
● Audio processing pip line
– Nodes (FFT , Gain, Filter...)
– http://chimera.labs.oreilly.com/books/1234000001552/
● Join to destination (Speakers)
● + Visualizations

19. GetUserMedia API fundamentals
● Audio + Video capture
● Request authorization
● Capture
● Draw (Hidden)
● Analyze + Process
● Re-Draw

20. And finally...
लगता है बर्फर िगर रही है!
https://github.com/IonicaBizau/WebcamSnowhttp://ionicabizau.net/WebcamSnow/