Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Computer Vision - now working

in over 2 Billion Web Browsers!
Rob Manson

CEO & co-founder
Sebastian Montabone

Computer ...
So what is Mixed Reality?
Here’s a short demo of Milgram’s Mixed Reality Continuum - all running in a browser.
awe.media
A brief/biased history of Computer Vision
1957 - Russel A. Kirsch scans first photo with a computer
1960 - Larry Roberts pu...
How does Computer Vision

work in the browser?
awe.media
camera -> gUM -> video -> canvas -> pixels -> vision algorithms
HTMLVideoElement
This is a container for decoding and presenting video streams.
This brought plugin free video to the web....
awe.media
Canvas, WebGL & the ArrayBuffer
The 2D Canvas gave us the ability to convert a video stream into pixel data.
Web...
awe.media
JSARToolkit
In 2011 Billinghurst & Kato's ARToolkit was ported to Javascript.
awe.media
Enter WebRTC's getUserMedia()
Some claim this has a latency that makes the web unusable for AR.

But here’s the ...
awe.media
WebRTC's getUserMedia()
FAST feature detection & Tigerstail in 2012
awe.media
WebRTC's getUserMedia()
Tracking.js released in 2012
awe.media
WebRTC's getUserMedia()
AR.js released in 2017
awe.media
Transpiling OpenCV
This brings a more general computer vision toolkit to the web!
Demo Time!
awe.media
awe.media
But there's no gUM on iOS?
For Vision based functionality we fallback to Visual Search
For Location based apps w...
Computer Vision - now working
 in over 2 Billion Web Browsers!
Upcoming SlideShare
Loading in …5
×

Computer Vision - now working
 in over 2 Billion Web Browsers!

1,206 views

Published on

This was presented at Augmented World Expo in Santa Clara (#AWE2017). A video of the live Natural Feature Tracking demo will be uploaded and linked from here soon.

The key benefit of using AR in your web browser is how quick and easy it is to share. You can send a single web link through social media or email and the recipient can just tap on the link and it works. But up until recently this has not included Computer Vision based AR for a number of technical and market reasons. This presentation will position the web browser in the overall context of Computer Vision history, and we'll look at how this has evolved through developments including jsartoolkit.js, tracking.js and AR.js. We'll then dive deeper into the latest developments to show how OpenCV performs running in the browser and how this compares to native applications. This deep dive will compare the different feature detection/extraction algorithms and how they perform on some well known image data sets. The session will conclude with demos that show how this all works right now in over 2 Billion capable web browsers.

Published in: Internet
  • Be the first to comment

Computer Vision - now working
 in over 2 Billion Web Browsers!

  1. 1. Computer Vision - now working
 in over 2 Billion Web Browsers! Rob Manson
 CEO & co-founder Sebastian Montabone
 Computer Vision Engineer Mixed Reality. In the web. On any device. https://try.awe.media
  2. 2. So what is Mixed Reality? Here’s a short demo of Milgram’s Mixed Reality Continuum - all running in a browser. awe.media
  3. 3. A brief/biased history of Computer Vision 1957 - Russel A. Kirsch scans first photo with a computer 1960 - Larry Roberts publishes thesis at MIT 1964 - First facial recognition system (unamed intelligence agency) 1976 - UK Police create first License Plate recognition system 1978 - David Marr proposes edge detection framework at MIT 1985 - Lockheed Martin/Carnegie Mellon create first self-driving land vehicle 1992 - Tom Caudell at Boeing coins the term Augmented Reality 1999 - Billinghurst & Kato publish/demo ARToolkit at IWAR/SIGGRAPH 2000 - Windows only alpha version of OpenCV launched at CVPR 2007 - OpenCV 1.0 released 2008 - ARToolkit ported to Flash by @saqoosha 2011 - ARToolkit ported to Javascript by Ilmari Heikkinen 2011 - FastCV/Vuforia 1.0 released 2017 - Facebook adds Computer Vision to their camera app 2017 - OpenCV in the browser demonstrated here awe.media
  4. 4. How does Computer Vision
 work in the browser? awe.media camera -> gUM -> video -> canvas -> pixels -> vision algorithms
  5. 5. HTMLVideoElement This is a container for decoding and presenting video streams. This brought plugin free video to the web. awe.media
  6. 6. awe.media Canvas, WebGL & the ArrayBuffer The 2D Canvas gave us the ability to convert a video stream into pixel data. WebGL brought 3D Canvases with access to the GPU. But most importantly WebGL gave us ArrayBuffers
 which allowed us to access the pixel data for the first time.
  7. 7. awe.media JSARToolkit In 2011 Billinghurst & Kato's ARToolkit was ported to Javascript.
  8. 8. awe.media Enter WebRTC's getUserMedia() Some claim this has a latency that makes the web unusable for AR.
 But here’s the numbers running on a Pixel - the max difference is ~200ms 200-250ms - Camera stream in a native AR 350-400ms - gUM stream in a web app
  9. 9. awe.media WebRTC's getUserMedia() FAST feature detection & Tigerstail in 2012
  10. 10. awe.media WebRTC's getUserMedia() Tracking.js released in 2012
  11. 11. awe.media WebRTC's getUserMedia() AR.js released in 2017
  12. 12. awe.media Transpiling OpenCV This brings a more general computer vision toolkit to the web!
  13. 13. Demo Time! awe.media
  14. 14. awe.media But there's no gUM on iOS? For Vision based functionality we fallback to Visual Search For Location based apps we fallback to 360°/VR (like Pokemon Go with the camera off) And remember “video see thu” is not the only form of AR

×