This document discusses how computer vision will change augmented reality and the world. It summarizes how science fiction concepts like space travel, weather control and television were envisioned before becoming reality. Augmented reality aims to abolish the interface between digital and real worlds by turning everything seen into an augmentable medium. The document outlines how sensors define reality and how computer vision is crucial for augmented reality. It provides examples like a Yelp app that overlays ratings and names on storefronts and discusses how sensor data can be imperfect. It explores how computer vision can replace hardware by recognizing text, tracking motion, reconstructing 3D scenes and more. The conclusion is that cameras will define future augmented reality and turn more science fiction concepts into science fact

2. Sci-Fi to Sci-Fact
How Computer Vision Will Change AR and the World
Ben Newhouse. April 21, 2010.

3. Sci-Fi as a Lens
Space Flight - Jules Verne
Weather Control - John Jacob Astor IV
TV - H.G. Wells
Jetpacks

4. New Vision: AR

5. Augmented Reality
To abolish the interface between the digital
and the real
Turn everything you see into the medium

6. Today’s Topic
Sensors define the reality we wish to augment
Computer vision is crucial to this reality

7. Agenda
• Yelp Monocle
• Ideas that matter in Computer Vision
• A Concrete Example
• Conclude

8. Yelp Monocle
• Built by yours truly
• Point your phone down
the street, see ratings
and names overlaid on
store fronts
• Reality defined by
compass, accelerometer
and GPS.

9. Sensor Data is Dirty
GPS coordinates are
accurate to ~+/-50m (i.e.
an entire city block)
This is where I actually am

10. Yesterday Tomorrow

11. Keep In Mind
What piece(s) of hardware
is each technology replacing?

12. Optical Character
Recognition
iPhone Handwritten Digit OCR circa Fall 2009

13. Chinese OCR circa ?

14. Motion Tracking
Head-Controlled Mouse Circa Winter 2007

15. Virtual DJ Deck circa ?

16. 3D Reconstruction
UNC Realtime Urban Reconstruction circa Fall 2007

17. 3D Store Inventory Scanning circa ?

18. Even Simpler Things
Need Computer Vision
Square
Valued at $40 million before launch
Provides a audio jack dongle to read cards
Struggling to meet hardware demand

19. Why bother with hardware, when 45
lines of code and a camera will do
the same?
Credit Card Recognition circa Last Night at 2 a.m.

20. And Many More
Logo and Storefront Recognition
Barcodes/QR Codes
Medical Image Diagnostics
(Cool new Yelp things)

21. The Revolution Is
Happening Now
Real-time camera access in iPhone 4.0
(public release in June)
HD Video decoding chips can be co-opted for
computer vision

22. In Conclusion
Cameras will define our future augmented reality
In doing so they will further the software crusade
The restraining factor is not technology, it’s is the
ambition to execute.
Let’s make Sci-Fi become Sci-Fact
Thanks for listening! (follow @newhouseb for more)