Interfaces of the future now available augmented reality - google glass - 3 d sensors

Interfaces of the Future,
Available Now
Augmented Reality Displays, Google Glass, 3D Sensors bringing
computers into the 3D World.
March 10th, 2014
Where’s My Flying Car?
So many future promises from movies, so little delivery!
Minority Report Hand Gesture Interface, Now Available
Leap Sensor ($70) Demonstration
Finger and palm tracking, object translation, scaling and rotation.
Talk Outline
● Consumer Technology Enabled Experiences
○ Augmented Reality
○ Wearable & Pervasive Computing
○ Gestural Control
● Technology
○ 3D Sensing
○ Displays
○ Wearable
● New device survey
● Societal Implications
Terminator vision
Sentrinsic Background
● Formed in 2006 on patented linear sensing tech from
GaTech lab
● Developed embedded control
system to improve efficiency of
Air-Operated Diaphragm Pumps
Sentrinsic Background● Mobile Integration
○ Bluetooth Pump Monitoring
○ Customer iPhone app
■ Scan pumps for documentation, order
parts, track maintenance
■ Product: EquipCodes.com
● Augmented Reality
○ Maintenance
○ Operation Theory
○ Training
● Google Glass Manufacturing
Assembly Training app EquipCodes app
Augmented Reality
● What is it?
● Examples
○ 1st & 10 yellow line
○ Simple text
■ translations
■ nearby attractions
■ instructions
Augmented Reality Demos
● Pump Maintenance
http://www.youtube.com/watch?v=NYpl0PD5Le4
http://www.youtube.com/watch?v=nHfY56lHZjU
● Circuit Building Guide
http://www.youtube.com/watch?v=AfVQ4N-u0sk
Vein Visualization
Evana Eyes-On™ Glasses
multi-spectral imaging
Enabling Technology
● Improved cameras, processors, memory
● Pose estimation
○ Fiducial tracking
○ Texture feature-based tracking
○ SLAM - Simultaneous Location and Mapping
○ 3D Sensors
Fiducial Tracking
http://www.hitl.washington.
edu/artoolkit/Papers/ART02-Tutorial.pdf
ARToolKit Algorithm
Goal: find pose of
camera (Rotation and
Translation matrix)
Texture Feature Matching
● Find interesting points (corners)
○ independent of orientation, scale, lighting
● Use surrounding area to develop fingerprint of feature
(128 pt vector, robust to skew).
● Based on neuron
receptors in visual
cortex, sensitive
to spatial gradients
Distinctive Image Features from Scale-
Invariant Keypoints, Lowe 2004
Also used for Image Stitching - Using Feature Recognition
Texture Tracking
Tracking Texture Features tracked using Vuforia SDK
Texture Feature Matching
● Compare feature vector against database (Euclidian
distance).
● Can’t be regular pattern (checker board)
● High contrast required
● Glare problematic
● Tracker must be large
relative to graphics to
prevent jitter, misalignment.
Extensible Tracking
● Environment unknown beforehand
● SLAM (Simultaneous Localization and Mapping) or PTAM (Parallel
Tracking and Mapping)
○ Structure from Motion - Autodesk’s 123D
○ Scene should be static
○ For overlays, requires initial registration, not for games
○ Videos: http://www.youtube.com/watch?v=i7ierVkXYa8 (3D structure from 2D pictures)
○ http://www.youtube.com/watch?v=Y9HMn6bd-v8#t=50 (real-time map generation on iPhone)
● For overlays that must match exactly, still need some initial alignment.
○ Edge Based tracking using 3D Models
■ Uses extensible tracking, initialized with best
fit edge of model. (Metaio SDK, Sept, 2013)
○ Small tracker can still be used
Cameraless Tracking
GPS, compass, gryo,
accelerometer
Illuminated Tracking
CastAR’s head tracking
uses IR LEDs
Nokia City Lens
Depth Sensors
● Skips over many challenges with regular
camera tracking
○ Directly measures object depth
○ Can handle “featureless” textures (everything same color)
○ Still requires object recognition
○ Primarily used for body / hand tracking
● Examples
○ Kinect, Leap Motion, Laser range finder
How Kinect (ver 1) works
● Structured Lighting (Infrared)
video: http://www.youtube.com/watch?v=uq9SEJxZiUg
Kinect (ver 1) Drawbacks
● Affected by Sunlight
● Multiple kinects would conflict
● Minimum distance several feet - bad for
robotics and hand tracking (low resolution)
● Very cheap, though!
Time of Flight Sensor
● Uses speed of light and arrival time of reflected pulses
to calculate distance
● Extremely small times:
○ D = t * c / 2, 1cm → 66ps
○ 3 GHz cpu → 333ps
● Advantages: more compact (no separation between
sensor and emitter), simplier calculation, higher
resolution
● Kinect v2, Soft Kinetic sensor.
3D Sensing Applications
● Augmented Reality
● Body tracking
○ general computer gestural control, more natural
○ gaming - Dance games
○ head tracking - CastAR, Virtual Reality
○ 3D design / printing
Google Project Tango
● Adding depth sensors to phones
● Incorporating mapping and tracking
technology
● Applications
○ visually impaired navigation
○ instant measurement of
home dimensions
○ store & shelf navigation
video: http://www.youtube.com/watch?v=Qe10ExwzCqk
Wearable Phones?
Many of the applications make
more sense if you don’t have to
hold a phone!
● Maintenance - hard to hold
iPad with wrench!
● Mapping
Wearable / Pervasive Computing
● Google Glass
○ Bad for AR?
■ Battery life
■ Camera can’t be on constantly
■ Small field of View, not in direct
line of sight
○ Can’t focus on glass and outside
world - looking on wiki mid conversation
Interfaces of the future now available   augmented reality - google glass - 3 d sensors
Google Glass Field of View - Vignette
Greg Roberts GlassFAQ
Google Glass Pros
● We check our phones 110 times / day
● Better for contextual information
○ Incoming emails and texts
○ nearby places of interest
○ Instructions - following and creating (mfr DEMO)
● Voice recognition is great
● Capturing spontaneous or 1st person videos
● Looks cool, lightweight, viable wearable
● No tether
Head Mounted Display History
Steve Mann
Thad Starner - GaTech!
from 1993
Wearable Tech not new! Just
more compact and marketed!
Battery & CPU finally getting
small and cheap enough.
More head mounted options
● GlassUP
○ $300
○ small screen
○ tethered
● Recon Jet
designed for Sporting
$600, shipping May?
● Meta Pro
○ Does what people think Google Glass does
○ built-in depth sensor, potential
Iron Man / Minority report int.
○ 40 degree Field of View
(vs 14 degree on Glass)
○ $3650, shipping Sept
○ tethered controller
CastAR & Spacial AR
● Projection on reflective mat
● Head tracking via LEDs
● Hologram effect
● form of Spatial AR
interaction
w/ projections
Oculus Rift / Crystal Cove
● Primarily for gaming, Virtual Reality
● Fully immersive 110 deg Field of View
● Head tracking via accel, gyro sensors,
markers on ver2
● Some have added cameras to achieve
Augmented Reality effect
○ Solves delay problems with see-thru displays
● Motion sickness problem - display must
react quickly to head movement.
● Requires fast frame rates to trick brain
Video: Oculus Rift with 2 Cameras
achieves Augmented Reality
http://www.youtube.com/watch?
v=Bc_TCLoH2CA
Technology Hurdles
● Latency: see-through AR must be faster.
○ Google glass small screen prevents motion sickness,
but lag will be more jarring in large screens
● Dimming screen
○ per-pixel dimming not viable yet, see-through text can
get washed out. (Video pass-thru helps)
● Better, Smaller, Faster (Battery, CPU, screen)
● Object & Gestural recognition algorithms
Societal Implications
● Always on camera
● How do we have a conversation with computer
distractions? Wearable manners?
● Engineering - Design in 3D with hands, more natural
gestural control
● Job Creation?
○ Reverse of robotic automation - use adept human dexterity
○ Enhance human capabilities via internet, AI or remote assistance.
○ Remote diagnostics, field service, DIY repair
Conclusion & Final Thoughts
● Consumer Computers sense and project in
3D space
● Technology more pervasive / always on via
wearable implementations
● Content is major bottleneck
● If useful, appearance won’t matter
Contact
● Scott Driscoll
○ Scott.Driscoll@sentrinsic.com
○ plus.google.com/+ScottDriscoll
○ blog: ImponderableThings.com
○ youtube: CuriousInventor
● Company Website: EquipCodes.com
● Software Tools
○ Vuforia, Unity, Metaio, Google App Engine
1 of 39

Recommended

Virtual Reality: Learn to Maximize Present and Future Creative Possibilities! by
Virtual Reality: Learn to Maximize Present and Future Creative Possibilities!Virtual Reality: Learn to Maximize Present and Future Creative Possibilities!
Virtual Reality: Learn to Maximize Present and Future Creative Possibilities!Stephan Tanguay
1.5K views53 slides
Developing for Google Glass by
Developing for Google GlassDeveloping for Google Glass
Developing for Google GlassMark Billinghurst
13.3K views61 slides
Google glass, All you need to know by
Google glass, All you need to knowGoogle glass, All you need to know
Google glass, All you need to knowVivek Jha
14.2K views18 slides
Google glass by
Google glassGoogle glass
Google glassPrasad Kale
986 views18 slides
GOOGLE,ANDROID,GOOGLE GLASS by
GOOGLE,ANDROID,GOOGLE GLASSGOOGLE,ANDROID,GOOGLE GLASS
GOOGLE,ANDROID,GOOGLE GLASSVetri Vel
5.3K views28 slides
Google Glass by
Google GlassGoogle Glass
Google GlassAMITH A
312 views22 slides

More Related Content

What's hot

Google glass by
Google glassGoogle glass
Google glassDhaval Pujara
1.2K views20 slides
Google glass. by
Google glass.Google glass.
Google glass.ABhay Panchal
3.4K views21 slides
GOOGLE GLASS by
GOOGLE GLASSGOOGLE GLASS
GOOGLE GLASSAbhay Nigam
9.8K views15 slides
Google Glasses PPT by
Google Glasses PPTGoogle Glasses PPT
Google Glasses PPTSachin Gumber
5.5K views15 slides
Google Glass Seminar PPT by
Google Glass Seminar PPTGoogle Glass Seminar PPT
Google Glass Seminar PPTShashank Naik
5.5K views21 slides
google glass by
google glassgoogle glass
google glassVipin Sudhakar
16.3K views29 slides

Viewers also liked

Accelerometers to Augmented Reality by
Accelerometers to Augmented RealityAccelerometers to Augmented Reality
Accelerometers to Augmented Realityjblocksom
3.2K views35 slides
e-health & quality of care - business research and practice - medicinfo & twe... by
e-health & quality of care - business research and practice - medicinfo & twe...e-health & quality of care - business research and practice - medicinfo & twe...
e-health & quality of care - business research and practice - medicinfo & twe...Bart Brandenburg
977 views32 slides
426 Lecture 8: Mobile Augmented Reality by
426 Lecture 8: Mobile Augmented Reality426 Lecture 8: Mobile Augmented Reality
426 Lecture 8: Mobile Augmented RealityMark Billinghurst
2.9K views141 slides
Augmented reality by
Augmented reality Augmented reality
Augmented reality Satyendra Gupta
2.7K views15 slides
Augmented reality vs virtual reality by
Augmented reality vs virtual realityAugmented reality vs virtual reality
Augmented reality vs virtual realityheretohelpyou
4K views7 slides
Virtual Reality vs Augmented Reality - Knowing the Difference by
Virtual Reality vs Augmented Reality - Knowing the DifferenceVirtual Reality vs Augmented Reality - Knowing the Difference
Virtual Reality vs Augmented Reality - Knowing the DifferenceAugment
17.7K views7 slides

Viewers also liked(13)

Accelerometers to Augmented Reality by jblocksom
Accelerometers to Augmented RealityAccelerometers to Augmented Reality
Accelerometers to Augmented Reality
jblocksom3.2K views
e-health & quality of care - business research and practice - medicinfo & twe... by Bart Brandenburg
e-health & quality of care - business research and practice - medicinfo & twe...e-health & quality of care - business research and practice - medicinfo & twe...
e-health & quality of care - business research and practice - medicinfo & twe...
Bart Brandenburg977 views
426 Lecture 8: Mobile Augmented Reality by Mark Billinghurst
426 Lecture 8: Mobile Augmented Reality426 Lecture 8: Mobile Augmented Reality
426 Lecture 8: Mobile Augmented Reality
Mark Billinghurst2.9K views
Augmented reality vs virtual reality by heretohelpyou
Augmented reality vs virtual realityAugmented reality vs virtual reality
Augmented reality vs virtual reality
heretohelpyou4K views
Virtual Reality vs Augmented Reality - Knowing the Difference by Augment
Virtual Reality vs Augmented Reality - Knowing the DifferenceVirtual Reality vs Augmented Reality - Knowing the Difference
Virtual Reality vs Augmented Reality - Knowing the Difference
Augment17.7K views
Electronics past,present and future by Rajat Dhiman
Electronics past,present and futureElectronics past,present and future
Electronics past,present and future
Rajat Dhiman33.6K views
Effects of Technological Device to Students by Kollins Lolong
Effects of Technological Device to StudentsEffects of Technological Device to Students
Effects of Technological Device to Students
Kollins Lolong267.1K views
SEO Mobile-First: Re-enfocando tu proceso SEO a un mundo móvil #TheInbounder by Aleyda Solís
SEO Mobile-First: Re-enfocando tu proceso SEO a un mundo móvil #TheInbounder SEO Mobile-First: Re-enfocando tu proceso SEO a un mundo móvil #TheInbounder
SEO Mobile-First: Re-enfocando tu proceso SEO a un mundo móvil #TheInbounder
Aleyda Solís3.7K views
Impact of technology on education by Max Inder
Impact of technology on educationImpact of technology on education
Impact of technology on education
Max Inder274.7K views
IT in Healthcare by NetApp
IT in HealthcareIT in Healthcare
IT in Healthcare
NetApp69.5K views
[Infographic] How will Internet of Things (IoT) change the world as we know it? by InterQuest Group
[Infographic] How will Internet of Things (IoT) change the world as we know it?[Infographic] How will Internet of Things (IoT) change the world as we know it?
[Infographic] How will Internet of Things (IoT) change the world as we know it?
InterQuest Group193.2K views

Similar to Interfaces of the future now available augmented reality - google glass - 3 d sensors

Seminar on Google Glass by
Seminar on Google GlassSeminar on Google Glass
Seminar on Google GlassRamdev Darak
103 views28 slides
Google glass by
Google glassGoogle glass
Google glassRahulShah345
50 views16 slides
From Cardboard to Daydream - The Evolution of VR on Android by
From Cardboard to Daydream - The Evolution of VR on AndroidFrom Cardboard to Daydream - The Evolution of VR on Android
From Cardboard to Daydream - The Evolution of VR on AndroidOscar Salguero
202 views42 slides
Introduction google glass en - rev 20 - codemotion by
Introduction google glass   en - rev 20 - codemotionIntroduction google glass   en - rev 20 - codemotion
Introduction google glass en - rev 20 - codemotionCodemotion
12 views67 slides
Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni... by
Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni...Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni...
Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni...AugmentedWorldExpo
951 views26 slides

Similar to Interfaces of the future now available augmented reality - google glass - 3 d sensors(20)

Seminar on Google Glass by Ramdev Darak
Seminar on Google GlassSeminar on Google Glass
Seminar on Google Glass
Ramdev Darak103 views
From Cardboard to Daydream - The Evolution of VR on Android by Oscar Salguero
From Cardboard to Daydream - The Evolution of VR on AndroidFrom Cardboard to Daydream - The Evolution of VR on Android
From Cardboard to Daydream - The Evolution of VR on Android
Oscar Salguero202 views
Introduction google glass en - rev 20 - codemotion by Codemotion
Introduction google glass   en - rev 20 - codemotionIntroduction google glass   en - rev 20 - codemotion
Introduction google glass en - rev 20 - codemotion
Codemotion12 views
Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni... by AugmentedWorldExpo
Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni...Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni...
Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni...
AugmentedWorldExpo951 views
AN Introduction to Augmented Reality(AR) by Jai Sipani
AN Introduction to Augmented Reality(AR)AN Introduction to Augmented Reality(AR)
AN Introduction to Augmented Reality(AR)
Jai Sipani75 views
IRJET-Advance Technology- Google Glass by IRJET Journal
IRJET-Advance Technology- Google GlassIRJET-Advance Technology- Google Glass
IRJET-Advance Technology- Google Glass
IRJET Journal182 views
2020 vision - the journey from research lab to real-world product by KTN
2020 vision - the journey from research lab to real-world product2020 vision - the journey from research lab to real-world product
2020 vision - the journey from research lab to real-world product
KTN129 views
3d Video Conferencing: Opportunities and Pitfalls by TrueConf
3d Video Conferencing: Opportunities and Pitfalls3d Video Conferencing: Opportunities and Pitfalls
3d Video Conferencing: Opportunities and Pitfalls
TrueConf1K views
3d Video Conferencing: Opportunities and Pitfalls by TrueConf__
3d Video Conferencing: Opportunities and Pitfalls3d Video Conferencing: Opportunities and Pitfalls
3d Video Conferencing: Opportunities and Pitfalls
TrueConf__142 views
IEEE - Consumer Electronics Trends Opportunities (2015) by Prabindh Sundareson
IEEE - Consumer Electronics Trends Opportunities (2015)IEEE - Consumer Electronics Trends Opportunities (2015)
IEEE - Consumer Electronics Trends Opportunities (2015)

Recently uploaded

STKI Israeli Market Study 2023 corrected forecast 2023_24 v3.pdf by
STKI Israeli Market Study 2023   corrected forecast 2023_24 v3.pdfSTKI Israeli Market Study 2023   corrected forecast 2023_24 v3.pdf
STKI Israeli Market Study 2023 corrected forecast 2023_24 v3.pdfDr. Jimmy Schwarzkopf
24 views29 slides
"Running students' code in isolation. The hard way", Yurii Holiuk by
"Running students' code in isolation. The hard way", Yurii Holiuk "Running students' code in isolation. The hard way", Yurii Holiuk
"Running students' code in isolation. The hard way", Yurii Holiuk Fwdays
24 views34 slides
Design Driven Network Assurance by
Design Driven Network AssuranceDesign Driven Network Assurance
Design Driven Network AssuranceNetwork Automation Forum
19 views42 slides
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ... by
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...Jasper Oosterveld
27 views49 slides
Future of Indian ConsumerTech by
Future of Indian ConsumerTechFuture of Indian ConsumerTech
Future of Indian ConsumerTechKapil Khandelwal (KK)
24 views68 slides
Unit 1_Lecture 2_Physical Design of IoT.pdf by
Unit 1_Lecture 2_Physical Design of IoT.pdfUnit 1_Lecture 2_Physical Design of IoT.pdf
Unit 1_Lecture 2_Physical Design of IoT.pdfStephenTec
15 views36 slides

Recently uploaded(20)

STKI Israeli Market Study 2023 corrected forecast 2023_24 v3.pdf by Dr. Jimmy Schwarzkopf
STKI Israeli Market Study 2023   corrected forecast 2023_24 v3.pdfSTKI Israeli Market Study 2023   corrected forecast 2023_24 v3.pdf
STKI Israeli Market Study 2023 corrected forecast 2023_24 v3.pdf
"Running students' code in isolation. The hard way", Yurii Holiuk by Fwdays
"Running students' code in isolation. The hard way", Yurii Holiuk "Running students' code in isolation. The hard way", Yurii Holiuk
"Running students' code in isolation. The hard way", Yurii Holiuk
Fwdays24 views
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ... by Jasper Oosterveld
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
Unit 1_Lecture 2_Physical Design of IoT.pdf by StephenTec
Unit 1_Lecture 2_Physical Design of IoT.pdfUnit 1_Lecture 2_Physical Design of IoT.pdf
Unit 1_Lecture 2_Physical Design of IoT.pdf
StephenTec15 views
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院 by IttrainingIttraining
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f... by TrustArc
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...
TrustArc72 views
Piloting & Scaling Successfully With Microsoft Viva by Richard Harbridge
Piloting & Scaling Successfully With Microsoft VivaPiloting & Scaling Successfully With Microsoft Viva
Piloting & Scaling Successfully With Microsoft Viva
"Surviving highload with Node.js", Andrii Shumada by Fwdays
"Surviving highload with Node.js", Andrii Shumada "Surviving highload with Node.js", Andrii Shumada
"Surviving highload with Node.js", Andrii Shumada
Fwdays33 views

Interfaces of the future now available augmented reality - google glass - 3 d sensors

  • 1. Interfaces of the Future, Available Now Augmented Reality Displays, Google Glass, 3D Sensors bringing computers into the 3D World. March 10th, 2014
  • 3. So many future promises from movies, so little delivery!
  • 4. Minority Report Hand Gesture Interface, Now Available
  • 5. Leap Sensor ($70) Demonstration Finger and palm tracking, object translation, scaling and rotation.
  • 6. Talk Outline ● Consumer Technology Enabled Experiences ○ Augmented Reality ○ Wearable & Pervasive Computing ○ Gestural Control ● Technology ○ 3D Sensing ○ Displays ○ Wearable ● New device survey ● Societal Implications Terminator vision
  • 7. Sentrinsic Background ● Formed in 2006 on patented linear sensing tech from GaTech lab ● Developed embedded control system to improve efficiency of Air-Operated Diaphragm Pumps
  • 8. Sentrinsic Background● Mobile Integration ○ Bluetooth Pump Monitoring ○ Customer iPhone app ■ Scan pumps for documentation, order parts, track maintenance ■ Product: EquipCodes.com ● Augmented Reality ○ Maintenance ○ Operation Theory ○ Training ● Google Glass Manufacturing Assembly Training app EquipCodes app
  • 9. Augmented Reality ● What is it? ● Examples ○ 1st & 10 yellow line ○ Simple text ■ translations ■ nearby attractions ■ instructions
  • 10. Augmented Reality Demos ● Pump Maintenance http://www.youtube.com/watch?v=NYpl0PD5Le4 http://www.youtube.com/watch?v=nHfY56lHZjU ● Circuit Building Guide http://www.youtube.com/watch?v=AfVQ4N-u0sk
  • 11. Vein Visualization Evana Eyes-On™ Glasses multi-spectral imaging
  • 12. Enabling Technology ● Improved cameras, processors, memory ● Pose estimation ○ Fiducial tracking ○ Texture feature-based tracking ○ SLAM - Simultaneous Location and Mapping ○ 3D Sensors
  • 14. Texture Feature Matching ● Find interesting points (corners) ○ independent of orientation, scale, lighting ● Use surrounding area to develop fingerprint of feature (128 pt vector, robust to skew). ● Based on neuron receptors in visual cortex, sensitive to spatial gradients Distinctive Image Features from Scale- Invariant Keypoints, Lowe 2004
  • 15. Also used for Image Stitching - Using Feature Recognition
  • 16. Texture Tracking Tracking Texture Features tracked using Vuforia SDK
  • 17. Texture Feature Matching ● Compare feature vector against database (Euclidian distance). ● Can’t be regular pattern (checker board) ● High contrast required ● Glare problematic ● Tracker must be large relative to graphics to prevent jitter, misalignment.
  • 18. Extensible Tracking ● Environment unknown beforehand ● SLAM (Simultaneous Localization and Mapping) or PTAM (Parallel Tracking and Mapping) ○ Structure from Motion - Autodesk’s 123D ○ Scene should be static ○ For overlays, requires initial registration, not for games ○ Videos: http://www.youtube.com/watch?v=i7ierVkXYa8 (3D structure from 2D pictures) ○ http://www.youtube.com/watch?v=Y9HMn6bd-v8#t=50 (real-time map generation on iPhone) ● For overlays that must match exactly, still need some initial alignment. ○ Edge Based tracking using 3D Models ■ Uses extensible tracking, initialized with best fit edge of model. (Metaio SDK, Sept, 2013) ○ Small tracker can still be used
  • 19. Cameraless Tracking GPS, compass, gryo, accelerometer Illuminated Tracking CastAR’s head tracking uses IR LEDs Nokia City Lens
  • 20. Depth Sensors ● Skips over many challenges with regular camera tracking ○ Directly measures object depth ○ Can handle “featureless” textures (everything same color) ○ Still requires object recognition ○ Primarily used for body / hand tracking ● Examples ○ Kinect, Leap Motion, Laser range finder
  • 21. How Kinect (ver 1) works ● Structured Lighting (Infrared) video: http://www.youtube.com/watch?v=uq9SEJxZiUg
  • 22. Kinect (ver 1) Drawbacks ● Affected by Sunlight ● Multiple kinects would conflict ● Minimum distance several feet - bad for robotics and hand tracking (low resolution) ● Very cheap, though!
  • 23. Time of Flight Sensor ● Uses speed of light and arrival time of reflected pulses to calculate distance ● Extremely small times: ○ D = t * c / 2, 1cm → 66ps ○ 3 GHz cpu → 333ps ● Advantages: more compact (no separation between sensor and emitter), simplier calculation, higher resolution ● Kinect v2, Soft Kinetic sensor.
  • 24. 3D Sensing Applications ● Augmented Reality ● Body tracking ○ general computer gestural control, more natural ○ gaming - Dance games ○ head tracking - CastAR, Virtual Reality ○ 3D design / printing
  • 25. Google Project Tango ● Adding depth sensors to phones ● Incorporating mapping and tracking technology ● Applications ○ visually impaired navigation ○ instant measurement of home dimensions ○ store & shelf navigation video: http://www.youtube.com/watch?v=Qe10ExwzCqk
  • 26. Wearable Phones? Many of the applications make more sense if you don’t have to hold a phone! ● Maintenance - hard to hold iPad with wrench! ● Mapping
  • 27. Wearable / Pervasive Computing ● Google Glass ○ Bad for AR? ■ Battery life ■ Camera can’t be on constantly ■ Small field of View, not in direct line of sight ○ Can’t focus on glass and outside world - looking on wiki mid conversation
  • 29. Google Glass Field of View - Vignette Greg Roberts GlassFAQ
  • 30. Google Glass Pros ● We check our phones 110 times / day ● Better for contextual information ○ Incoming emails and texts ○ nearby places of interest ○ Instructions - following and creating (mfr DEMO) ● Voice recognition is great ● Capturing spontaneous or 1st person videos ● Looks cool, lightweight, viable wearable ● No tether
  • 31. Head Mounted Display History Steve Mann Thad Starner - GaTech! from 1993 Wearable Tech not new! Just more compact and marketed! Battery & CPU finally getting small and cheap enough.
  • 32. More head mounted options ● GlassUP ○ $300 ○ small screen ○ tethered
  • 33. ● Recon Jet designed for Sporting $600, shipping May? ● Meta Pro ○ Does what people think Google Glass does ○ built-in depth sensor, potential Iron Man / Minority report int. ○ 40 degree Field of View (vs 14 degree on Glass) ○ $3650, shipping Sept ○ tethered controller
  • 34. CastAR & Spacial AR ● Projection on reflective mat ● Head tracking via LEDs ● Hologram effect ● form of Spatial AR interaction w/ projections
  • 35. Oculus Rift / Crystal Cove ● Primarily for gaming, Virtual Reality ● Fully immersive 110 deg Field of View ● Head tracking via accel, gyro sensors, markers on ver2 ● Some have added cameras to achieve Augmented Reality effect ○ Solves delay problems with see-thru displays ● Motion sickness problem - display must react quickly to head movement. ● Requires fast frame rates to trick brain Video: Oculus Rift with 2 Cameras achieves Augmented Reality http://www.youtube.com/watch? v=Bc_TCLoH2CA
  • 36. Technology Hurdles ● Latency: see-through AR must be faster. ○ Google glass small screen prevents motion sickness, but lag will be more jarring in large screens ● Dimming screen ○ per-pixel dimming not viable yet, see-through text can get washed out. (Video pass-thru helps) ● Better, Smaller, Faster (Battery, CPU, screen) ● Object & Gestural recognition algorithms
  • 37. Societal Implications ● Always on camera ● How do we have a conversation with computer distractions? Wearable manners? ● Engineering - Design in 3D with hands, more natural gestural control ● Job Creation? ○ Reverse of robotic automation - use adept human dexterity ○ Enhance human capabilities via internet, AI or remote assistance. ○ Remote diagnostics, field service, DIY repair
  • 38. Conclusion & Final Thoughts ● Consumer Computers sense and project in 3D space ● Technology more pervasive / always on via wearable implementations ● Content is major bottleneck ● If useful, appearance won’t matter
  • 39. Contact ● Scott Driscoll ○ Scott.Driscoll@sentrinsic.com ○ plus.google.com/+ScottDriscoll ○ blog: ImponderableThings.com ○ youtube: CuriousInventor ● Company Website: EquipCodes.com ● Software Tools ○ Vuforia, Unity, Metaio, Google App Engine