Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
1
Technological Trends in
Wearable Computing
and Augmented Reality
Steven Feiner
Computer Graphics & User Interfaces Lab
Department of Computer Science
Columbia University
New York, NY 10027
Startup Village, Skolkovo Innovation City, Moscow, June 1–3, 2014
2
Wearable Computing
Clock watch
http://en.wikipedia.org/wiki/File:Henlein_Taschenuhr.jpg
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
3
Wearable Computing
Wristwatch
www.watchtime.com/wristwatch-industry-news/watches/iwc-portuguese-grande-complication/
4
Wearable Computing
Halden Calculex Slide Rule
https://sites.google.com/site/bobscalculatorsandsliderules/home/slide-rules/halden
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
5
Wearable Computing
Cavalry Sketch Board
www.gemmary.com
6
Wearable Computing
Plus Four Wristlet Route Indicator
© Phillip Tosscano
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
7
Wearable Computing
Pocket Diary
www.walesonline.co.uk/news/wales-news/captain-scotts-diary-prepared-ill-fated-1812944
8
Edward Thorp & Claude Shannon
Wearable Computing
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
9
Wearable Computing
Steve Mann
10
Wearable Computing
MIT Cyborgs
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
11
AR Restaurant Guide, 2001
Where: Back
Columbia Touring Machine
monet.cs.columbia.edu/projects/mars
12
Where: Wrist
Pebble Smart Watch
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
13
Where: Eye
Google Glass
www.independent.co.uk/incoming/article8526907.ece/BINARY/original/IsabelleOlsson_GoogleGlass.jpg
14
Where: Clothes
B. Stern, L. Buechley
http://web.media.mit.edu/~leah/LilyPad/share.html
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
15
Trends
16
Augmented Reality
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
17
AR vs. VR google.com/trends
18
AR vs. VR google.com/trends
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
19
Virtual Reality Back for Real
Oculus Rift
20
Eyewear
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
21
Google Glass prototype, 2012
Eyewear
MicroOptical Corp. clip-on, late 20th C. IBM Visionpad, late 20th C.
22
Lumus
Konica Minolta
Sony
Warmer Still…
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
23
2014: See-Through Head-Worn
Displays++
Integrate ≥ 1 of
Orientation
tracker
Input device
RGB camera
Depth camera
Computer
Radio
GlassUp
Meta
Google
Epson
Laster Optinvent
Vuzix
24
Courtesy of a time traveler
The Time Machine
2XXX eyewear in 1999
Appearance
Comfort
FOV
Per-pixel opacity
Variable depth of field
Integrated sensors,…
Production plans
Computer? Networking? Content?
The milieu is missing!
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
25
Personal Area Network
Devices wirelessly connected to a
smartphone hub
E.g., Bluetooth LE
26
Recolonizing the Wrist
Smartwatch
Fitness tracker
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
27
Natural Feature Tracking
Qualcomm Vuforia
28
Body Input
Other input: Myo
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
29
Cameras: RGB and Depth
Softkinetic DS325
30
Cameras: RGB and Depth
Leap Motion Controller
www.eyalo.com/wp-content/uploads/2013/07/leaprev7.jpg
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
31
Ideas Refined by 3D Printing
MakerBot Thing-O-Matic
www.domusweb.it/en/news/2012/02/28/print-3d-an-exhibition.html
32
Application Domains
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
33
Helmets and Masks
Helmets http://www.fusar-tech.com/
http://www.skullysystems.com/
34
ARMAR: Workpiece Task
Assistance
S. Henderson & S. Feiner, ISMAR 2011
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
35
ARMAR: Workpiece Task
Assistance Study
Is AR better than conventional (computer)
documentation?
Assemble aircraft engine combustion chamber
Select bottom and top, align correctly, pin
+ =
36
ARMAR: Workpiece Task
Assistance Study: LCD Condition
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
37
ARMAR: Workpiece Task
Assistance Study: AR Condition
38
ARMAR: Workpiece Task
Assistance Study: AR Condition
Optical
See-Through
HWD
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
39
ARMAR: Workpiece Task
Assistance (Optical See-Through)
40
O. Oda, M. Sukan, S. Feiner, and B. Tversky, 3DUI 2013
Remote Localization and
Referencing in Collaborative AR
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
41
Collaborative Main
Remote Localization and
Referencing in Collaborative AR
O. Oda, M. Sukan, S. Feiner, and B. Tversky, 3DUI 2013
Remote SME View
Local Technician View
42
Remote Localization and
Referencing in Collaborative AR
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
43
Multiple Displays
44
Environment Management
AR as computational æther embedding other
tracked displays and the environment
How do we control/manage/coordinate what
information goes where?
Head space?
Body space?
Device space?
World space?
A. Butz et al., IWAR 1999
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
45
Multiple Displays: Hybrid UIs
Combining a shared tabletop display with
personal head-tracked see-through displays and
phones
N. Dedual, C. Elvezio, and S. Feiner
46
Multiple Displays: Hybrid UIs
Data about POIs from Yelp, Foursquare,
Instagram, CityMaps, NYC 311
Displayed in summary grids of pictograms
N. Dedual, C. Elvezio, and S. Feiner
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
47
Multiple Displays: Hybrid UIs
N. Dedual, C. Elvezio, and S. Feiner
48
Multiple Displays: Hybrid UIs
N. Dedual, C. Elvezio, and S. Feiner
Steven Feiner, Technological Trends in Wearable Computing and Augmented Reality
Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014
49
Acknowledgments
Nick Dedual
Carmine Elvezio
Steve Henderson
Ohan Oda
Mengu Sukan
Barbara Tversky
Lauren Wilcox
USMC personnel at Marine
Corps Logistics Base: Mike
Shellem, Curtis Williams,
Andrew Mitchell, Alan
Butterworth
USMC cadre and students at
Aberdeen Proving Ground
Supported in part by NSF,
ONR, Raytheon, USMC, and
gifts from, Google, Microsoft,
Nokia, VTT, Vuzix, Canon
Any opinions, findings and conclusions or
recommendations expressed in this material
are those of the author(s) and do not
necessarily reflect the views of the National
Science Foundation

Steven Feiner Startup Village WC & AR Technological trends June 2014

  • 1.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 1 Technological Trends in Wearable Computing and Augmented Reality Steven Feiner Computer Graphics & User Interfaces Lab Department of Computer Science Columbia University New York, NY 10027 Startup Village, Skolkovo Innovation City, Moscow, June 1–3, 2014 2 Wearable Computing Clock watch http://en.wikipedia.org/wiki/File:Henlein_Taschenuhr.jpg
  • 2.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 3 Wearable Computing Wristwatch www.watchtime.com/wristwatch-industry-news/watches/iwc-portuguese-grande-complication/ 4 Wearable Computing Halden Calculex Slide Rule https://sites.google.com/site/bobscalculatorsandsliderules/home/slide-rules/halden
  • 3.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 5 Wearable Computing Cavalry Sketch Board www.gemmary.com 6 Wearable Computing Plus Four Wristlet Route Indicator © Phillip Tosscano
  • 4.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 7 Wearable Computing Pocket Diary www.walesonline.co.uk/news/wales-news/captain-scotts-diary-prepared-ill-fated-1812944 8 Edward Thorp & Claude Shannon Wearable Computing
  • 5.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 9 Wearable Computing Steve Mann 10 Wearable Computing MIT Cyborgs
  • 6.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 11 AR Restaurant Guide, 2001 Where: Back Columbia Touring Machine monet.cs.columbia.edu/projects/mars 12 Where: Wrist Pebble Smart Watch
  • 7.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 13 Where: Eye Google Glass www.independent.co.uk/incoming/article8526907.ece/BINARY/original/IsabelleOlsson_GoogleGlass.jpg 14 Where: Clothes B. Stern, L. Buechley http://web.media.mit.edu/~leah/LilyPad/share.html
  • 8.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 15 Trends 16 Augmented Reality
  • 9.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 17 AR vs. VR google.com/trends 18 AR vs. VR google.com/trends
  • 10.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 19 Virtual Reality Back for Real Oculus Rift 20 Eyewear
  • 11.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 21 Google Glass prototype, 2012 Eyewear MicroOptical Corp. clip-on, late 20th C. IBM Visionpad, late 20th C. 22 Lumus Konica Minolta Sony Warmer Still…
  • 12.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 23 2014: See-Through Head-Worn Displays++ Integrate ≥ 1 of Orientation tracker Input device RGB camera Depth camera Computer Radio GlassUp Meta Google Epson Laster Optinvent Vuzix 24 Courtesy of a time traveler The Time Machine 2XXX eyewear in 1999 Appearance Comfort FOV Per-pixel opacity Variable depth of field Integrated sensors,… Production plans Computer? Networking? Content? The milieu is missing!
  • 13.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 25 Personal Area Network Devices wirelessly connected to a smartphone hub E.g., Bluetooth LE 26 Recolonizing the Wrist Smartwatch Fitness tracker
  • 14.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 27 Natural Feature Tracking Qualcomm Vuforia 28 Body Input Other input: Myo
  • 15.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 29 Cameras: RGB and Depth Softkinetic DS325 30 Cameras: RGB and Depth Leap Motion Controller www.eyalo.com/wp-content/uploads/2013/07/leaprev7.jpg
  • 16.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 31 Ideas Refined by 3D Printing MakerBot Thing-O-Matic www.domusweb.it/en/news/2012/02/28/print-3d-an-exhibition.html 32 Application Domains
  • 17.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 33 Helmets and Masks Helmets http://www.fusar-tech.com/ http://www.skullysystems.com/ 34 ARMAR: Workpiece Task Assistance S. Henderson & S. Feiner, ISMAR 2011
  • 18.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 35 ARMAR: Workpiece Task Assistance Study Is AR better than conventional (computer) documentation? Assemble aircraft engine combustion chamber Select bottom and top, align correctly, pin + = 36 ARMAR: Workpiece Task Assistance Study: LCD Condition
  • 19.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 37 ARMAR: Workpiece Task Assistance Study: AR Condition 38 ARMAR: Workpiece Task Assistance Study: AR Condition Optical See-Through HWD
  • 20.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 39 ARMAR: Workpiece Task Assistance (Optical See-Through) 40 O. Oda, M. Sukan, S. Feiner, and B. Tversky, 3DUI 2013 Remote Localization and Referencing in Collaborative AR
  • 21.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 41 Collaborative Main Remote Localization and Referencing in Collaborative AR O. Oda, M. Sukan, S. Feiner, and B. Tversky, 3DUI 2013 Remote SME View Local Technician View 42 Remote Localization and Referencing in Collaborative AR
  • 22.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 43 Multiple Displays 44 Environment Management AR as computational æther embedding other tracked displays and the environment How do we control/manage/coordinate what information goes where? Head space? Body space? Device space? World space? A. Butz et al., IWAR 1999
  • 23.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 45 Multiple Displays: Hybrid UIs Combining a shared tabletop display with personal head-tracked see-through displays and phones N. Dedual, C. Elvezio, and S. Feiner 46 Multiple Displays: Hybrid UIs Data about POIs from Yelp, Foursquare, Instagram, CityMaps, NYC 311 Displayed in summary grids of pictograms N. Dedual, C. Elvezio, and S. Feiner
  • 24.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 47 Multiple Displays: Hybrid UIs N. Dedual, C. Elvezio, and S. Feiner 48 Multiple Displays: Hybrid UIs N. Dedual, C. Elvezio, and S. Feiner
  • 25.
    Steven Feiner, TechnologicalTrends in Wearable Computing and Augmented Reality Startup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014 49 Acknowledgments Nick Dedual Carmine Elvezio Steve Henderson Ohan Oda Mengu Sukan Barbara Tversky Lauren Wilcox USMC personnel at Marine Corps Logistics Base: Mike Shellem, Curtis Williams, Andrew Mitchell, Alan Butterworth USMC cadre and students at Aberdeen Proving Ground Supported in part by NSF, ONR, Raytheon, USMC, and gifts from, Google, Microsoft, Nokia, VTT, Vuzix, Canon Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation