AR/VR in perspective
Tech , Effect and Value
Carl Henrikson & Ahmed Assal
Building the right
things
Building things
right
Running things
efficiently
Carl Henrikson
Senior Consultant
Business Advisory
Virtual reality
- an old dream which is
finally becoming reality
Stanford University Libraries
1966
AUGMENTED REALITY
The Ultimate Display – a.k.a. Damocles Sword
Head mounted augmented reality display
Ivan Sutherland 1968
80-ies and 90-ies...
But finally...
2014: R
irtual
enaissance
V
VRAR
Learning and training
Decision making
Communication
Sales
Marketing
Immersive design
Location based
Entertainment
and information
Emerging applications
Learning and training
Virtual manuals
Simulation
AR and Industrial
maintenance
Assistive AR/VR tools
Decision making
Real time data and
analytics visualization
What if scenarios
Remote systems control
and analysis
Communication
Augmented vision
Remote experts Remote collaboration
Telepresence
Remote Collaboration
Remote Expert
Telepresence
25
Sales
Showcases and demos
Configurators &
customizers
Trials
Marketing
Experential campaigns
Brand recognition Virtual characters
Store attraction
Immersive design
Conceptual design Design review
Location based
Virtual tours On location gaming
Entertainment
& information
Games
Movies
Concerts
Emerging applications
Crime scene
reconstruction
Therapy, Pain relief Virtual workspaces
Immersive journalism
MemoAR
Technology Effect Value
The TECHNOLOGY The EFFECT that can be / need to be achived The business/customer MOTIVE or DRIVER
Interactive VR
Consuming VR
VR 360
Cameras
Empathy
engine
360 experience
Virtual
presence
Real time
virtual
presence
Near real
experience /
simulation
Capture your
experience
Anyone can
create a VR
environment
Be able to
build/interact
with a world
around you
Cheap, Easy
access
Time & place
travel
Manipulate the
brain
Experience
something
which is not
built
Different for each customer
and use case
The business
value and
customer
motive
Begin with...
The effect
which is
needed
then...
The technology
needed to
achieve the
effect
and finally...
But, how? ...
Outside-in metodology to understand users
gain
gain
gain
gain
Charity organization working with homeless
people
Main objective: communicate what it’s like to be homeless so that people get
emphatic with their cause and donate money and time
”Empathy engine”
Outside-in metodology to understand users
To get the most from this technology we must...
Become imagineers!
* Stolen with pride from Walt Disney Inc.
*
KEEP CALM
AR + VR
IS CURRENTLY AT
1%
AR / VR / MR / 3D Realtime
Graphics
A Technical Perspective
VR / AR / MR /
Realtime 3D
Current - Future Future+
Software
Hardware
Content
Procedural
Content
Generation
Automatic and
Semi –automatic
Content
Generation
Automatic
Animation
Creation Automatic
Mesh Creation
Automated, Mixed-authoring and Procedural
Content Generation
44
Automatic Animation Generation
45
Natural Environements
47
Spectacular Hyper-realism
48
Mapbox & Unity 3D
49
Mapbox Styling
50
L-systems and Plants – SpeedTree
51
L-systems and Plants – Xfrog
52
Cities, Streets, Buildings & CityEngine
53
Cities, Streets, Buildings & CityEngine
54
Crowd Creation & Animation – Anima 3
55
Today’s journey
56
AR Works - Occlusion
57
“AR systems never “work” or “don’t work”. It’s always a question of do things
work well enough in a wide enough range of situations.”
58
Google search volumes (AR) over time (trends)
PokemonGo
Distribution: July 2016
22-sept-2017: initial release Iphone 8
19-sept-2017: Initial release of iOS 11 iOS
NEW API: ARKit
April 2012: Google unveiled Project Glass
“Similarly, in your business, be sure that before you focus on the
results or outcome, that you make it crystal clear what problems
you solve, or why people might need what you offer” - Ian Altman
at Forbes
Jan 2015
AR technologies compared
61
Microsoft Hololens Meta 2 ODG R-7
Magic Leap, Google Glass 2.0, Vuzix, ODG R-9, Daqri Smart glasses, ...
Daqri Moverio BT-300
+ universal use
+ poweful, open platform
+ standalone device
+ strong go-to-market push
- limited battery life
+ universal use
+ poweful platform
- connected device
+ universal use
+ lightweight
- limited FoV
- limited battery life
+ industry use tailored
+ powerful platform
- industry use only
- limited battery life
+ industry use tailored
+ powerful platform
- industry use only
- limited FoV
AR SDKs
Regular Handheld Devices
62
Vuforia
ARCore
ARKit
Dedicated HW
HMD Tango Tablet
Vuforia
ARCore
ARKit
Hololens
Tango
AR SDKs on Handheld and HMD Devices
63
Augmented Reality
SLAM
Sensor Fusion
Computer Vision
Pose Deblurring
Visual-inertial Odometry
Optimal Correction
Surface Estimation
Camera Calibration
Nonlinear OptimizationScene Understanding
Inertial Measurement Unit
Triangulation
AR SDKs on Handheld and HMD Devices
64
It enables developers to position and orient virtual objects, such as
3D models and other media, in relation to real world images when
these are viewed through the camera of a mobile or HMD device.
Motion Tracking
Visual Inertial Odometry (VIO) – to track your position in space (your 6dof pose) in real-time
Your pose is recalculated in-between every frame refresh on your display by matching a point
in the real world to a pixel on the camera sensor each frame
AR SDKs on Handheld and HMD Devices
65
It enables developers to position and orient virtual objects, such as
3D models and other media, in relation to real world images when
these are viewed through the camera of a mobile or HMD device.
3D Reconstruction (point cloud)
66
ARKit vs Tango vs Hololens
HW Design
SW Stack
"RGB-IR" camera for color image and infrared depth detection
Fisheye motion camera
Accelerometer, gyroscope & magnetometer
Ambient light sensors
Microsoft Holographic Processing Unit
VIO (motion tracking)
Sparse Mapping (area learning)
Dense 3D reconstruction (depth perception).
Regular phone camera
Accelerometer & gyroscope
Highly accurate Dead Reckoning Guess to
compensate lack of stereoscopic images
ARKit – Augmented Reality for iOS
67
• API that lets third-party developers to build Augmented Reality apps for iOS
• Available only for processors Apple A9 and later (6S, 6S Plus and later) due to performance
• Available only operatig system iOS 11 and later (officially released on 19-sept-2017)
Optimal performance from model iPhone 8
• Depth sensing camera system calibrated for AR
• Lo-light and 60 fps
• New gyro and accelerometer
• Accurate motion tracking
Microsoft Hololens
68
• Accelerometer, gyroscope, magnetometer
• Spatial Mapping
• Gesture recognition
• 3D display (FOV: 30°×17.5°)
• Video (2.4 megapixel)
• Sound
• Bluetooth
• Wi-Fi
VR everywhere at MWC!
VR was the big attraction! ..but the real use cases were few!
VR devices in all ranges!
March $599April $799
2nd gen $99
May $200
Oct $499
Multiple version $5
ConsumingInteracting
But Google DayDream and ARM next Gen processors will drive mobile VR
from consuming to interactive
71
THANKS FOR LISTENING!

AR VR Meetup in Göteborg 2017-12-06

  • 1.
    AR/VR in perspective Tech, Effect and Value Carl Henrikson & Ahmed Assal
  • 2.
    Building the right things Buildingthings right Running things efficiently Carl Henrikson Senior Consultant Business Advisory
  • 3.
    Virtual reality - anold dream which is finally becoming reality
  • 6.
  • 7.
  • 9.
  • 10.
    The Ultimate Display– a.k.a. Damocles Sword Head mounted augmented reality display Ivan Sutherland 1968
  • 12.
  • 13.
  • 14.
  • 15.
    Learning and training Decisionmaking Communication Sales Marketing Immersive design Location based Entertainment and information Emerging applications
  • 16.
    Learning and training Virtualmanuals Simulation AR and Industrial maintenance Assistive AR/VR tools
  • 17.
    Decision making Real timedata and analytics visualization What if scenarios Remote systems control and analysis
  • 18.
    Communication Augmented vision Remote expertsRemote collaboration Telepresence
  • 19.
  • 21.
  • 23.
  • 24.
  • 25.
  • 26.
    Marketing Experential campaigns Brand recognitionVirtual characters Store attraction
  • 27.
  • 28.
    Location based Virtual toursOn location gaming
  • 29.
  • 30.
    Emerging applications Crime scene reconstruction Therapy,Pain relief Virtual workspaces Immersive journalism
  • 31.
  • 32.
    Technology Effect Value TheTECHNOLOGY The EFFECT that can be / need to be achived The business/customer MOTIVE or DRIVER Interactive VR Consuming VR VR 360 Cameras Empathy engine 360 experience Virtual presence Real time virtual presence Near real experience / simulation Capture your experience Anyone can create a VR environment Be able to build/interact with a world around you Cheap, Easy access Time & place travel Manipulate the brain Experience something which is not built Different for each customer and use case
  • 33.
    The business value and customer motive Beginwith... The effect which is needed then... The technology needed to achieve the effect and finally... But, how? ...
  • 34.
    Outside-in metodology tounderstand users gain gain gain gain
  • 35.
    Charity organization workingwith homeless people Main objective: communicate what it’s like to be homeless so that people get emphatic with their cause and donate money and time
  • 36.
  • 38.
    Outside-in metodology tounderstand users
  • 39.
    To get themost from this technology we must... Become imagineers! * Stolen with pride from Walt Disney Inc. *
  • 40.
    KEEP CALM AR +VR IS CURRENTLY AT 1%
  • 41.
    AR / VR/ MR / 3D Realtime Graphics A Technical Perspective
  • 42.
    VR / AR/ MR / Realtime 3D Current - Future Future+ Software Hardware Content Procedural Content Generation Automatic and Semi –automatic Content Generation Automatic Animation Creation Automatic Mesh Creation
  • 43.
    Automated, Mixed-authoring andProcedural Content Generation 44
  • 44.
  • 45.
  • 46.
  • 47.
  • 48.
  • 49.
    L-systems and Plants– SpeedTree 51
  • 50.
  • 51.
  • 52.
  • 53.
    Crowd Creation &Animation – Anima 3 55
  • 54.
  • 55.
    AR Works -Occlusion 57 “AR systems never “work” or “don’t work”. It’s always a question of do things work well enough in a wide enough range of situations.”
  • 56.
    58 Google search volumes(AR) over time (trends) PokemonGo Distribution: July 2016 22-sept-2017: initial release Iphone 8 19-sept-2017: Initial release of iOS 11 iOS NEW API: ARKit April 2012: Google unveiled Project Glass “Similarly, in your business, be sure that before you focus on the results or outcome, that you make it crystal clear what problems you solve, or why people might need what you offer” - Ian Altman at Forbes Jan 2015
  • 57.
    AR technologies compared 61 MicrosoftHololens Meta 2 ODG R-7 Magic Leap, Google Glass 2.0, Vuzix, ODG R-9, Daqri Smart glasses, ... Daqri Moverio BT-300 + universal use + poweful, open platform + standalone device + strong go-to-market push - limited battery life + universal use + poweful platform - connected device + universal use + lightweight - limited FoV - limited battery life + industry use tailored + powerful platform - industry use only - limited battery life + industry use tailored + powerful platform - industry use only - limited FoV
  • 58.
    AR SDKs Regular HandheldDevices 62 Vuforia ARCore ARKit Dedicated HW HMD Tango Tablet Vuforia ARCore ARKit Hololens Tango
  • 59.
    AR SDKs onHandheld and HMD Devices 63 Augmented Reality SLAM Sensor Fusion Computer Vision Pose Deblurring Visual-inertial Odometry Optimal Correction Surface Estimation Camera Calibration Nonlinear OptimizationScene Understanding Inertial Measurement Unit Triangulation
  • 60.
    AR SDKs onHandheld and HMD Devices 64 It enables developers to position and orient virtual objects, such as 3D models and other media, in relation to real world images when these are viewed through the camera of a mobile or HMD device. Motion Tracking Visual Inertial Odometry (VIO) – to track your position in space (your 6dof pose) in real-time Your pose is recalculated in-between every frame refresh on your display by matching a point in the real world to a pixel on the camera sensor each frame
  • 61.
    AR SDKs onHandheld and HMD Devices 65 It enables developers to position and orient virtual objects, such as 3D models and other media, in relation to real world images when these are viewed through the camera of a mobile or HMD device. 3D Reconstruction (point cloud)
  • 62.
    66 ARKit vs Tangovs Hololens HW Design SW Stack "RGB-IR" camera for color image and infrared depth detection Fisheye motion camera Accelerometer, gyroscope & magnetometer Ambient light sensors Microsoft Holographic Processing Unit VIO (motion tracking) Sparse Mapping (area learning) Dense 3D reconstruction (depth perception). Regular phone camera Accelerometer & gyroscope Highly accurate Dead Reckoning Guess to compensate lack of stereoscopic images
  • 63.
    ARKit – AugmentedReality for iOS 67 • API that lets third-party developers to build Augmented Reality apps for iOS • Available only for processors Apple A9 and later (6S, 6S Plus and later) due to performance • Available only operatig system iOS 11 and later (officially released on 19-sept-2017) Optimal performance from model iPhone 8 • Depth sensing camera system calibrated for AR • Lo-light and 60 fps • New gyro and accelerometer • Accurate motion tracking
  • 64.
    Microsoft Hololens 68 • Accelerometer,gyroscope, magnetometer • Spatial Mapping • Gesture recognition • 3D display (FOV: 30°×17.5°) • Video (2.4 megapixel) • Sound • Bluetooth • Wi-Fi
  • 65.
    VR everywhere atMWC! VR was the big attraction! ..but the real use cases were few!
  • 66.
    VR devices inall ranges! March $599April $799 2nd gen $99 May $200 Oct $499 Multiple version $5 ConsumingInteracting
  • 67.
    But Google DayDreamand ARM next Gen processors will drive mobile VR from consuming to interactive 71
  • 68.

Editor's Notes

  • #3 At cybercom we work in the full lifecycle of software development from advisory, innovation, creation testing and management. Lean startup methodology
  • #4  Jules Verne
  • #5 Mankind has dreamt of going to other worlds for as long as civilization has existed. And not only to go there but also to show other people our visions. The journey to the moon was a science fiction movie from 1902 by George Melies inspired by Jules Verne
  • #6 Melies envisioned what it would be like going to the moon in 1902, I don’t think anyone in the audience believed it would be a reality before the end of the centuary. The important point here I think is that in order to even start working on a technical solution for a goal you must first imagine what might be possible
  • #7 This analog Head Mounted Display or steropticon was popular from the mid 1850-ies until it was replaced by cinemas a few decades later. Man has had this dream of entering other worlds -- I think forever
  • #8 The idea of computer generated virtual reality however is younger – here is an episode from Star Trek from 1966
  • #10 It is time to celebrate augmented reality Augmented reality is in fact soon 50 years in just a few weeks time
  • #11 He must have had guts to wear a head mounted display with cathode ray tubes with 15000 volts just a few centrimetres from his skull.
  • #13 The 80-ies and nineties were I think were a mix of hope and disappointment
  • #14 In 2014 VR had a virtual renaissance when Faceboook bought Oculus and this created a lot of interest in the entire industry
  • #15 So if we look at where we are now, AR and VR are maturing There are five stages in the Gartner Hype Cycle and according this this hype cycle from July VR has left the through of disillusionment and is now in the slop of enlightenment while AR is still in the through of disillusionment.
  • #17 Use cases
  • #20 In the F35 fighter jet they have now eliminated the heads-up display. The F35 helmet allows the pilot to see through the planes fuselage and through his own body. So for example by looking down he sees the ground through the camera. It also has night vision. But you should get some value for $400.000 a piece! http://www.dailymail.co.uk/news/article-3022720/Helmet-allows-pilots-plane-Airmen-new-1trillion-F-35-fighter-ground-thanks-400-000-piece-headgear.html
  • #21 Remote collboration
  • #23 Augmented reality is in fact soon 50 years in just a few weeks time
  • #24 Elevator manufacturer Thussen Krupp uses Hololens for training as well as lett
  • #25 Augmented reality is in fact soon 50 years in just a few weeks time
  • #26 One example of teleprescence or enchanced vision is Loglift which allows users to see outside the truck cabin while still being inside.
  • #28 75% of top brands have VR projects
  • #29 Conceptual design at Volvo Cars
  • #31 Legos new Arkit based game 360syria wich really gives you the sense of being there which is a much more moving experience than ordinary nesw broadcasts for example. Virtual japanese pop star Hatsune Miku. Stadsmissionen 5 minutes of homelessness
  • #33 One such emerging application is MemoAR which allows patients with Alzhemers get help to recognize friends and family
  • #34 Technology - effect - value map
  • #36 Use the outside-in methodology meaning to talk to customers as early as possible and get to know their jobs to be done, pains and gains One way to visualize this is the value proposition canvas.
  • #37 Main challenge: to communicate what it’s like to be homeless so that peopleget emphatic with their cause and donate
  • #38 Augmented reality is in fact soon 50 years in just a few weeks time
  • #40 Use the outside-in methodology meaning to talk to customers as early as possible and get to know their jobs to be donem, pains and gains One way to visualize this is the value proposition canvas.
  • #41 We must become imagineers
  • #42  Jules Verne
  • #48 Used in movies like Avatar Procedural genearation for many natural elements like terrain, vegetaion, clouds, and skies Great eco system
  • #49 Hype realistic quality that can be used in realtime and game engines as well
  • #59 Pokemon Go isn’t particularly sophisticated — or truly AR — the game continues to be a catalyst, inspiring companies and governments to take a closer look at how immersive AR solutions can benefit their business processes. Pokemon Go distribution day Australien 6 juli 2016 Europa 13 juli–24 juli 2016 Japan 22 juli 2016 Sverige 16 juli 2016 USA 6 juli 2016 22-sept-2017 initial release Iphone 8, ”first iPhone deeply integrated for Augmented Reality” 19-sept-2017 Initial release of iOS 11 iOS mobile operating system developed by Apple . NEW: ARKit API that lets third-party developers build augmented reality apps.
  • #63 Vuforia is an Augmented Reality Software Development Kit (SDK) for mobile devices. AR applications developed using Vuforia are therefore compatible with a broad range of mobile devices including the iPhone, iPad, and Android phones and tablets. (it seems like an Hybrid system such as React Native)
  • #64 It enables developers to position and orient virtual objects, such as 3D models and other media, in relation to real world images when these are viewed through the camera of a mobile device. 
  • #65 Source: https://medium.com/super-ventures-blog/why-is-arkit-better-than-the-alternatives-af8871889d6a Source: https://en.wikipedia.org/wiki/Vuforia_Augmented_Reality_SDK Source: https://www.cnbc.com/video/2017/09/12/watch-apple-demonstrate-the-new-iphone-8s-augmented-reality-features.html
  • #66 The films shown are from project Tango, although performance must be similar with Hololens. 3D reconstruncion allows: * positioning in complex environments * navigation both indoor and outdoors
  • #67 With this slide I want to explain only that the HW available in Tango and Hololens makes them much better for 3D Reconstruction and that give them an edge over ARKit (which indeed it is only good in plane detection but not in 3D modelling) Dead Reckoning explained: https://medium.com/super-ventures-blog/why-is-arkit-better-than-the-alternatives-af8871889d6a
  • #70 The Catalunya tourist board offered virtual tours of the region so you never even had to leave the convention center Either this is going to create an unstoppable momentum that pushes VR this year into the mainstream. Or, the mobile industry is chasing fool’s gold and is going to be in for a huge disappointment this year. Gartner 5 till 10 år How virtual reality can create the ultimate empathy machine “VR is the next platform, where anyone can create and experience anything they want,” Zuckerberg said. “For right now, it’s mostly used for gaming. That’s quickly evolving
  • #71 We had a rage of devices launched or displayed – some missing like PlayStation vr Samsung and google are on their second generation HTC pre-booking -> 15k in 10min Oculus is now shipped in july if booked now You have a choice depending on your use case – Computer (not MAC due to graphic card) or phone Phones will be computer equivalent in end of 2016 and during 2017 some say 2020, manufacturers would have shipped as many as 30 million virtual reality headsets and that revenue from hardware sales will exceed $4 billion by that time. We favor Sony’s PlayStation platform as an initial driver of VR penetration due to the high concentration of hardcore gamers on the platform, the likely seamless integration of the VR and host machines (due to the same company designing and selling the VR, software, and hardware), and content availability. Sony has announced its flagship racing title, GranTurismo, will be available on PSVR. Alcatel OneTouch, for example, debuted a new handset, the Idol 4. While it looks like a great budget smartphone in its own right, what's truly interesting is ... its packaging. The Idol 4's box is a plastic affair that, when re-assembled, turns into a sturdy version of Google Cardboard facing camera. HTC and Valve have also developed a Chaperone system that allows you to stay connected with the world around you. It does this by tracking your body's position and also adding information from a front-facing camera over your virtual view
  • #73 VR is the next platform, where anyone can create and experience anything they want,”Zuckerberg said. “For right now, it’s mostly used for gaming. That’s quickly evolving.” Prices on vr, capacity, shipments, mc I hope this (VR) will be one of the killer apps for 5G,” Zuckerberg said in an onstage interview on the first official day of the conference. Samsung announced it’ll be throwing in a free Gear VR and six games if you preorder one of the new Galaxy phones before March 18
  • #74 When doing enterprise use cases it has been difficult to create the content but now you even have consumer devices for that. This will also drive the social use case with upload to internet LG and Samsung has not released prices but a guess some few 100$ - nokias is 60k dollar That you can create your own content I think is the biggest change coming on MWC, this will both drive consumer adoption of the VR technology for consumption. Ricoh, Kodak also launched cameras
  • #75  Jules Verne