ENABLING
TECHNOLOGIES FOR
THE IMMERSIVE
TRANSFORMATION
Vinay Narayan, Vice President, HTC
Nirav Desai, Chief Technologist, Booz Allen Hamilton
AUGMENTED WORLD EXPO, SANTA CLARA, CA, MAY 31, 2018
INTRODUCTIONS
• Pioneering Enterprise applications of VR,
including training, mission planning, a integrated
solutions for security and operations
• Structured several deals connecting innovative
startups with large enterprises and governments
• Edge technology startup mentor
Nirav S. Desai
Chief Technologist
Strategic Innovation Group
Booz Allen Hamilton
Vinay Narayan
Vice President
Products and Operations
HTC America
• Led product management for first commercial
room-scale VR platform
• XR Ecosystem expansion and Enterprise
development
• XR and AI Board Member and Advisor
SINCE THE DAWN OF WRITING, WE HAVE BEEN 3D BEINGS
COMMUNICATING IN A 2D WORLD.
I M M E R S I V E T E C H N O L O G Y W I L L C H A N G E T H I S .
Bison cave painting from Altamira, Spain – Smithsonian.com
THE IMMERSIVE TRANSFORMATION PRESENTS A HUGE MARKET
POTENTIAL
GROWTH CATEGORIES IN VR
AI
FORECAST
Mobile Transformation
THE IMMERSIVE TRANSFORMATION
Earlier 1980 1985 1990 1995 2000 2005 2010 2015 2020 2025
Mainframe
Transformation
PC Transformation Immersive Transformation
intuitive, pervasive, &
natural computing
AI
Immersive
+
THE IMMERSIVE TRANSFORMATION WILL STRUCTURE AND
PROVIDE COHERENCE EMERGING TECHNOLOGIES.
B I N G O
IOT Smart Agents Gesture/Voice
Computer
Vision
AI
Gaze Tracking Wearables MoCap VR
Neuromorphic
Hardware
Cloud
Volumetric
Display
Free NLP Immersive ML
MR BCI Affective Tech Telepresence Big Data
Photo-
grammetry
Spatial
Mapping
AR Haptics 5G
THE IMMERSIVE TRANSFORMATION WILL STRUCTURE AND
PROVIDE COHERENCE EMERGING TECHNOLOGIES.
IOT/BCI/Wearables
Voice/Chatbots
Machine Intelligence
Status Quo Horizon 1 Horizon 3Horizon 2
VR Immersive Trans. Immersive Trans.
2018 ~2019 ~2020-2021 ~2022+
AR/MR
B I N G O
IOT Smart Agents Gesture/Voice
Computer
Vision
AI
Gaze Tracking Wearables MoCap VR
Neuromorphic
Hardware
Cloud
Volumetric
Display
Free NLP Immersive ML
MR BCI Affective Tech Telepresence Big Data
Photo-
grammetry
Spatial
Mapping
AR Haptics 5G
Hierarchy Individual Level Team Level Enterprise/Government Level
Technology
Enablers
• Individual and multi-user training
environments
• Room-scale VR, Location-based AR
• Multi-user, multi-location Mixed Reality
• Shared spatial mapping
• Multi-perspective collaboration
• Integration of real-time data
• Multi-dimensional visualization
LET’S MAP THE APPLICATION OF THESE TECHNOLOGIES AGAINST
VARIOUS WORK ACTIVITIES.
7
Operational/Front-line Activities
Technologies that workers will use day-to-day to perform their core business activities
Training Activities
Technologies that are used to train a workforce of front-line workers
Planning Activities
Technologies that are used by business decision makers to visualize data or inform decisions
Hierarchy Individual Level Team Level Enterprise/Government Level
Technology
Enablers
• Individual and multi-user training
environments
• Room-scale VR, Location-based AR
• Multi-user, multi-location Mixed Reality
• Shared spatial mapping
• Multi-perspective war gaming
• Integration of real-time data
• Multi-dimensional visualization
LET’S MAP THE APPLICATION OF THESE TECHNOLOGIES AGAINST
VARIOUS WORK ACTIVITIES.
8
Operational/Front-line Activities
Technologies that workers will use day-to-day to perform their core business activities
Training Activities
Technologies that are used to train a workforce of front-line workers
Planning Activities
Technologies that are used by business decision makers to visualize data or inform decisions
Virtual Reality Augmented Reality
Mixed Reality
Hierarchy Individual Level Team Level Enterprise/Government Level
Technology
Enablers
• Individual and multi-user training
environments
• Room-scale VR, Location-based AR
• Multi-user, multi-location Mixed Reality
• Shared spatial mapping
• Multi-perspective war gaming
• Integration of real-time data
• Multi-dimensional visualization
LET’S MAP THE APPLICATION OF THESE TECHNOLOGIES AGAINST
VARIOUS WORK ACTIVITIES.
9
Operational/Front-line Activities
Technologies that workers will use day-to-day to perform their core business activities
Training Activities
Technologies that are used to train a workforce of front-line workers
Planning Activities
Technologies that are used by business decision makers to visualize data or inform decisions
Virtual Reality Augmented Reality
Mixed Reality
IoT Integration
Advanced Headsets Volumetric Display
Eye & Gaze
Affective Comp.
Affective Comp.
Neuromorphic HW
Computer Vision
Multi-user RT Datafeeds
Gesture/Voice Cloud Immersive ML
Telepresence Affective Comp.
Cloud Immersive ML
Fiduciary MarkersSpatial MappingTrackers
Haptics Smart Agents
BCI MoCap
Spatial Mapping
Volumetric Display
Horizon 1 Horizon 2 Horizon 3
BCI 5G
Cloud
DIGITAL
ENVIRONMENT
ENHANCED
ENVIRONMENT
WHAT IS
IMMERSIVE
TECHNOLOGY?
DESIGN REVIEW / VISUALIZATION SHOWROOM
TRAINING LOGISTICS TRAINING / PLANNING / SIMULATION
WHAT IS AI?
ARTIFICIAL INTELLIGENCE ( AI) IS A RESEARCH AREA OF HOW
TO MAKE MACHINES THINK AND BEHAVE INTELLIGENTLY.
CV • Pattern Recognition & Computer
Vision
ML • Machine Learning
NLP • Natural Language Processing
ASR • Speech Recognition
ES • Expert Systems
IMMERSIVE AI EXAMPLES
13Booz Allen Hamilton Internal
AI.Solve’s Children’s Hospital LA
Trauma Center AI-based Training
Booz Allen uses computer vision to identify drone threats and Mixed
Reality to help security professionals secure high value targets.
ENFORCE
FIELDOPERATIONAL COMPUTER VISION
TO DETECT AND CLASSIFY THREATS
FOR RAPID RESPONSE
WHAT IS EDGE COMPUTING?
(LEVERAGE REMOTE I/O TO INFORM COMPUTING: IOT, BCI,
WEARABLES
Emotiv Epoc BCI Headset | emotiv.com
MS Kinect for Azure | Microsoft.com
Fitbit | fitbit.com
MIT's DuoSkin | Duoskin.media.mit.edu
IMMERSIVE EDGE COMPUTING EXAMPLES
Valorem’s HoloBeam uses computer vision to
detect a body frame and project a holographic
representation of a user across location.
Vive X accelerator graduate Neurable uses BCI
for a brainwave-controlled VR gaming.
HOW WILL THIS BE ROLLED OUT ACROSS THE ENTERPRISE?
16
NowFuture
TRAINING PLANNING OPERATIONS
Booz Allen is helping train airmen how to safely exit
a burning aircraft.
Osso’s Virtual Surgery helps surgeons, sales teams,
and hospital staff develop skills.
U.S. Army TRADOC is developing requirements for a multi-user, cross-
geography, multi-platform training/simulation environment.
HaptX’s haptic glove could help surgeons feel tactile,
pressure, and heat sensations to further the realism
of virtual surgery.
HOW WILL THIS BE ROLLED OUT ACROSS THE ENTERPRISE?
17
NowFuture
TRAINING PLANNING OPERATIONS
Dirtt Environmental is using VR to design building
interiors.
Booz Allen is working with the U.S. Navy to help
submariners plan their missions in VR.
Integration of CAD and Revit Models with IOT sensors will allow us to
model, run exercises, and monitor real environments in 3D.
Image: Jurassic Park
Several companies are already working on 3D
operations centers.
Image: James Cameron's Avatar
HOW WILL THIS BE ROLLED OUT ACROSS THE ENTERPRISE?
18
NowFuture
TRAINING PLANNING OPERATIONS
Raymond is working with Free Range Games to
develop immersive trainers to make forklift
operators more effective on the job.
Taqtile is using mixed reality to support 3D technical work
instruction for maintenance of industrial equipment.
3D telepresence drones will allow workers to perform
functions via robotics in dangerous environments.
Virtual teleconferencing through augmented reality will allow
collaboration across geography.
Image: LucasArts
KEY TAKE-AWAYS
19
@ V I N AY _ N A R AYA N
V P P ro d u c t a n d O p e ra t i o n s
H TC V I V E
@ N I R AV S D E S A I
C h i e f Te c h n o l o g i s t
B o o z A l l e n H a m i l t o n

Nirav Desai (Booz Allen Hamilton): Enabling Technologies for the Immersive Transformation

  • 1.
    ENABLING TECHNOLOGIES FOR THE IMMERSIVE TRANSFORMATION VinayNarayan, Vice President, HTC Nirav Desai, Chief Technologist, Booz Allen Hamilton AUGMENTED WORLD EXPO, SANTA CLARA, CA, MAY 31, 2018
  • 2.
    INTRODUCTIONS • Pioneering Enterpriseapplications of VR, including training, mission planning, a integrated solutions for security and operations • Structured several deals connecting innovative startups with large enterprises and governments • Edge technology startup mentor Nirav S. Desai Chief Technologist Strategic Innovation Group Booz Allen Hamilton Vinay Narayan Vice President Products and Operations HTC America • Led product management for first commercial room-scale VR platform • XR Ecosystem expansion and Enterprise development • XR and AI Board Member and Advisor
  • 3.
    SINCE THE DAWNOF WRITING, WE HAVE BEEN 3D BEINGS COMMUNICATING IN A 2D WORLD. I M M E R S I V E T E C H N O L O G Y W I L L C H A N G E T H I S . Bison cave painting from Altamira, Spain – Smithsonian.com
  • 4.
    THE IMMERSIVE TRANSFORMATIONPRESENTS A HUGE MARKET POTENTIAL GROWTH CATEGORIES IN VR AI FORECAST
  • 5.
    Mobile Transformation THE IMMERSIVETRANSFORMATION Earlier 1980 1985 1990 1995 2000 2005 2010 2015 2020 2025 Mainframe Transformation PC Transformation Immersive Transformation intuitive, pervasive, & natural computing AI Immersive +
  • 6.
    THE IMMERSIVE TRANSFORMATIONWILL STRUCTURE AND PROVIDE COHERENCE EMERGING TECHNOLOGIES. B I N G O IOT Smart Agents Gesture/Voice Computer Vision AI Gaze Tracking Wearables MoCap VR Neuromorphic Hardware Cloud Volumetric Display Free NLP Immersive ML MR BCI Affective Tech Telepresence Big Data Photo- grammetry Spatial Mapping AR Haptics 5G
  • 7.
    THE IMMERSIVE TRANSFORMATIONWILL STRUCTURE AND PROVIDE COHERENCE EMERGING TECHNOLOGIES. IOT/BCI/Wearables Voice/Chatbots Machine Intelligence Status Quo Horizon 1 Horizon 3Horizon 2 VR Immersive Trans. Immersive Trans. 2018 ~2019 ~2020-2021 ~2022+ AR/MR B I N G O IOT Smart Agents Gesture/Voice Computer Vision AI Gaze Tracking Wearables MoCap VR Neuromorphic Hardware Cloud Volumetric Display Free NLP Immersive ML MR BCI Affective Tech Telepresence Big Data Photo- grammetry Spatial Mapping AR Haptics 5G
  • 8.
    Hierarchy Individual LevelTeam Level Enterprise/Government Level Technology Enablers • Individual and multi-user training environments • Room-scale VR, Location-based AR • Multi-user, multi-location Mixed Reality • Shared spatial mapping • Multi-perspective collaboration • Integration of real-time data • Multi-dimensional visualization LET’S MAP THE APPLICATION OF THESE TECHNOLOGIES AGAINST VARIOUS WORK ACTIVITIES. 7 Operational/Front-line Activities Technologies that workers will use day-to-day to perform their core business activities Training Activities Technologies that are used to train a workforce of front-line workers Planning Activities Technologies that are used by business decision makers to visualize data or inform decisions
  • 9.
    Hierarchy Individual LevelTeam Level Enterprise/Government Level Technology Enablers • Individual and multi-user training environments • Room-scale VR, Location-based AR • Multi-user, multi-location Mixed Reality • Shared spatial mapping • Multi-perspective war gaming • Integration of real-time data • Multi-dimensional visualization LET’S MAP THE APPLICATION OF THESE TECHNOLOGIES AGAINST VARIOUS WORK ACTIVITIES. 8 Operational/Front-line Activities Technologies that workers will use day-to-day to perform their core business activities Training Activities Technologies that are used to train a workforce of front-line workers Planning Activities Technologies that are used by business decision makers to visualize data or inform decisions Virtual Reality Augmented Reality Mixed Reality
  • 10.
    Hierarchy Individual LevelTeam Level Enterprise/Government Level Technology Enablers • Individual and multi-user training environments • Room-scale VR, Location-based AR • Multi-user, multi-location Mixed Reality • Shared spatial mapping • Multi-perspective war gaming • Integration of real-time data • Multi-dimensional visualization LET’S MAP THE APPLICATION OF THESE TECHNOLOGIES AGAINST VARIOUS WORK ACTIVITIES. 9 Operational/Front-line Activities Technologies that workers will use day-to-day to perform their core business activities Training Activities Technologies that are used to train a workforce of front-line workers Planning Activities Technologies that are used by business decision makers to visualize data or inform decisions Virtual Reality Augmented Reality Mixed Reality IoT Integration Advanced Headsets Volumetric Display Eye & Gaze Affective Comp. Affective Comp. Neuromorphic HW Computer Vision Multi-user RT Datafeeds Gesture/Voice Cloud Immersive ML Telepresence Affective Comp. Cloud Immersive ML Fiduciary MarkersSpatial MappingTrackers Haptics Smart Agents BCI MoCap Spatial Mapping Volumetric Display Horizon 1 Horizon 2 Horizon 3 BCI 5G Cloud
  • 11.
  • 12.
    DESIGN REVIEW /VISUALIZATION SHOWROOM TRAINING LOGISTICS TRAINING / PLANNING / SIMULATION
  • 13.
    WHAT IS AI? ARTIFICIALINTELLIGENCE ( AI) IS A RESEARCH AREA OF HOW TO MAKE MACHINES THINK AND BEHAVE INTELLIGENTLY. CV • Pattern Recognition & Computer Vision ML • Machine Learning NLP • Natural Language Processing ASR • Speech Recognition ES • Expert Systems
  • 14.
    IMMERSIVE AI EXAMPLES 13BoozAllen Hamilton Internal AI.Solve’s Children’s Hospital LA Trauma Center AI-based Training Booz Allen uses computer vision to identify drone threats and Mixed Reality to help security professionals secure high value targets. ENFORCE FIELDOPERATIONAL COMPUTER VISION TO DETECT AND CLASSIFY THREATS FOR RAPID RESPONSE
  • 15.
    WHAT IS EDGECOMPUTING? (LEVERAGE REMOTE I/O TO INFORM COMPUTING: IOT, BCI, WEARABLES Emotiv Epoc BCI Headset | emotiv.com MS Kinect for Azure | Microsoft.com Fitbit | fitbit.com MIT's DuoSkin | Duoskin.media.mit.edu
  • 16.
    IMMERSIVE EDGE COMPUTINGEXAMPLES Valorem’s HoloBeam uses computer vision to detect a body frame and project a holographic representation of a user across location. Vive X accelerator graduate Neurable uses BCI for a brainwave-controlled VR gaming.
  • 17.
    HOW WILL THISBE ROLLED OUT ACROSS THE ENTERPRISE? 16 NowFuture TRAINING PLANNING OPERATIONS Booz Allen is helping train airmen how to safely exit a burning aircraft. Osso’s Virtual Surgery helps surgeons, sales teams, and hospital staff develop skills. U.S. Army TRADOC is developing requirements for a multi-user, cross- geography, multi-platform training/simulation environment. HaptX’s haptic glove could help surgeons feel tactile, pressure, and heat sensations to further the realism of virtual surgery.
  • 18.
    HOW WILL THISBE ROLLED OUT ACROSS THE ENTERPRISE? 17 NowFuture TRAINING PLANNING OPERATIONS Dirtt Environmental is using VR to design building interiors. Booz Allen is working with the U.S. Navy to help submariners plan their missions in VR. Integration of CAD and Revit Models with IOT sensors will allow us to model, run exercises, and monitor real environments in 3D. Image: Jurassic Park Several companies are already working on 3D operations centers. Image: James Cameron's Avatar
  • 19.
    HOW WILL THISBE ROLLED OUT ACROSS THE ENTERPRISE? 18 NowFuture TRAINING PLANNING OPERATIONS Raymond is working with Free Range Games to develop immersive trainers to make forklift operators more effective on the job. Taqtile is using mixed reality to support 3D technical work instruction for maintenance of industrial equipment. 3D telepresence drones will allow workers to perform functions via robotics in dangerous environments. Virtual teleconferencing through augmented reality will allow collaboration across geography. Image: LucasArts
  • 20.
    KEY TAKE-AWAYS 19 @ VI N AY _ N A R AYA N V P P ro d u c t a n d O p e ra t i o n s H TC V I V E @ N I R AV S D E S A I C h i e f Te c h n o l o g i s t B o o z A l l e n H a m i l t o n

Editor's Notes

  • #9 Expand relevant tech list on the bottom – list tech instead of investments here.
  • #10 Expand relevant tech list on the bottom – list tech instead of investments here.
  • #11 Expand relevant tech list on the bottom – list tech instead of investments here.
  • #12 Define what we are talking about Immersion: Brain Engagement Function of XR: help us manipulate content and data in a much more natural way.  Versus sitting in front of a computer
  • #13 VOLKSWAGEN AS EXAMPLE OF ALL-ENCOMPASSING VR USE CASES
  • #14 Computers deal with structured data Unstructured Data Humans have a hard way of dealing with 5G upto 10 Gigabits/second, LTE: 1/s
  • #17 Combining advanced technologies with distributable platforms and hardware is the basis for Immersive Transformation