A talk from the Work Track at AWE USA 2018 - the World's #1 XR Conference & Expo in Santa Clara, California May 30- June 1, 2018. Detlef Schiron (Airbus Defence & Space): Towards a Next Generation Fighter Jet Helmet Mounted Display Symbology – a Playground Case-Study based on Microsoft’s HoloLens This talk will take a look at a case-study where Microsoft’s HoloLens is used to prototype Next-Generation-Fighter-Aircraft Helmet-Mounted-Display symbology and to enhance existing simulator environments be it for training or for development. http://AugmentedWorldExpo.com

1. Detlef Schiron, Expert Simulation and Visualization Technology
30 May 2018
Towards a Next Generation Fighter
Jet Helmet Mounted Display
Symbology
A Playground Case-Study based on Microsoft’s
HoloLens

2. The World of Airbus
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology2
unclassified

3. 30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology3
unclassified

4. HMD – An AR System
• HMD = Helmet-Mounted-Display
• It’s a Augmented Reality Device
• It projects information into the pilot‘s eyes, to
• provide aircraft status information
• enhance situational awareness
• cue weapons systems
What is a HMD?
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology4
unclassified

5. Use Case 1
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology5
Simulated Helmet Mounted Displays
unclassified

6. HMD – Application in Flight Simulators
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology6
unclassified
• Original HW is
• Expensive
• Complex (to integrate)
• Difficult to modify (even if you are the OEM)
• Bulky (CRT based)
• Not suited for a simulator environment
• Simulation (e.g. overlay on a dome screen) is not
as representative as required
• Eurofighter HMD currently is mono-color
-> subject of change
Problem description

7. HoloLens based HMD Playground
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology7
unclassified
• Use of commercial Mixed Reality devices
• As HMD replacement in flight simulators
• For prototyping of future multi-color / holographic content
Approach
• Are mixed, but promising …
Results

8. HMD – Lessons Learned
Lessons Learned
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology8
unclassified
+ Tracking in a simulation dome worked surprisingly well
+ Focal distance of HL matches almost exactly the radius of our simulation dome
o The field-of-view
- WiFi-only data transmission is a MASSIVE problem (NATO-secret environment)
- HoloLens seems not suited for head-fixed content
-> color separation
- Video streaming onto the system is a problem
@Microsoft:
PLEASE give the thing cable-connectivity again and maybe allow configuring the motion
compensation

9. MR HMD - Evolution
• Other look-through HW has to be evaluated.
Preferably with „off-board“ rendering and cable connection.
Hardware
• Mono -> Multi Color
• Lines -> Faces
• 2D -> „2.5D“ (by shading)
• 2D -> 3D (by holograms)
Graphical Elements
• GIS-data and meta-data
Data Sources
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology9
unclassified

10. Use Case 2
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology10
Simulator Extensions
unclassified

11. MR Glasses as Flight Simulator Enhancement
• There are tasks which are very difficult to train
in a simulator adequately
• Some of them are related to stereo-vision, e.g.
• Air to air refuelling
• Landing on moving platforms
Status Quo / problem description
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology11
unclassified

12. MR glasses as Flight Simulator enhancements
• Use a MR device to render the relevant parts of the
simulated scene
(Probe-basket-system / landing pad)
Approach
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology12
unclassified

13. MR glasses as Flight Simulator enhancements
• First prototypes have shown:
This is actually much more difficult than thought…
Status
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology13
unclassified

14. What actually the problems are…
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology14
unclassified

15. MR glasses as Flight Simulator enhancements
• Improve understanding of
on-screen-objects vs. holograms
• Derive strategies for solutions
• Look for students…
Plan
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology15
unclassified

16. MR glasses as Flight Simulator enhancements
• When MR glasses will have an
• increased FoV (>150°)
• higher resolution (~8k) and
• are more comfortable to wear
When dreams come true…
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology16
unclassified
Why not replacing the visual display system of a simulator
completely by MR?

17. Thank you

18. Backup / Excurse
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology18
ÁR/MR/VR in Airbus…
unclassified

19. Excurse - AR/MR/VR In Airbus
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology19
unclassified
• Over the last years Airbus identified a number of rising technologies
which have to be integrated into the Airbus working environment
• But how?
• From the past we know:: Top-down does not work…
(slow, ineffective, resistance, …)
Challenge

20. Excurse - AR/MR/VR In Airbus
• Generate environments to support bottom-up
innovation, ideation, prototyping, maturation and
industrialization
• @Airbus we have different initiatives for that
purpose
• One is called ProtoSpace
Idea
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology20
unclassified

21. Excurse - AR/MR/VR In Airbus
• Open to ALL Airbus employees
• Place to meet and to test, prototype and implement their ideas
• Develop expertise through concrete experiments
• Share insights broadly and openly in the ProtoSpace network
ProtoSpace
30 May 2018 Towards a Next Generation Fighter Jet HMD Symbology21
unclassified
<