A talk from the Work Track at AWE USA 2018 - the World's #1 XR Conference & Expo in Santa Clara, California May 30- June 1, 2018. Sascha Goldner (Airbus Defence and Space): A Case-Study for a UAV Operator Environment Based on Mixed Reality How can a consumer technology like VR/AR “rolled out” in a traditional and in some aspects even conservative company like Airbus Defence and Space? Beside of answering this question, a case-study shall be presented in which Mixed Reality is used in order to provide an enhanced and very isolated environment for a UAV operator. http://AugmentedWorldExpo.com

1. Sascha Goldner, Head of End to End Support Services
01 June 2018
A Case-Study for a UAV Field
Operator Environment Based on
Mixed Reality

2. The World of Airbus
2
unclassified
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

3. AR/MR/VR at Airbus
3
unclassified
How can consumer technology like
VR/AR be “rolled out” in a “traditional”
company like Airbus?
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

4. Challenge
4
unclassified
• Digitalization is triggering many new ideas and we must find out quickly
and cost-effectively what works and what does not (“Deliver Fast, Learn Fast Mentality”)
• Our core information system has high natural entry-barrier for new solutions,
regardless of target maturity (procurement, security, project process, …)
two-speed model in order to respond to these challenges.
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

5. Digital initiatives use one or more of these capabilities
The Airbus Digital Incubator
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

6. WHAT is ProtoSpace?
„Idea + new technology  prototype“
„ProtoSpace provides all Airbus employees an open door
to make their ideas real and prototype them.“
„ProtoSpace is a network”
USUAL setup
SPRINT / Brainstorming room(s):
Innovate, collaborate, exchange
Prototyping Lab(s):
try and fail, experiment, learn and demonstrate
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

7. 7
What happens in ProtoSpace?
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

8. Use Case 1
8
UAV Field Operator Environment Based on Mixed Reality
unclassified
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

9. UAV Field Operator Environment
9
unclassified
• UAV Field Operators often have to work in
small, busy and very noisy areas
• This makes it very hard for them to focus and
concentrate on operating the Ground Control
station
• The Ground Control station is an Android-
based application installed on a ASUS
Transformer AiO 18,4“ tablet
Problem description
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

10. HTC Vive based UAV Operator Environment
10
unclassified
• Use of commercial Virtual Reality devices to
• Create a closed environment
• Mirror the Ground station control panel into VR using AirDroid
• Enable input via controllers and gestures (LeapMotion)
• Enable switching between VR and real-world using the built-in front
camera
Approach
• Are promising but…
Results
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

11. HTC Vive based UAV Operator Environment
11
unclassified
• Use of commercial Virtual Reality devices to
• Create a closed environment
• Mirror the Ground station control panel into VR using AirDroid
• Enable input via controllers and gestures (LeapMotion)
• Enable switching between VR and real-world using the built-in front
camera
Approach
• Are promising but…
Results
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

12. Lessons Learned
• Acceptance by UAV Field Operators was surprisingly good
• HTC Vive due to computing requirements not suitable for field use
• Hand based-interaction (in this case with Leap Motion) is a MUST
Lessons Learned
• Improve hand-based interaction
• Replace AirDroid app
• Find alternative solution to HTC Vive
Next steps
12
unclassified
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

13. Q & A
13
unclassified
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

14. 01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality14

15. Use Case 2
15
AR-based eLearning & maintenance for an aircraft
“Transparent EUROFIGHTER”
unclassified
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

16. Transparent EUROFIGHTER
16
unclassified
• For training applications, road
shows and exhibitions it is
advantageous, if technical
features of a product can be
visualized on the product itself
or on a model of it.
Problem description
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

17. Transparent EUROFIGHTER
17
unclassified
• Using augmented reality to visualize hidden structures of a
EUROFIGHTER
• Marker-less approach (pattern matching)
• Any CAD models can be used (need to be optimized though)
• The software is initialized from different positions around the EF. After
successful initialization the display model is being tracked, i.e the position
of the user relative to the aircraft is continuously re‐calculated
• Consequently, if the user moves around the EF display model, the
projected CAD models are re‐calculated to reflect the modified position of
the user.
• Windows-based tablet PC
Approach
• Are promising but…
Results
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

18. Lessons Learned
• Marker-less tracking works “ok”, nevertheless having similar
looking shapes challenge the software
• High contrast between structure and background required  only
limited outdoor usability
Lessons Learned
• Combine the marker-less approach with a marker-based feature
in order to gain stability while user moves
• Integration of AR-glasses and iOS/Android devices
Next steps
18
unclassified
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality

19. Video
19
unclassified
01 June 2018 A Case-Study for a UAV Operator Environment Based on Mixed Reality