2017 Serious Play Conference, Washington DC

1. Blackbird
Technologies
Tutorial, Serious Play Conference
Jim Piechocki
Raytheon IIS
Virtual Reality: Tech Disruptor
or Pet Rock

2. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 2
Blackbird
Technologies
Introduction: Your Name and
What Interests You About VR/AR

3. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 3
Blackbird
Technologies
Welcome and Introductions

4. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 4
Blackbird
Technologies
 Safety
 Determine the advantages of immersive learning as an
ROI.
 Craft an immersive learning experience – the most
important elements.
 Communicate benefits, opportunities and potential risks
of VR and AR to your organization, stakeholders and
clients.
 Assess and measure success in the field.
 Use cases – civilian and military
 Interrupt me!!
Tutorial Goals

5. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 5
Blackbird
Technologies
Agenda
TUTORIAL (1 hr. 30 min.)
 What is VR?
 Avoid the Pet Rock Syndrome – Situational Awareness
 VR Diffusion – Gartner Data
 Why Should You Use VR?
University of Washington Data
 Should You Implement VR?
 What is Learning? Quick Instructional Design Primer
 VR Visual and Graphic Design – Avatars or Live Video?
 Crafting Effective VR Scenarios and Characters
 VR Deployment – Head Mounted Displays (HMDs)
 Assessing VR Learning Results

6. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 6
Blackbird
Technologies
What Is Virtual Reality?

7. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 7
Blackbird
Technologies
Avoid the Pet Rock Syndrome – Situational
Awareness

8. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 8
Blackbird
Technologies
VR Diffusion – Gartner Top 10 Strategic
Technologies

9. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 9
Blackbird
Technologies
VR Diffusion – Gartner Study
• Gartner: BY 2020 one in 10 PCs shipped VR-ready

10. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 10
Blackbird
Technologies
Use Case #1: Confined Spaces Virtual Training

11. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 11
Blackbird
Technologies
Use Case #2: Medical Mishaps Training

12. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 12
Blackbird
Technologies
 Relevance and Retention: University of Washington Data
– 38% reduction in defense plant injuries
– Hours lost due to safety-related incidents: declined to 1/3 after
intervention
Why Should You Implement VR?

13. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 13
Blackbird
Technologies
Summary of Data – DSOC / DLA Pilot Test
- $780,000 – Defense
Logistics Agency
- Pilot programs applying 3D
virtual reality to safety needs
- Four sites – 500 exposed,
500 control each
- Warner Robbins AFB, GA
- Tracy Army Depot, CA
- Puget Sound Navy, WA
- Corpus Christi Helicopter
Repair, TX

14. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 14
Blackbird
Technologies
Summary of Data – Injury Reduction
Percent Decrease Injury Rates
Estimated effect of training on
injury rates: -25%
Source: University of Washington Department of Health Services
5.4
4.3
3.4
3.2
0
1
2
3
4
5
6
Intervention
Grioup
Comparison
Group
Corpus Christi
Army Depot
5.4
4.3
3.4
3.2
0
1
2
3
4
5
6
Intervention
Grioup
Comparison
Group
Corpus Christi
Army Depot

15. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 15
Blackbird
Technologies
 Types of Injuries
– Sprains and strains (37%)
– Contusions (20%)
– Trauma (18%)
– Exertion (10%)
 Average cost per injury $3,022
 Cost savings per 1,000 workers =
$45,300 medical expenses (Corpus Christi)
+ Wage replacement payments temporary disability
+ Reduced productivity of work unit
+ Long-term economic benefits >1 year (“enormous”)
 x 26,000 DLA peacetime workers = 1.2M, <200k investment
Source: University of Washington Department of Health Services
Summary of Data – Return on Investment
37%
20%
18%
10%
15%
Sprains and Strains
Contusions
Trauma
Exertion
Other

16. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 16
Blackbird
Technologies
Summary of Data – Days Lost to Injury
3.98
1.2
1
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Category 1
Year 1 Year 2 Year3

17. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 17
Blackbird
Technologies
Activity 1: Brainstorm VR Learning Applications
Why Should You Implement VR?

18. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 18
Blackbird
Technologies
Activity 1 Debrief – Good Use of VR?
 Project / Application  Benefit

19. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 19
Blackbird
Technologies
 VR most suitable for situational awareness – Think in three
dimensions
 What is situational awareness? Problems which involve
multidimensional spaces, components or activities. e.g.,
– X: Government regulatory body
– Y: Company rules and practices
– Z: Workplaces
– X: Missile components
– Y: Deployment terrain
– Z: Military regulations
– X: Confined space interior
– Y: Confined space scaffolding - exterior
– Z: Rescue equipment
Should You Implement VR?

20. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 20
Blackbird
Technologies
Situational Awareness – Use Case #3:
Electrical Line Worker Chop and Drop

21. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 21
Blackbird
Technologies
Situational Awareness – Use Case #4:
IDS Patriot Missile VR Training

22. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 22
Blackbird
Technologies
Situational Awareness – Use Case #5:
University of Edinburgh Emergency Response

23. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 23
Blackbird
Technologies
Situational Awareness – Use Case #6:
Collaborative Learning – Racing Gamified

24. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 24
Blackbird
Technologies
Activity 2: XYZ Situational Awareness
Why Should You Implement VR?

25. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 25
Blackbird
Technologies
What Is Learning? - Quick Instructional
Design Primer

26. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 26
Blackbird
Technologies
 Mager/Kirkpatrick Criterion Referenced Model
– Most important element of most communication: PowerPoints, images,
persuasion  Corporate poetry
– Most important elements of training: fix problems, change behaviors
 Assessment of needs
 Objectives
 Measurement of:
– Level 1: Reaction – Attitude assessment via Likert Scale
– Level 2: Knowledge delta measured in assessments
– Level 3: Behavior delta from OTJ follow up
• Level 4: Business delta: the best criterion for success – defined ROI
metrics used to demonstrate results.
What Is Learning? - Quick Instructional Design
Primer (cont.)

27. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 27
Blackbird
Technologies
 How do you integrate learning into a POV-driven
experience?
 Blended Theoretical model
1. Demonstration
2. Guided practice
3. Solo practice
Blended Virtual Learning Model

28. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 28
Blackbird
Technologies
 Basics of “blended” VR model
– Demonstration: narrated/hosted moment in “launch” environment, darkened
fringe no head-turns
– Guided Practice/Performance: POV Activities: Learning objectives reflected
in full virtual display
 Eyes scan hot points, such as a gallery with images or data, launch fully
visualized missions
 Return to launch point
– Level Design: Utilize to add variety by making POV activities faster, more
challenging (e.g., more difficult tool or procedure)
Blended Virtual Learning Model

29. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 29
Blackbird
Technologies
VR Visual and Graphic Design
Link to: 10 Design Tips (handout/link)
http://www.cobblestonelearning.com/1
0-design-tips-instructional-designers/

30. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 30
Blackbird
Technologies
VR Visual and Graphic Design – Live Video or
Avatars?
 Benefits/Disadvantages of
animated “virtual”
characters?
 Suspension of Disbelief
concept
?

31. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 31
Blackbird
Technologies
Activity 3: Virtual Animated or Virtual “Real”
Benefits / Disadvantages of Each
Why Should You Implement VR?

32. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 32
Blackbird
Technologies
Planning Considerations – Risks and
Dependencies
 Production/Post: Top 5
Issues
– Where do you put grip and
lights? Crew?
– Horizon is an issue in 360
immersive.
– Color and exposure: matching
up to 10 cameras.
– Stitching: out of box software
better but need seasoned DIT,
VR editor.
– $60,000/3 minutes
?

33. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 33
Blackbird
Technologies
Planning Considerations – Risks and
Dependencies
 Solution
– Production: Experienced DIT on
production team
– Consider animation for cost
reasons
– Experienced Editor - $60,000/3
minutes
– 360  160-180 degrees
“blended” format
?

34. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 34
Blackbird
Technologies
VR Image Gathering
3D: Fuji W3, GoPro3 twin
genlocked stereoscopic,
Panasonic prosumer T1000
Today : $40,000 >9 cam, 4 mic
Nokia Ozo
Jaunt

35. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 35
Blackbird
Technologies
Activity 4: Instructional Objectives: Develop Instructional approach:
1. Demonstration
2. Guided Practice
3. Performance
Or… Whatever application you are excited about!
Why Should You Implement VR?

36. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 36
Blackbird
Technologies
Crafting Effective VR Scenario and Characters
– Sabido Method
 Prof. Miguel Sabido
pioneered Mexican
telenovela format
– Mass media application of
Social Learning Theory
(Bandura)
 Sabido Method - method for
blending drama with
behavior change
messaging.
 Basics go back 3000 years
to ancient Greeks - teaching
myths.

37. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 37
Blackbird
Technologies
 Sabido 4-step Questioning Process –
 Parasocial Identification
1. What is my “premise”? E.g., ignoring safety procedures leads to injury;
ruthless ambition leads to its own destruction (Macbeth); technology without
moral restraint leads to the end of humanity (Terminator series); power is
more influential than love and compassion (Game of Thrones)
2. Who is my “protagonist” – embodies change in behavior, on the fence
3. What is my “protagonist behavior arc” – character on the fence must decide
4. Who is my “antagonist or obstacle character” – barrier to behavior change.
Crafting Effective VR Scenario and Characters
– Sabido Method (cont.)
Premise
Obstacle
Character
Protagonist
Learning
Objective

38. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 38
Blackbird
Technologies
 HMDs: Head Mounted Displays, variety of technologies
 Mobile phone: low-cost spectrum, 300-400 dpi, pixelated,
wireless (under $100)
 Midrange: HTC Vive
 Oculus Rift: 4K, wired to computer
– 16-32 gig RAM
– Advanced graphics card (Nvidia)
– Storage per program: varies with format
– Cost: $2000, incl. laptop
VR Head-Mounted Display (HMD) Systems

39. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 39
Blackbird
Technologies
 Augmented Reality – Microsoft Hololens wearable computer
– Learning use case 1: Global learning meeting
– Use case 2: Construction, rapid development (Japan)
– Use case 3: HUD vehicle repair US Postal Service, Honda NOMAD
Augmented Reality HMD Technology

40. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 40
Blackbird
Technologies
 What is VR in learning?
 How do we avoid the pet rock syndrome?
 Situational awareness: Why should you use VR in learning?
 Should you use VR learning?
 What are best practices for designing VR characters and
scenarios?
 What are Kirkpatrick’s Measurement Levels 1-4
 What are two options for 3D HMDs?
 What is Augmented Reality and how does it differ from the
Virtual Reality space?
REVIEW

41. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 41
Blackbird
Technologies
QUESTIONS

42. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 42
Blackbird
Technologies
Activity 5: Hands on review of VR HMD systems
VR/AR HMD Technology

43. RAYTHEON PROPRIETARY/EXPORT CONTROLLED – See Cover
2017 43
Blackbird
Technologies
 Participant Manual
 Handouts: Gartner Research
 Jaunt workflow summary
Thank You! Leave-Alongs