This slideshow is about an independent research team's investigation into the value of a new proprietary augmented reality industrial sales and service training system called, "ScopeAR." The university research team conducted a pre-test/post-test study involving employees from three North American industrial sites, using the New Krikpatrick Model (2009) to evaluate the viability of this training system versus traditional face-to-face and computer-mediated training. This slideshow introduces AR, the Scope AR training system, and the research method before presenting results from the study. This study determined that the Scope AR training program surpassed any previous face-to-face or computer-mediated accentuator valve training experience.

2. OUTLINE
• Introduction
• Background
• Methodology
• Results
• Discussion
• Challenges and recommendations
• Concluding remarks

3. INTRODUCTION
AR technology – innovative solution to rapidly-evolving industry needs:
• Reduces training time and $
• Increases workplace safety
• Supplies contemporary expert instruction on demand (Remote AR)
Scope AR training package:
• Digital multimedia instruction delivered on site via iPad and/or AR glasses
• Active, real time, “see-and-do” learner-paced approach
• Animated step-by-step instruction using holographic overlay on real objects

4. BACKGROUND
Augmented reality (AR):
• Projection of digital sensory information (e.g., images, sounds) onto real world
objects (Azuma, 1997; Höllerer & Feiner, 2004; Richardson et al., 2014)
Scope AR training system:
• Registers and renders digital image overlays on
real world valves using mobile and wearable
devices
• Enables learners to align, explore, and manipulate
3D virtual and physical objects in real time

5. BACKGROUND
Theoretical framework:
* New World Kirkpatrick model (Kirkpatrick, Kirkpatrick, & Kirkpatrick, 2009):
• Level 1: Reaction (Satisfaction, Engagement, Relevance themes)
• Level 2: Learning (Knowledge, Skills, Attitudes, Confidence, Commitment)
• Level 3: Behaviours (Application of Learning)
• Level 4: Results (Target Achieved)
* Study focused on first 3 levels

6. METHODOLOGY
• Pragmatic mixed methods approach
• First 3 New World Kirkpatrick model levels (Reaction, Learning, Behaviours)
• Data collection:
• 3 North American worksites
• Online, mobile-friendly pre-training post-training quantitative and qualitative surveys
• Follow-up telephone interviews 1 – 4 months after training with purposive sub-sample
• Data analysis:
• Single snapshot assessment
• Demographics
• Previous training materials and resources
• New AR training materials and resources

7. RESULTS
Instrument completion:
• Pre-survey N=41
• Post-survey N=31 (76% of pre-survey respondents)
• Interview N=8
Scope AR resources:
• All respondents used an iPad
• 58% had access to actuator, valve, positioner assembly
• 55% used AR glasses

8. PRE- AND POST-SURVEY RESULTS
Demographics:
• 92% male
• Even spread of ages from 20-59
• 85% employees under supervision
• 66% completed university; 27% finished trade/technical school
Digital literacy:
• 100% used computers for learning; moderate to high level of confidence
• 76% used mobile devices for learning; moderate to high level of confidence
• 71% had not used AR technology for any reason in the past

9. PRE- AND POST-SURVEY RESULTS
2%
2%
10%
22%
22%
42%
Type of Previous Valve Training
Paper & Computer
Computer & Hands-On
Hands-On
Paper &Hands-On
All Three
No Training
Type of Training
Figure 1. Type of previous training (N = 41)

10. PRE- AND POST-SURVEY RESULTS
Summary of previous & new AR training by Kirkpatrick theme:
• Response to AR training overwhelmingly positive, regardless of previous or no
training history
• New AR training received higher scores than previous training for all themes
• Previous hands-on training received second highest cumulative ranking
• Prior paper- and computer-based training ranked poorly in all thematic areas
• Computer-based training least valued

11. 0.00% 10.00% 20.00% 30.00% 40.00% 50.00%
Target Achieved
App of Learning
Commitment
Skills
Confidence
Knowledge
Attitude
Engagement
Relevance
Satisfaction
% of All Kirkpatrick Levels
Interview Comment Units by Kirkpatrick Theme
Level Categories
Kirkpatrick Level
Level 1: Reaction
Level 2: Learning
Level 3: Behaviours
Level 4: Results
TELEPHONE INTERVIEW RESULTS
Figure 2. Results of interview comment units identified by categories within each level of the New World Kirkpatrick training
evaluation model, expressed in percent of all units (unit = one sentence).

12. DISCUSSION
Mitigating factors:
• 42% did not have access to real valve assembly; 29% of these respondents did
not have previous valve training either
“This is a very great concept, but there was no hands-on to apply what I learned. Also, I have
never tried to repair a valve so I am unfamiliar with its parts.”
• 45% did not have access to AR glasses
• Since AR training is “hands-on,” it is not surprising that it shared similar scores with
previous hands-on training.
• Unusual fluctuations in the theme “Relevance and Effectiveness” scores reflected
difference between current and future job

13. CHALLENGES AND RECOMMENDATIONS
Not all respondents agreed with each other:
“The tablet AR module was easy and straight forward to follow. I like the idea of being able to put it down
and do the hands-on and being able to revert back to the tablet. This is truly cutting edge.”
“The one thing I didn't like about the mobile device is having to put it down every time to complete a task.”
AR Technology Concerns:
• Most prevalent issue was image manipulation (e.g., “a little glitchy,” “lack of multiple angle
feedback,” “image too opaque”)
• App availability (current app and other future industry-related apps)
• Environmental and safety issues (e.g., cold weather environments, safety-rated AR glasses)

14. CHALLENGES AND RECOMMENDATIONS
AR Instruction:
• Recommended tweaks to minor content changes (e.g., some more
granular instructional steps)
• Better quality and greater number of images
• FAQs, including links to manuals
• More safety content
• Respondents were very pleased with the time it took to learn valve
repair (typically 1 hour), but desired future opportunities to
practice

15. CONCLUDING REMARKS
• Scope AR valve repair and maintenance instructional system enthusiastically supported
• Scope AR favoured over previous training
• Scope AR deemed highly efficient, cost-effective, cutting edge, on-demand learning tool
that sales personnel, customers, and “16 year-old nephews” can use, “put[ting] the
company ahead of its competitors”
• Some technical glitches and instructional content issues
• Concern with availability of current and future apps
• Respondents envisioned a future with remote AR training possibilities; Scope AR is now
employing remote industrial AR training systems
• Pervasive sentiment of trainee respondents is “Please get this technology ASAP.”

17. Dr. Mohamed Ally
mohameda@athabascau.ca
Norine Wark
norinewark@gmail.com

18. REFERENCES
• Azuma, R. (1997). A survey of augmented reality. Presence, Teleoperators and Virtual Environments,
6(4), 355-385.
• Hӧllerer, T., & Feiner, S. (2004). Chapter 9: Mobile augmented reality. In H. Karimi and A. Hammad
(Eds.), Telegeoinformatics: Location-based computing and services (pp. 1-38). Oxford, UK: Taylor &
Francis Books Ltd.
• Kirkpatrick, D., Kirkpatrick, J., & Kirkpatrick, W. (2009). The New Kirkpatrick Model. Retrieved from
http://www.kirkpatrickpartners.com/OurPhilosophy/TheNewWorldKirkpatrickModel/tabid/303/D
efault.aspx
• Richardson, T., Gilbert, S., Holub, J., Thompson, F., MacAllister, A., Radkowski, R., Winer, E., Davies, P.,
& Terry, S. (2014, December). Fusing self-reported and sensory data from mixed reality training.
Paper presented at the Interservice/Industry Training, Simulation, and Education Conference
(I/ITSEC) 2014, Orlando, Florida.