1. michael.hildebrandt@ife.no

2. resilient procedure use

3. resilient procedure use study 2014
video: https://vimeo.com/98273509

4. products

6. EOP flowchart Process overview display
Micro-task tablet tool Eye tracking
simulator plug-in

10. nuc
IFE ProcSee
what if…

11. operation training validation research

12. operation training validation research
EOP tool ✔ ✔ ✔ ✔
Overview
display ✔ ✔ ✔
Micro-task
tablet app ✔ ✔ ✔
Eye tracking
plugin ✔ ✔ ✔

13. micro tasks
s w i p e t o s t a r t

14. video: https://vimeo.com/131387407

15. problem

16. How to test performance benefits
of new interface solutions

17. How to increase confidence in
HRA results and safety cases

18. solution

19. benefits

20.  Generate quantitative results about
performance benefits (time and reliability)
of digital/innovative CR interfaces
 Highly efficient, objective method
 Compare to existing reference data
 Qualitative insights via eye tracking
 Data directly supports HRA
 Highly customisable and precise
Generate exactly the data needed for the HRA

21. big picture

22. Micro task
tablet app
Standardized
question sets
Database
training
hra interface
design
gamification
evaluation of training
validation
a mobile companion for
HRA / HF specialists
benchmarking
within / across organisations

23. example

24. 2014 micro task study

25. innovative vs conventional displays

26. innovative vs conventional displays

27. pumps
conventional innovative

28. mass balance
conventional innovative
Do the innovative displays lead to faster,
more reliable identifications and decisions?

29. mass balance
conventional innovative
Is performance (time and reliability) with innovative
displays at least as good as with conventional?

30. method

31.  Large number of questions
Related to systems, components, procedures, etc.
 Varying levels of difficulty
Including higher-level decision making
 Different display conditions
E.g. innovative vs. conventional displays
 We measure response time / accuracy
 Compare data between conditions
If needed, review eye tracking recordings
to understand anomalies
detection/decision tasks under time pressure

32. example

33. 4 operators
working in the
simulator
individually
(no communication)

34. We control
which displays
are available
(eg innovative, conventional)

35. Instructions:
”Please answer the
questions correctly,
but also as quickly
as possible.
It is very important
that you work as
fast as you can.”

36. Run 1
s w i p e t o s t a r t

37. How many condensate pumps
are running on turbine 31?
1 2 3
s w i p e t o
c o n t i n u e

38. “How many condensate pumps
are running on turbine 31?”
Average identification time
Conventional 8 sec
Innovative 6 sec

39. Is the subcooling margin sufficient?
Yes
s w i p e t o
c o n t i n u e
No

40. Which steam generators
are faulty?
SG-1
s w i p e t o
c o n t i n u e
SG-2 SG-3

41. Should safety injection
be stopped?
Yes
s w i p e t o
c o n t i n u e
No

42. What is the narrow-range level
in steam generator 1?
s w i p e t o
c o n t i n u e
%
1 2 3 4 5 6 7 8 9 0
37

43. up to 200 questions per hour
with 4 operators in the simulator working individually,
that means we can run up to 800 questions per hour

44. results

45. data source
where does this data come from?

46. 6840 data points
3420 response time measures
3420 accuracy measures

47. 20 operators

48. 5 hours simulator time
in total for the whole data collection

49. innovative displays are superior

50. HSI mean performance times
Current effect: F(2, 8,003 )=21,889, p=,00057
Vertical bars denote 0,95 confidence intervals
LSD (Innovative) OWD (Conventional) LSD and OWD
15
16
17
18
19
20
21
22
23
24
25
26
Performancetime(sec)
Operators were faster with innovative
displays than with conventional displays
innovative
conventional
HSI mean performance times
Current effect: F(2, 8,003 )=21,889, p=,00057
Vertical bars denote 0,95 confidence intervals
21
22
23
24
25
26
etime(sec)
Highly statistically
significant difference

51. Slightly more accurate with innovative
displays than conventional displays
Percentage of correct answers
innovative conventional innovative

52. Performance variability

53. conclusion

54.  As reliable as conventional displays
 Faster to read
innovative displays in the 2014 study are

55. what next?

57. Compare analog CR to HAMMLAB

58. Obtain reference data about analog CR

59. Interested parties / collaborators
 AREVA Germany
 Idaho National Labs
 Virginia Tech University
 GSE Systems
 U.S. NRC

60. features

61.  MT app trigger simulator events
 Simulator events trigger MT app
 Integration with WL methods
 Database prototype