PhD Thesis available on request

1. Leveraging sensing-based interaction
for supporting reflection at work:
the case of crisis training
Simone Mora
AP Photo/Asahi Shimbun, Noboru Tomura

2. Outline
• Problem domain
• Research questions
• Theoretical underpinning
• Research methodology
• Technology tools
• Contributions
• Conclusions and future work
2

3. Image REUTERS/Mainichi Shimbun
During 1992-2012 disasters have
affected 4.4 billion people and caused
USD 2 trillion in damage worldwide
Source: The UN Office for Disaster Risk Reduction
3

4. Crisis Training approaches
Maximising learning
outcomes from experience-
based training events
4
Problem:
COST
REALISM
low highmoderateonerious
Serious
game
Physical
simulation
Tabletop
exercise
Protocol
training

5. Research questions
I. How sensing-based interfaces can be designed
to enable unobtrusive experience collection
during crisis work?
II. How sensing-based interfaces can be designed
to trigger and support reflection activities?
III. How sensing-based interfaces for supporting
reflection can be rapidly prototyped?
5
What are the opportunities introduced by combining reflective learning
theories with sensing-based interfaces for supporting crisis training?
Crisis training
Reflective
learning
Sensing-based
interfaces

6. Reflective learning
WORK EXPERIENCE REFLECTIVE PROCESS OUTCOMES
+ New perspectives
+ Change in
behaviour
+ Readiness
+ Commitment
Sharing of work experience + Reflection = New knowledge
6
[Boud, D., Keogh, R., & Walker, D. (1985). Reflection: turning experience into learning.]

7. Doctoral theses at NTNU, 2015:123
Simone Mora
Leveraging sensing-based
interaction for supporting
reflection at work: the case
of crisis training
NTNU
NorwegianUniversityof
ScienceandTechnology
Thesisforthedegreeof
PhilosophiaeDoctor
FacultyofInformationTechnology,
MathematicsandElectricalEngineering
DepartmentofComputerandInformation
Science
Doctoralthesis
This PhD work
Computer supported reflective learning
7
new
reflection
cycle
C
hange
O
utcom
e
Fram
e
new reflection cycle
Plan and do
work
Plan work
Do work
Monitor work
Initiate
reflection
Set objective
Involve others
Plan session
new
reflection
cycle
Data
Apply outcome
Decide on change to
work (e.g. what, who)
Decide how to make the
change
Decide whether further
reflection is needed
Conduct reflection session
Make related experiences available
Reconstruct or envision work experience
Understand meaning
Articulate meaning
Critique experience
Reach a resolution
Check applicability of reflection outcome
ing of reflection
new cycle)
next stage
e cycle
ge
at may be
ed
EU-IST/FP7 2010-2014 Empirical data
Validation of theories
Technology tools
Reflection Theory
Evaluation framework
[Krogstie, B. R., Prilla, M., & Pammer, V. (2013). Understanding and Supporting
Reflective Learning Processes in the Workplace: The CSRL Model.]

8. Sensing-based interaction and tangible user interfaces
8
Tangible interaction denote system that rely on
embodied interaction, physical representation
of data and embeddedness in the real space
(Hornecker and Buur 2006)
CSCW
new
reflection
cycle
C
hange
O
utcom
e
Fram
e
new reflection cycle
Plan and do
work
Plan work
Do work
Monitor work
Initiate
reflection
Set objective
Involve others
Plan session
new
reflection
cycle
Data
Apply outcome
Decide on change to
work (e.g. what, who)
Decide how to make the
change
Decide whether further
reflection is needed
Conduct reflection session
Make related experiences available
Reconstruct or envision work experience
Understand meaning
Articulate meaning
Critique experience
Reach a resolution
Check applicability of reflection outcome
Triggering of reflection
(new cycle)
Input to next stage
in the cycle
Stage
Activities that may be
involved
Legend:
Generate experiences Capture experiences
Re-create experiences
DATA
WORK
REFLECTION
LEARNING
INTUITIVE
PLAYFUL
EMBODIED
EMBEDDED

9. Methodology overview
9
ENVIRONMENT KNOWLEDGE BASEDESIGN SCIENCE
Hardware and
software rapid
prototyping
Formative
evaluation
Computer
Supported
Reflective
Learning
Crisis
Training
Sensing-based
interaction
Relevance
cycle
Rigor
cycle
Exploratory
field studies
Evaluation
field studies
Design
cycle
[Hevner, Alan R (2007). “A three cycle view of design science research”.]

10. Exploratory studies
10
• OBSERVATIONS
• SHADOWING
• VIDEO
• INTERVIEWS
METHODS:

11. Prototypes
11
2010 2011 2012 2013 2014
Exploratory field studies
Prototypes
F2F1
F6F5
G1 G2 G3 G4
W1 W4W2
D1 D2
W3
C1 C2
Research abroad
Papers
CITY MIT
P7 P1 P4
P6
P5
P2
P4
Field evaluations
F3 F4
Formative Evaluations
RELEVANCE
CYCLE
DESIGN
CYCLE
RIGOR
CYCLE
W4W3
W2W1
C2C1
D2D1

12. Evaluation studies
12
• QUESTIONNAIRES
• INTERVIEWS
• OBSERVATIONS
METHODS:

13. 13
0" 1" 2" 3" 4" 5"
To interact with W. I had to shift my sight
away from the rescue work
To interact with W. I had to shift my hands
away from the rescue work
To interact with W. I had to shift my
attention away from the rescue work
I believe that after some usages I will
interact with W. without shifting attention
W. empowered me to capture data without
interupting the rescue work
W. Interfered with the rescue work
Novices"
Experts"
Version 1.0
guidance throughout the reflection process (CAs, M=4.13, SD=0.42). The acceptance rate is
higher in the population of participants with more than 5 years of experience (M=4.26,
SD=0.41), than among novices (M=3.96, SD=0.37). A higher acceptance rate is found
among experts across all the app-specific questions asked (see table X), the performed (two-
tailed) t-test shows that the difference is statistically significant (p=0.036).
Table X. Means and standard deviations for app specific questions
ID Question All
(N=35)*two
users didn’t
report yrs of
experience
M (SD)
At least 5
years of
experience
(N=18)
Less than 5
years of
experience
(N=15)
CA2 WATCHiT helped me to reflect on
experiences
4.06 (0.59) 4.28 (0.46) 3.73 (0.59)
CA6 WATCHiT helped me to reconstruct a
work experience
4.29 (0.52) 4.39 (0.50) 4.13 (0.52)
CA7 WATCHiT helped me by capturing my
reflection outcomes
4.15 (0.56) 4.18 (0.53) 4.07 (0.59)
CA12 WATCHiT helped me by providing
accurate information about my work
4.03 (0.51) 4.22 (0.55) 3.80 (0.41)
CA40 WATCHiT provided relevant content for
reflection
4.29 (0.52) 4.44 (0.62) 4.07 (0.27)
CA41 WATCHiT guided me through the
reflection process
3.91 (0.61) 4.00 (0.59) 3.80 (0.68)
In both experiments we have added in the pre-questionnaire the short reflection scale (CRs).
Results show that participants have a positive inclination towards reflection both on an
individual level (M=4.38, SD=0.30) and as a team (M=4.22, SD=0.38). As the evaluations
Evaluation studies

14. Outline
• Problem domain
• Research questions
• Theoretical underpinning
• Research methodology
• Technology tools
• Contributions
• Conclusions and future work
14

15. WATCHiT
P2 P3 P6 P7
RQ1 RQ2
C2 C4C1
CAPTURE EXPERIENCE
15

16. 16

17. 17
TAG
HAPTIC
FEEDBACK

18. 18

19. You're in: New York City, U.S.
CroMAR 8:24 AM
MoreMapAR NotesLayers
Face Time
Share
Timeline
"I'm trapped in my car
#panic!"
@marco via twitter
Next tweet containing
#panic (1 of 10+)
0.7 Km
5" later
MMS from +3934667xxx
Tags: car, obstacle
Noise level
95 dB CCTV Video
"Agent Dokes in
sector 5 needs an
ambulance"
Yesterday (Oct, 27th)
14:00 16:00 18:00 20:00 22:00 24:00
CroMAR
19
P1 P2 P6 P7
RQ2 RQ3
C3 C4C1
RE-CREATE EXP

20. 20
PHYSICAL EXPLORATION
RADAR & MAPSMULTIPLE LAYERS
TIMELINE NAVIGATION
REFLECTION
OUTCOMES EDITOR

21. 21

22. P4 P5 P7
RQ2 RQ3
C3 C4
Don’t Panic
C1
GENERATE EXP
22

23. 23
NAVIGATE
CALM, MOVE OR RESTRAIN CITIZENS
MANAGE INFORMATION UNDER STRESS

24. 24

25. Outline
• Problem domain
• Research questions
• Theoretical underpinning
• Research methodology
• Technology tools
• Contributions
• Conclusions and future work
25

26. Papers
26
P1: Mora, S., Boron, A., & Divitini, M. (2012). CroMAR: Mobile
Augmented Reality for Supporting Reflection on Crowd
Management. International Journal of Mobile Human
Computer Interaction.
P2: Mora, S., & Divitini, M. (2014). Supporting Debriefing with
Sensor Data: A Reflective Approach to Crisis Training. In
Proceedings of Information Systems for Crisis Response and
Management in Mediterranean Countries, ISCRAM-MED.
P3: Mora, S., & Divitini, M. (2014). WATCHiT: a modular and
wearable tool for data collection in crisis management and
training. In Proceedings of the European Conference in
Ambient Intelligence, AMI.
P4: Di Loreto, I., Mora, S., & Divitini, M. (2012). Don’t Panic:
Enhancing Soft Skills for Civil Protection Workers. In
Proceedings of International Conference on Serious Games
Development Applications, SGDA.
P5: Mora, S., Di Loreto, I., & Divitini, M. The interactive-token
approach to board games. Ready for submission.
P6: Müller, L., Divitini, M., Mora, S., Rivera-Pelayo, V., &
Stork, W. (2014). Context Becomes Content: Sensor Data for
Computer Supported Reflective Learning. IEEE Transactions
on Learning Technologies.
P7: Mora, S., & Farshchian, B. A. (2010). A Unified
Architecture for Supporting Direct Tag-Based and Indirect
Network-Based Resource Discovery. In Proceedings of the
International Conference on Ambient Intelligence, AMI.

27. 27
C1
C2
C3
C4
P2
ISCRAM-
med
´14
P3
AMI
‘14
P4
SGDA
‘12
P6
IEEETLT
‘15
P7
AMI
‘10
P1
IJMHCI
‘12
P5
‘15
Instantiation of
CSRL model
Design of tools for
experience-capture
Novel sensing-based
interaction techniques
Challenges in building
prototypes
Conference Journal Submitted Contribution
Theory
Technology
TEL ISCRAM TEI

28. Contribution I
28
Implementation and evaluation of MIRROR Computer
Supported Reflective Learning (CSRL) theory
C2
C3 C4
P3
AMI
‘14
P7
AMI
‘10
P1
IJMHCI
‘12
P5
‘15
C1 P2
ISCRAM-
med
´14
P4
SGDA
‘12
P6
IEEETLT
‘15

29. Contribution II
29
Knowledge about designing experience-capturing tools for
crisis workers
1. Mobility of work and sensing
2. Different crises, different relevant data
3. Different types of data
4. Sensor data and user-submitted data
5. Different use, different sharing
6. Intuitive, hands-free interaction
7. Automate and discrete capturing
C1
P4
SGDA
‘12
P6
IEEETLT
‘15
C3 C4
P7
AMI
‘10
P1
IJMHCI
‘12
P5
‘15
C2
P3
AMI
‘14
P2
ISCRAM-
med
´14

30. Contribution III
30
Novel sensing-based interaction techniques to support
crisis training
CAPTURE EXPERIENCES RE-CREATE EXPERIENCES GENERATE EXPERIENCES
C2
P2
ISCRAM-
med
´14
C1
P4
SGDA
‘12
P6
IEEETLT
‘15
C4
P7
AMI
‘10
C3P1
IJMHCI
‘12
P5
‘15
P3
AMI
‘14

31. Contribution IV
31
Knowledge about implementing prototypes to be deployed
into the wild
C3P1
IJMHCI
‘12
C2
P2
ISCRAM-
med
´14
C1
P4
SGDA
‘12
P6
IEEETLT
‘15
C4
P7
AMI
‘10
P5
‘15
P3
AMI
‘14

32. 32
C2
C3 C4
C1
The CSRL model can be instantiated into an
ecology of sensing-based interfaces for the
capture, re-creation and generation of crisis
work experiences
Future work will further map with technology
the activities described in the model
Conclusions and future work
Theory in the field of tangible, embodied and
embedded computer can drive the design of
user interfaces for supporting reflection in
crisis training
Future work will generalise findings to other
application domains
Building sensing-based interfaces require a
wide skill set. Further technology to be tested
during (simulated) crisis work has to be build
for high resilience
Future work will develop tools to ease the
development of sensing-based systems
Unpredictably of relevance for the data and
highly varying crisis scenarios make hard to
design experience-capturing tools. A design
space is provided in a set of design choices
Future work will explore further
implementation of the design space

33. Thanks!
The work here presented is co-funded by EU-ICT 7FP MIRROR project
(http://www.mirror-project.eu). Photos by ANPAS Piemonte and Simone Mora.