This document provides background information about Evangelos Karapanos, including his education and work experience. It outlines some of his research interests, such as designing technologies for social inclusion in primary schools, citizen participation using mobile technologies, and awareness technologies for families and schools. The document discusses his work on designing technologies to raise awareness of resource consumption and induce feelings of accountability. It aims to support awareness within families through mobile sensing of social activity.

1. Measuring users’ experiences
or, the memory of them?
Trajectory reminders EmoSnaps Footprint tracker iScale
Evangelos Karapanos
Skopje, 5 June 2012
Wednesday, June 6, 12

2. My
background
BSc Physics / microelectronics, U Patras, Greece (2004)
Thesis: Model based design and evaluation of walk-up-and-use interfaces (HCI
Group, ECE department)
MSc HCI / UCL Interaction Centre, UK (2005)
Thesis: User acceptance of nomadic user interfaces (Philips Research,
Eindhoven)
PhD HCI / TU Eindhoven, NL (2010)
Title: Quantifying diversity in user experience
Assist. Prof. HCI / Madeira ITI (2010-)
Design for Experience in pervasive computing
Wednesday, June 6, 12

3. Soft Reliability
48% of returned products are not attributed to a
violation of product specifications
Wednesday, June 6, 12

4. problems rooted early
in (concept) design phase
Wednesday, June 6, 12

5. failure to truly incorporate it in one’s life
Wednesday, June 6, 12

6. 436 Studies in Computational Intelligence 436
The series Studies in Computational Intelligence (SCI) publishes new developments
Karapanos
and advances in the various areas of computational intelligence – quickly and with
high quality. The intent is to cover the theory, applications, and design methods
of computational intelligence, as embedded in the fields of engineering, computer
science, physics and life sciences, as well as the methodologies behind them.
The series contains monographs, lecture notes and edited volumes in computational
intelligence spanning the areas of neural networks, connectionist systems, genetic
algorithms, evolutionary computation, artificial intelligence, cellular automata,
self-organizing systems, soft computing, fuzzy systems, hybrid intelligent, and
virtual reality systems. Of particular value to both the contributors and the
Coming!
readership are the short publication timeframe and the world-wide distribution,
which enable both wide and rapid dissemination of research output.
Over the past decade the field of Human-Computer Interaction has evolved from the
study of the usability of interactive products towards a more holistic understanding
Evangelos Karapanos
of how they may mediate desired human experiences.
This book identifies the notion of diversity in users? experiences with interactive
1
products and proposes methods and tools for modeling this along two levels:
Modeling Users'
Modeling Users' Experiences with Interactive Systems
(a) interpersonal diversity in users? responses to early conceptual designs, and
(b) the dynamics of users? experiences over time.
The Repertory Grid Technique is proposed as an alternative to standardized
June 2012
psychometric scales for modeling interpersonal diversity in users? responses to early
concepts in the design process, and new Multi-Dimensional Scaling procedures are
Experiences with
Interactive Systems
introduced for modeling such complex quantitative data.
iScale, a tool for the retrospective assessment of users? experiences over time is
proposed as an alternative to longitudinal field studies, and a semi-automated
technique for the analysis of the elicited experience narratives is introduced. Through
Foreword: Jean-Bernard Martens
these two methodological contributions, this book argues against averaging in the
subjective evaluation of interactive products. It proposes the development of
interactive tools that can assist designers in moving across multiple levels of
Closing note: Marc Hassenzahl
abstraction of empirical data, as design-relevant knowledge might be found on
all these levels.
Foreword by Jean-Bernard Martens and Closing Note by Marc Hassenzahl.
issn 1860-949X
isbn 978-3-642-30999-1
9 783642 309991
springer.com
13
Wednesday, June 6, 12

7. Wednesday, June 6, 12

8. Nuno
Nunes Vassilis
Kostakos Monchu
Chen
Laura Rodríguez Gonçalo
Gouveia Néstor
Catano
• 20
faculty Pedro
Campos Paulo
Sampaio Eduardo
Fermé
– 14
countries,
8
languages
• Areas: Larry
ConstanIne Jos
van
Leeuwen Barbara
Pizzileo
– 11
CS,
2
physics/electronics,
2
psychology,
2
architecture,
2
design,
2
art,
2
other
Ian
Oakley Luis
Gomes Ron
Salden
Leonel
Nóbrega ValenIna
Nisi Evangelos
Karapanos
David
Aveiro Luis
Gomes Yoram
Chisik
Wednesday, June 6, 12

9. MSc HCI & Entertainment Technology
Wednesday, June 6, 12

10. Industry Involvement
Wednesday, June 6, 12

11. Design
for
Experience
in
pervasive
compu3ng
Socially translucent eco-feedback technologies
How do eco-feedback technologies:
a) raise mutual awareness of family members’ consumption behaviors
! b) induce feelings of accountability on individuals regarding their consumption behaviors.
Technologies for Social Inclusion in primary schools
a) Using sociometric technologies to assess the inclusiveness of school communities
b) Designing Persuasive technologies that challenge pupils’ perceptions of diversity
Citizen participation on the go
How can we motivate citizen participation through mobile technologies?
•Public transit: The role of psychological empowerment: self-efficacy, sense of community,
and causal importance
Awareness technologies for parents, children and school
e Senseµ ( a) To support interpersonalinfer the physical,within family social activity of a pupil
b)
Using mobile sensors to
connectedness
verbal and
hat aims at supporting awareness in parent
c) To engage parents and school in ad-hoc communication
Location-aware narratives: Does locality matter?
Does the coupling between physical and virtual space result to increased immersion in
the narrative world?
Wednesday, June 6, 12

12. Design
for
Experience
in
pervasive
compu3ng
Socially translucent eco-feedback technologies
How do eco-feedback technologies:
a) raise mutual awareness of family members’ consumption behaviors
! b) induce feelings of accountability on individuals regarding their consumption behaviors.
Technologies for Social Inclusion in primary schools
a) Using sociometric technologies to assess the inclusiveness of school communities
b) Designing Persuasive technologies that challenge pupils’ perceptions of diversity
Citizen participation on the go
How can we motivate citizen participation through mobile technologies?
•Public transit: The role of psychological empowerment: self-efficacy, sense of community,
and causal importance
Awareness technologies for parents, children and school
e Senseµ ( a) To support interpersonalinfer the physical,within family social activity of a pupil
b)
Using mobile sensors to
connectedness
verbal and
hat aims at supporting awareness in parent
c) To engage parents and school in ad-hoc communication
Location-aware narratives: Does locality matter?
Does the coupling between physical and virtual space result to increased immersion in
the narrative world?
Wednesday, June 6, 12

13. Design
for
Experience
in
pervasive
compu3ng
Socially translucent eco-feedback technologies
How do eco-feedback technologies:
a) raise mutual awareness of family members’ consumption behaviors
! b) induce feelings of accountability on individuals regarding their consumption behaviors.
Technologies for Social Inclusion in primary schools
a) Using sociometric technologies to assess the inclusiveness of school communities
b) Designing Persuasive technologies that challenge pupils’ perceptions of diversity
Citizen participation on the go
How can we motivate citizen participation through mobile technologies?
•Public transit: The role of psychological empowerment: self-efficacy, sense of community,
and causal importance
Awareness technologies for parents, children and school
e Senseµ ( a) To support interpersonalinfer the physical,within family social activity of a pupil
b)
Using mobile sensors to
connectedness
verbal and
hat aims at supporting awareness in parent
c) To engage parents and school in ad-hoc communication
Location-aware narratives: Does locality matter?
Does the coupling between physical and virtual space result to increased immersion in
the narrative world?
Wednesday, June 6, 12

14. Design
for
Experience
in
pervasive
compu3ng
Socially translucent eco-feedback technologies
How do eco-feedback technologies:
a) raise mutual awareness of family members’ consumption behaviors
! b) induce feelings of accountability on individuals regarding their consumption behaviors.
Technologies for Social Inclusion in primary schools
a) Using sociometric technologies to assess the inclusiveness of school communities
b) Designing Persuasive technologies that challenge pupils’ perceptions of diversity
Citizen participation on the go
How can we motivate citizen participation through mobile technologies?
•Public transit: The role of psychological empowerment: self-efficacy, sense of community,
and causal importance
Awareness technologies for parents, children and school
e Senseµ ( a) To support interpersonalinfer the physical,within family social activity of a pupil
b)
Using mobile sensors to
connectedness
verbal and
hat aims at supporting awareness in parent
c) To engage parents and school in ad-hoc communication
Location-aware narratives: Does locality matter?
Does the coupling between physical and virtual space result to increased immersion in
the narrative world?
Wednesday, June 6, 12

15. Design
for
Experience
in
pervasive
compu3ng
Socially translucent eco-feedback technologies
How do eco-feedback technologies:
a) raise mutual awareness of family members’ consumption behaviors
! b) induce feelings of accountability on individuals regarding their consumption behaviors.
Technologies for Social Inclusion in primary schools
a) Using sociometric technologies to assess the inclusiveness of school communities
b) Designing Persuasive technologies that challenge pupils’ perceptions of diversity
Citizen participation on the go
How can we motivate citizen participation through mobile technologies?
•Public transit: The role of psychological empowerment: self-efficacy, sense of community,
and causal importance
Awareness technologies for parents, children and school
e Senseµ ( a) To support interpersonalinfer the physical,within family social activity of a pupil
b)
Using mobile sensors to
connectedness
verbal and
hat aims at supporting awareness in parent
c) To engage parents and school in ad-hoc communication
Location-aware narratives: Does locality matter?
Does the coupling between physical and virtual space result to increased immersion in
the narrative world?
Wednesday, June 6, 12

16. Measuring users’ experiences
or, the memory of them?
Trajectory reminders EmoSnaps Footprint tracker iScale
Skopje, 5 June 2012
Wednesday, June 6, 12

17. Technology Assisted Reconstruction
Trajectory reminders EmoSnaps Footprint tracker iScale
Skopje, 5 June 2012
Wednesday, June 6, 12

18. User Experience defined
a momentary,
primarily evaluative feeling (good-bad)
while interacting with a product or service
Hassenzahl, 2008
Wednesday, June 6, 12

19. Most of our evaluation tasks
rely on memory
Wednesday, June 6, 12

20. Heuristics and biases
• Peak-and-end phenomenon
• Summary judgments can be best predicted by a simple
average of the most extreme and the last experience
(Fredrickson and Kahneman, 1993)
• Replicated in HCI - Summarizing mental effort to
perceived usability: end matters (Hassenzahl and
Sandweg, 2004)
• Other biases (e.g., Rosy retrospection, Focusing illusion
etc.)
Wednesday, June 6, 12

21. Why do memory biases exist?
Robison & Clore (2002)
“The emotional experience can neither
be stored nor retrieved”
it is reconstructed from recalled
contextual details
Wednesday, June 6, 12

22. Why do memory biases exist?
Robison & Clore (2002)
event, but instead, every attempt to Type of Knowledge Source of information Type of Self-Report
often altered representation of the
“The emotional experience can neither
articipants to recall an unfamiliar Experiential Online emotion, e.g.
Episodic
be stored nor retrieved”
d 20 hours before. Recalled stories
Knowledge Experience Sampling
nal one in missing details, altering
ance is reconstructedin applying
it of events, or from recalled
Retrospective, e.g.
contextual details
erpretations to the original story.
Episodic Episodic memory
Day Reconstruction
r distorted through repeated
Situation-specific
Semantic Exit questionnaires
bering is an act of reconstruction belief
ction has received wide support. At
ction lies the distinction between
Identity-related
memory [69]. While episodic Semantic
belief
Exit questionnaires
a particular event from the past,
ot tied to any particular event but
Robinson & Clore (2002)
n generalizations (i.e. beliefs) that Figure 1. Four sources of information in emotional self-repor
935). These two types of memory according to Robinson and Clore [63]. Figure adapted from [6
uch as learning new information
Wednesday, June 6, 12

23. bottom line...
If you want to know what the user really experiences,
ask her at that exact moment!
Wednesday, June 6, 12

24. Experience Sampling Method
Prompts at random, or computationally
estimated times, to self-report on ongoing
behaviors and experiences.
– Where are you?
– What are you doing?
– How far is your mobile phone?
– How do you feel?
Karapanos, E. (2012) Experience Sampling, Day Reconstruction, what’s next?
Towards Technology-Assisted Reconstruction. M-ITI internal report.
Wednesday, June 6, 12

25. Experience Sampling Method
Prompts at random, or computationally
estimated times, to self-report on ongoing
behaviors and experiences.
– Where are you?
– What are you doing?
– How far is your mobile phone?
– How dovenues would the ACM a substantial number
a few relevant
you used still miss Guide to Computing
of studies. We instead
feel? method while following a user-initiated diary approach.
The analysis of the remaining 49 studies is being reported
Literature querying for the term “experience sampling” below.
No of papers referring to
without constraining to particular venues. This query
returned 284 papers, published in more than fifty venues. Study length, sampling frequency, and response rate
Experience Sampling The majority (80%) of the studies had a duration of several
60 days up to one month with 14 studies (34%) lasting
between four and seven days (see figure 3). Only two
studies had a duration of more than a month.
45
15
30
15
10
0
2001 2003 2005 2007 2009 2011
5
Figure 2. 243 papers referring to experience sampling over a
Karapanos, E. (2012) Experience Sampling, Day Reconstruction, what’s next?
ten-year period. Retrieval took place on August 26th, 2011.
Towards Technology-Assisted0 Reconstruction. M-ITI internal1m > 1m
≤1h < 24h ≤ 3d ≤ 1w ≤ 2w ≤
report.
Fourty-one papers were excluded from further processing.
Wednesday, June 6, 12

26. What variables do ES studies measure? sampling method. Two of these provided no justifications
Experience Sampling Method
We distinguish below between self-reported measures of for their choice. Analyzing the remaining 19 papers resulted
behavior and experience (see Table 1). This distinction is to a total of 11 reasons for choosing alternative methods to
relevant as their reconstruction follows a different process ESM (see table 2).
whereas behavioral information may be directly accessible
through episodic memory while experiential information Table 2. Reasons for not selecting the Experience Sampling
has to be further inferred from recalled episodic cues [63]. Method along with frequency of occurence (No of papers).
Prompts at random, orES studies eliciting self-reported
Table 1. Number of computationally Reason No
measures of behavior, experience, or both.
estimated times, to self-report on ongoing Disrupts the activity 6
behaviors and experiences.
Type of measures that studies elicit No
Imposes high burden to participants 3
Self-reported measures of behavior 5
Requires high effort from researchers 3
– Where areSelf-reported measures of experience
you? 22
– What are you doing? of behavior & experience 18
Self-reported measures
Inappropriate for eliciting rich qualitative data 3
– How far is your mobile phone? Misses rare and brief events 3
– How dovenues would thethe participant Computing with analysisactivityremainingThe studiesshould be in control of when, what and
you used stillACM a substantial number method whileofbeing a user-initiated isdiary approach.
feel? miss Guide to was engaged The prior to following 49 user being reported
a few relevant Behavioral measures related most frequently to the
of studies. We instead that
(n=18) the 2
Literature querying for the term “experience sampling” (n=2) (e.g. [35]), the
interrupted (e.g. [30]), its duration below. how often to report
Noparticipant’s referringphysical location (n=15) (e.g.sampling frequency, and response rate
of papers current to
without constraining to particular venues. This query
returned 284 papers, published in more than fifty venues. Study length,
[17])
Experience Sampling (n=10), e.g. the number or nature ofof the studies hadsample size several
and the social context
The majority (80%) Limits a duration of 2
relationship of people that are in close proximity month with 14 studies (34%) lasting
days up to one or
60
participate with in a conversation (e.g. [34]). Other
between four and seven days (see figure 3). Only two
Depends on participants’ ability to articulate 2
measures of behavior related to modestudiestransitduration of moreongoing experience
of had a [25], than a month.
45
participants’ current physical engagement [18] and mode of
convrersation (e.g. f2f, fixed/mobile phone etc.) [32].
15
30 Poses privacy concerns 2
Experiential measures related to:
15
10
• Attitudes towards behaviors or events (n=15) such as Limits number of measured variables 1
0
being interrupted (e.g. [55]), disclosing information to
2001 2003 2005others (e.g. [17]), or being video recorded (e.g.
relevant 2007 2009 2011 Technology limitations 1
5
[58]).
Figure 2. 243 papers referring to experience sampling over a
• Measures of affectE. (2012) Experience Sampling, Day Reconstruction, the most frequent reason
Karapanos, and experience (n=18) such as mental
ten-year period. Retrieval took place on August 26th, 2011.
As expected, what’s next? for not selecting the
0
Towards [18]) and concentration [14], ≤ 3d ES 1w ≤ 2w was the interruptions that
method
engagement (e.g. Technology-Assisted Reconstruction. M-ITI internal1m > 1m
report. the method imposes
≤1h < 24h ≤ ≤
Fourty-one papers were excluded [27], further processing.
Wednesday, June 6, 12 satisfaction from mood and emotional states (e.g. [53], on the user’s activity (eg. [48], [5]). For instance, Lindley

27. Day Reconstruction Method
Kahneman et al. (2004)
Can a retrospective method help participants in
recalling more accurately their experiences?
ry attempt to Type of Knowledge Source of information Type of Self-Report
ntation of the
an unfamiliar Episodic
Experiential
Knowledge
Online emotion, e.g.
Experience Sampling
called stories
tails, altering
in applying Retrospective, e.g.
Episodic Episodic memory
riginal story. Day Reconstruction
h repeated
Situation-specific
Semantic Exit questionnaires
econstruction belief
e support. At
ion between
Identity-related
hile episodic Semantic
belief
Exit questionnaires
om the past,
lar event but Robinson & Clore (2002)
beliefs) that
Wednesday, June 6, 12
Figure 1. Four sources of information in emotional self-report

28. Day Reconstruction Method
Kahneman et al. (2004)
Can a retrospective method help participants in
recalling more accurately their experiences?
Wednesday, June 6, 12

29. Technology
Assisted
ReconstrucIon
Can mobile sensors assist participants in reconstructing
their daily experiences and whereabouts?
Trajectory reminders EmoSnaps Footprint tracker iScale
Wednesday, June 6, 12

30. Trajectory reminders in location-based preferences
Do trajectory reminders (locations visited before and after) increase the test-retest
reliability of the reconstruction process?
Control condition With trajectory reminders
! !
Wednesday, June 6, 12

31. Emosnaps - inferring emotion from self-face pics
Can self-face pictures assist in recalling momentary emotions?
If so, is it through a recognition or a reconstruction process?
Wednesday, June 6, 12

32. Emosnaps - inferring emotion from self-face pics
Can self-face pictures assist in recalling momentary emotions?
If so, is it through a recognition or a reconstruction process?
Wednesday, June 6, 12

33. Emosnaps - inferring emotion from self-face pics
Can self-face pictures assist in recalling momentary emotions?
If so, is it through a recognition or a reconstruction process?
Experience Sampling (Ground truth)
78% of pictures
could be used for
inferring emotions
Time-Day Photo-day Photo-week
control condition Emotion reconstruction Emotion recognition
Wednesday, June 6, 12

34. Emosnaps - inferring emotion from self-face pics
Can self-face pictures assist in recalling momentary emotions?
If so, is it through a recognition or a reconstruction process?
Experience Sampling (Ground truth)
78% of pictures
could be used for
inferring emotions
Time-Day Photo-day Photo-week
control condition Emotion reconstruction Emotion recognition
Wednesday, June 6, 12

35. Footprint tracker
How do visual cues (i.e., Sensecam), location cues, and context cues (SMS and calls
made or received) assist in reconstructing daily behaviors and experiences?
1. Sensecam
2. Location logging
3. Context logging
(SMS/calls made or received)
Wednesday, June 6, 12

36. Karapanos, E., Martens, J.-B., Hassenzahl, M. (2009) Reconstructing Experiences through Sketching. Arxiv preprint, arXiv:0912.5343.
Wednesday, June 6, 12

37. Cross-­‐sec8onal Repeated
sampling
“Longitudinal”
paradigms
in
HCI
Longitudinal Retrospec8ve
Wednesday, June 6, 12

38. Karapanos, E., Martens, J.-B., Hassenzahl, M. (2009) Reconstructing Experiences through Sketching. Arxiv preprint, arXiv:0912.5343.
Wednesday, June 6, 12

39. Karapanos, E., Martens, J.-B., Hassenzahl, M. (2009) Reconstructing Experiences through Sketching. Arxiv preprint, arXiv:0912.5343.
Wednesday, June 6, 12

40. Constructive Value-Account Control (no-graphing)
Constructive iScale, but not the Value-Account, performed better than
control condition
•More experience reports
•With more details (references to temporal information, discrete events)
•Higher test-retest consistency of time estimation (i.e., when did
an experience take place)
•Higher test-retest consistency of graphed patterns (over Value-
Account)
33
Karapanos, E., Martens, J.-B., Hassenzahl, M. (2009) Reconstructing Experiences through Sketching. Arxiv preprint, arXiv:0912.5343.
Wednesday, June 6, 12

41. 436 Studies in Computational Intelligence 436
The series Studies in Computational Intelligence (SCI) publishes new developments
Karapanos
and advances in the various areas of computational intelligence – quickly and with
high quality. The intent is to cover the theory, applications, and design methods
of computational intelligence, as embedded in the fields of engineering, computer
science, physics and life sciences, as well as the methodologies behind them.
The series contains monographs, lecture notes and edited volumes in computational
intelligence spanning the areas of neural networks, connectionist systems, genetic
algorithms, evolutionary computation, artificial intelligence, cellular automata,
self-organizing systems, soft computing, fuzzy systems, hybrid intelligent, and
virtual reality systems. Of particular value to both the contributors and the
Coming!
readership are the short publication timeframe and the world-wide distribution,
which enable both wide and rapid dissemination of research output.
Over the past decade the field of Human-Computer Interaction has evolved from the
study of the usability of interactive products towards a more holistic understanding
Evangelos Karapanos
of how they may mediate desired human experiences.
This book identifies the notion of diversity in users? experiences with interactive
1
products and proposes methods and tools for modeling this along two levels:
Modeling Users'
Modeling Users' Experiences with Interactive Systems
(a) interpersonal diversity in users? responses to early conceptual designs, and
(b) the dynamics of users? experiences over time.
The Repertory Grid Technique is proposed as an alternative to standardized
June 2012
psychometric scales for modeling interpersonal diversity in users? responses to early
concepts in the design process, and new Multi-Dimensional Scaling procedures are
Experiences with
Interactive Systems
introduced for modeling such complex quantitative data.
iScale, a tool for the retrospective assessment of users? experiences over time is
proposed as an alternative to longitudinal field studies, and a semi-automated
technique for the analysis of the elicited experience narratives is introduced. Through
Foreword: Jean-Bernard Martens
these two methodological contributions, this book argues against averaging in the
subjective evaluation of interactive products. It proposes the development of
interactive tools that can assist designers in moving across multiple levels of
Closing note: Marc Hassenzahl
abstraction of empirical data, as design-relevant knowledge might be found on
all these levels.
Foreword by Jean-Bernard Martens and Closing Note by Marc Hassenzahl.
issn 1860-949X
isbn 978-3-642-30999-1
9 783642 309991
springer.com
13
Wednesday, June 6, 12

42. Can we really trust 6-month old
memories?
• Validity? i.e. do memories reflect what we really
experienced?
• Reliability? i.e. in a second trial, will we recall
the same experiences?
Wednesday, June 6, 12

43. Memories are (sometimes)
more important than experiences
• Memories define how you evaluate your
past and how you decide on your future
• What do we measure for?
• Why do people drive irresponsibly?
• Why do people recommend their
products
Wednesday, June 6, 12