This paper explores the convergent validity of instruments that can provide higher temporal resolution when measuring user experience: the continuous self-perceived measurement system and psychophysiological measures. Specifically, we explore the extent to which primacy and recency effects may have an impact on the convergent validity of two constructs: valence and arousal. Using a Wilcoxon signed rank test, results suggest that users self-evaluate their valence more accurately at the end of each of the sequences than at the beginning while they evaluate their arousal more accurately at the beginning of each of the sequences. This suggests that the recency effect has more impact on valence and the primacy effect has more impact on arousal. These findings contribute to human-computer interaction research by providing more information about the psychophysiological measures that cause recency and primacy. https://link.springer.com/chapter/10.1007/978-3-319-91716-0_11

1. Testing the Convergent Validity
of Continuous Self-Perceived
Measurement Systems
An Exploratory Study
HCI International 2018, Las Vegas (USA), July 15th
- 20th
Sébastien Lourties
Pierre-Majorique Léger Ph.D.
Sylvain Sénécal Ph.D.
Marc Fredette Ph.D.
Shang Lin Chen
HEC Montréal

2. Research question
Are primacy and recency effects having
an impact on the convergent validity of
two constructs: valence and arousal?
Measure :
▪▪ Psychophysiological measures
(arousal and valence)
▪▪ Continuous self-perceived measure
(arousal and valence)
Contribution:
▪▪ Psychophysiological measures that
cause recency and primacy.
▪▪ An overview of continuous self-
perceived measurement validity

3. Girard JM, Aidan AG: DARMA: Software for dual axis rating and media annotation. Behavior research methods 1–8 (2017).
Bradley M, Lang PJ: Measuring Emotion: The Self-Assessment Semantic Differential Manikin and the Semantic
Differential. Journal of Behavioral Therapy and Experimental Psychiatry, 25(1), 49-59 (1994).
Measuring UX
with high tem-
poral resolution
instruments
Continuous self-perceived
measures of user experience
Self-reported scales (global retros-
pective evaluation): Sam Scale
Continuous self-perceived measure
(continuous retrospective evaluation):
DARMA
Advantage:
providing richer information

4. Measuring UX with high temporal
resolution instruments
Continuous physiological measures of user experience
Boucsein W: Electrodermal Activity. Springer US (2012).
Dirican AC, Göktürk M: Psychophysiological measures of human cognitive states applied in Human Computer
Interaction. Procedia Computer Science 3:1361–1367 (2011).
Arousal:
The arousal construct is used
to contrast states of low arousal
(e.g., calm) and high arousal
(e.g., surprise): electrodermal
activity (EDA)
Valence:
contrast states of pleasure
(e.g., happy) and displeasure
(e.g., angry): Automatic facial
analysis
Advantages:
an unobtrusive and implicit way to determine the user’s affective
or cognitive state on the basis of mindbody relations” (p.1362).

5. Hypothesis development
Primacy:
increased memory and over-
weighted influence of the first
moment of an experience
Recency:
the important influence of the last
moment of an experience
Retrospection:
the capacity to remember the
context of an experience
H1
Primacy and recency effect have
the same influence on the accu-
racy of self-reported evaluation
H2
Arousal has more influence than
valence on the recency effect
Murdoch BB, Lissner E, Marvin C: The serial position effect of free recall Journal of experimental psychology 64(5), 482-488 (1962).
Shteingart H, Neiman T, Loewenstein Y: The role of first impression in operant learning. Journal of Experimental Psychology: General, 142(2), 476-488 (2013).
Kahneman D, Fredrickson BL, Schreiber C a., Redelmeier D: When More Pain Is Preferred to Less: Adding
a Better End. Psychological Science 4(6), 401-405 (1993).
Craik FIM, Lockhart RS: Levels of Processing : A Framework for Memory Research. Journal of Verbal Learning
and Verbal Behavior 11(6), 671–684 (1972).
Kahneman D, Fredrickson BL, Schreiber C a., Redelmeier D: When More Pain Is Preferred to Less: Adding a
Better End. Psychological Science 4(6), 401-405(1993).
Primacy Recency
Memory
capacity
Time

6. Research method
▪▪ Participants: 13 (7 males) mean age of 32,
compensation of 100$
▪▪ Each participant had to report their
emotional valence and arousal for
14 short sequences (log in, book an
appointment) for a total of 182 sequences
for the overall sample.

7. Research method
Instruments and measures
▪▪ Acknowledge software mp150 sam-
pled at 500Hz (BIOPAC, Goleta, USA)
▪▪ The FaceReader software
(Noldus, Wageningen, Netherlands)
▪▪ Media Recorder (Noldus, Wageningen,
Netherlands)
▪▪ Observer XT (Noldus, Wageningen,
Netherlands)
▪▪ DARMA software set at 30 Hz,
a bin size of 0.25 seconds
Self reported
(retrospective)
Physiological
(during interaction)
Valence
Participant’s joystick position
on the valence axis at time t
(Darma)
Facial emotional valence at time t
(Facereader)
Arousal
Participant’s joystick position
on the arousal axis at time t
(Darma)
Electrodermal activity for time t
(Acqknowledge)

8. Results
▪▪ Participants report with more
precision their arousal at the
beginning
▪▪ Participants report more accu-
rately their valence at the end
of the task Test: Wilcoxon signed rank test
Pair of variables
(first and last 1s)
Mean
during first
impression
Mean
during last
impression
P value
distance_valence_first distance_valence_last .155 .1215 .0173
distance_arousal_first distance_arousal_last .0191 .0377 .0635

9. Discussion
H1: accepted:
they both biased the
retrospection
H2: rejected:
arousal should have been
best reported to influence
recency effect
For researcher:
▪▪ Using CSP, arousal is report
more accurately at the begin-
ning of each of the sequences
than at the end which contra-
dict researcher theories about
the peak-end rule
▪▪ Using CSP will not have the
same results as physiological
measure
For HCI industry:
▪▪ Using CSP will not have the
same results as physiological
measure
▪▪ Minimizing negative emotion
and maximizing positive at the
end of an experience
▪▪ Minimizing or maximizing
arousal at the beggining of the
experiment depending your the
area of activity

10. Conclusion
▪▪ Limits : Limited sample, utilitarian
context
▪▪ Emotion related experience are always
subject to bias.
▪▪ This give more support to physiological
measures to evaluate them
▪▪ Primacy and recency through the
influence of valence and arousal play an
important role is the experience that is
reported.

11. Thank you!