In this conceptual paper, we present a UX process reference model (UXPRM), explain how it builds on the related work and report our experience using it. The UXPRM includes a description of primary UX lifecycle processes, and a classification of UX methods and artifacts. This work draws an accurate picture of UX base practices and allows the reader to compare and select methods for different purposes. Building on that basis, our future work consists of developing a UX Capability/Maturity Model (UXCMM) intended for UX activity planning according to the organization’s UX capabilities. Ultimately, the UXCMM aims to facilitate the integration of UX processes in software engineering, which should contribute to reducing the gap between UX research and UX practice.

1. Specification of a UX Process Reference Model
towards the Strategic Planning of UX Activities
Suzanne Kieffer, Luka Rukonic, Vincent Kervyn de Meerendré, Jean Vanderdonckt

2. Part 1 – UX Process Reference Model
 Conceptual contribution

3. UX lifecycle Supporting UX methods/artifacts
Proposed UX Process Reference Model (UXPRM)

4. UX lifecycle, processes, outcomes and their interrelations
HUCAPP 2019 -- Kieffer et al. 2019 4
Processes
Outcome
Outcome

5. 5
to render a
first account
of the user
requirements
HUCAPP 2019 -- Kieffer et al. 2019

6. 6
to provide the
development
team with a
model to
follow during
coding
HUCAPP 2019 -- Kieffer et al. 2019

7. 7
to turn
design ideas
into testable
mockups
HUCAPP 2019 -- Kieffer et al. 2019

8. 8
to detect and
fix UX flaws
HUCAPP 2019 -- Kieffer et al. 2019
no

9. 9
to detect and
fix UX flaws
HUCAPP 2019 -- Kieffer et al. 2019
yes

10. 10
to check whether
users can use the
product successfully
HUCAPP 2019 -- Kieffer et al. 2019
no

11. 11
to check whether
users can use the
product successfully
HUCAPP 2019 -- Kieffer et al. 2019
yes

12. 12
Iterative and incremental
specification of user requirements
HUCAPP 2019 -- Kieffer et al. 2019

13. 13
Iterative and incremental
release of product
HUCAPP 2019 -- Kieffer et al. 2019

14. Supporting UX methods and artifacts
HUCAPP 2019 -- Kieffer et al. 2019 14

15. Behavioral and/or physiologic methods (what users do)
Methods Techniques Objectives UX activities
automated
experience
sampling
automated interaction
logs
to gain insights into the user experience
with a system based on automatic
logging of user actions
job/task analysis;
contextual inquiry; user
research; UX evaluation
constructive collage; drawings;
photographs; probes
to identify unexpected uses of a system
or concept
formative UX evaluation
experiment A/B testing; controlled
experiment; remote
experiment
to support, refute, or validate a
hypothesis about sample population,
task, system; to establish cause-and-
effect relationships
job/task analysis; user
research; UX evaluation
instrument-
based
experiment
experiment with
instrument (e.g. eye
tracker)
to gain insights into user behavioral,
emotional and physiologic responses
with a system (e.g. gaze, stress, etc.)
cognitive task analysis; UX
evaluation
observation field observation to identify how users perform tasks or
solve problems in their natural setting
contextual inquiry; user
research; UX evaluation
simulation paper-and-pencil
evaluation; Wizard of
Oz experiment
to detect UX problems; to identify the
use and effectiveness of a system which
has not been implemented yet
formative UX evaluation
15HUCAPP 2019 -- Kieffer et al. 2019

16. Artifacts about user needs
Artifact Objectives UX activities
customer journey
map
to depict key interactions users have with the system
over time (i.e., touch-points); to map touchpoints with
user thoughts, feelings and emotional responses
specification of the
context of use
service blueprint to depict relationships between different service
components (front-end, back-end and organizational
processes) that are directly tied to touchpoints in a
specific customer journey
specification of the
context of use
persona to depict key user profiles (personality, roles, goals and
motivations, frustrations, etc.)
specification of the
context of use
work model to depict the current work organization of users; to
depict intents, triggers, breakdowns in the tasks
(problems, user errors and workarounds)
specification of the
context of use
UX goals to establish specific qualitative and quantitative UX
goals that will drive UX design
UX goals setting
16HUCAPP 2019 -- Kieffer et al. 2019

17. Conceptual contribution
• Specification of a complete UXPRM: description of primary UX lifecycle
processes and UX methods/artifacts that serve as UX base practice
17HUCAPP 2019 -- Kieffer et al. 2019
4 processes
2 outcomes
their interrelations
5 families of methods
30 methods/artifacts
70 techniques
Usability Engineering
Human-Centered Design
User-Centered Design
Agile User-Centered Design

18. Identification and classification of UX methods/artifacts
HUCAPP 2019 -- Kieffer et al. 2019 18
Artifact-mediated communication methods
Knowledge elicitation methods with users
Traditional HCI/usability evaluation methods: survey research, interview, observation and experimentation
Usability Engineering methods: writing personas, work/task modelling and prototyping
UX evaluation techniques: questionnaires, probes and cards

19. Part 2 – Strategic planning of UX activities
 Methodological contribution

20. Which methods/artifacts will produce the highest UX ROI?
HUCAPP 2019 -- Kieffer et al. 2019 20

21. Use of the UXPRM in two industrial projects
• Two projects funded by Service Public de Wallonie (SPW)
• HAULOGY (2018-2020, convention n° 7767)
• VIADUCT (2018-2021, convention n° 7982)
• To partly assess the UX capability/maturity of our industrial partners
• Frequency of use of UX methods/artifacts  Achievement of UX processes
• Literacy about UX methods/artifacts  UX literacy
• UX staff, budget, equipment, infrastructure  UX resources
• Attitude towards users/UX  UX culture
HUCAPP 2019 -- Kieffer et al. 2019 21

22. Research hypothesis: aligning UX activities with the UX
capability/maturity of our partners should help to maximize the UX ROI
HUCAPP 2019 -- Kieffer et al. 2019 22

23. Method used to carry out UX capability/maturity assessment
• Survey research in two rounds
1. Questionnaire about UX methods and
artifacts  UX processes and literacy
2. Interview about UX resources and UX
culture
HUCAPP 2019 -- Kieffer et al. 2019 23

24. Methodological contribution
• Rely on partners’ data to make UX activity plans (UX processes)
• Data: identification of UX methods/artifacts already integrated into their current
software development model (e.g. hi-fi prototypes and user stories)
• Result: selection of UX methods/artifacts that will maximize the UX ROI (e.g. lo-fi
prototypes and user scenarios)
• Expand partners’ understanding of UX (UX literacy)
• Dispel mistaken beliefs about UX (e.g. user requirements analysis is the same as
functional analysis, UX is another term for UI, late design changes are not a big deal)
• Explain UX ROI: from better products to increased organizational efficiencies
HUCAPP 2019 -- Kieffer et al. 2019 24

25. Take-away
• Uses of the UXPRM
• As a general roadmap that can be followed during product development lifecycle
• As a catalog of UX methods/artifacts that can be used during a project
• As a supporting tool for assessing UX capability/maturity
• Purposes
• To increase UX literacy
• To align UX activity plans with UX capability/maturity
• To identify/anticipate potential barriers to UX activities (conflict with current
software development model, difficulty to involve users, not time for analysis…)
HUCAPP 2019 -- Kieffer et al. 2019 25

26. 26HUCAPP 2019 -- Kieffer et al. 2019

28. Knowledge elicitation methods without users
Methods Techniques Objectives UX activities
GOMS GOMS CMN-GOMS,
CPM-GOMS, NGOMSL,
Keystroke-Level Model
to produce quantitative and
qualitative predictions of how people
will use a proposed system
UX evaluation
hierarchical
task analysis
hierarchical task analysis to identify the cognitive skills, or
mental demands, needed to perform
a task proficiently
cognitive task
analysis
inspection cognitive walkthrough;
design or expert review;
heuristic evaluation
to predict the learnability of a
system; to predict usability and UX
problems
UX evaluation
literature
review
(systematic) literature
review; systematic map-
ping
to locate, analyze, synthetize relevant
published and/or unpublished work
about a topic; to understand the
current thinking and the state of the
marketplace about a topic
context-of-use,
stakeholder
analysis; user
research
28HUCAPP 2019 -- Kieffer et al. 2019

29. Knowledge elicitation methods with users
Attitudinal methods (how users feel)
Methods Techniques Objectives UX activities
cards cards; emocards;
emotion cards
to identify user mood and reactions
about their interaction with a system
UX evaluation
retrospective
interview
cognitive or
elicitation interview
to gain insights into particular
aspects of cognitive performance
during user past experience with a
system
cognitive task analysis;
contextual inquiry; UX
evaluation
survey interview;
questionnaire
to assess thoughts, opinions, and
feelings of a sample population
about a system
user research; UX
evaluation
think-aloud co-discovery; talk-
aloud protocol;
(retrospective)
think-aloud protocol
to gain insights into the participant’s
cognitive processes (rather than only
their final product); to make thought
processes as explicit as possible
during task performance
job/task analysis;
contextual inquiry; user
research; UX evaluation
Additional methods: experience sampling, group interview and prospective interview
29HUCAPP 2019 -- Kieffer et al. 2019

30. Artifact-mediated communication methods
About product design and evaluation
Artifact Objectives UX activities
affinity diagram to organize and cluster user data (typically from contextual
inquiry or brainstorming) based on their natural relationships
design ideation
card sort: closed or
open card sort
to organize and label topics into categories that make sense to
users
IA design; UX evaluation
task model to describe the tasks that the user and the system carry out to
achieve user goals; to review relationships between tasks
UX design
lo-fi prototype:
paper, sketch,
wireframe or video
to turn design ideas into testable mock-ups; to test-and-refine
design ideas; to fix UX problems early in the product lifecycle
UX design; formative UX
testing
hi-fi prototype:
coded, wireframe
or WOz
to turn mockups into highly-functional and interactive
prototypes; to evaluate how well the prototype meets UX
requirements
summative UX evaluation
general design
principles
to arrange screens in such a way that they are aesthetic and
consistent and communicate ideas clearly (color schemes;
fonts; interactors; semiotics)
graphic and/or visual
design
Additional artifacts: concept map, user scenario, user story and epic
30HUCAPP 2019 -- Kieffer et al. 2019

31. 31
• Similarities
• Iterative
• Terminology
• Sequence of processes
• Main difference about UR
• UE, HCD, UCD: UR is a process
• AUCDI: UR is an outcome

32. How does the UX community conduct UX evaluation?
• Traditional HCI/usability evaluation methods such as survey research,
interview, observation and experimentation (Bargas-Avila and Hornbæk,
2011; Daae and Boks, 2015; Gray, 2016; Roedl and Stolterman, 2013;
Vermeeren et al., 2010)
• Questionnaire: technique supporting UX data collection (Bargas-Avila and
Hornbæk, 2011; Law et al., 2014; Venturi et al., 2006)
HUCAPP 2019 -- Kieffer et al. 2019 32

33. Whether UX measures should integrate usability is a question that
divides the UX community (Law et al., 2008, 2014)
• Constructs of UX
• Pragmatic characteristics of the interactive product: usability, utility or safety
• Hedonic aspects of UX: visual aesthetics, beauty, joy of use or personal growth
• Approach for UX measurement
• Traditional HCI approach focused on task-oriented, instrumental goals in which UX is
a variation of the satisfaction construct of usability
• New paradigm in HCI focused on non-task oriented, non-instrumental goals in which
UX is a set of hedonic qualities different from usability
HUCAPP 2019 -- Kieffer et al. 2019 33

34. Whether UX measurement should follow a qualitative or a
quantitative approach is YET another question that divides the UX
community
• Bargas-Avila and Hornbæk (2011)
• 50% qualitative
• 33% quantitative
• 17% both
• Lallemand et al. (2015)
• Replication of Law et al. (2009)
• No clear answer on respondents’ attitude towards UX measurement
• Higher preference for qualitative approaches in industry
HUCAPP 2019 -- Kieffer et al. 2019 34