A presentation at the 2021 International Conference on Human-Computer Interaction (HCII) in the Session on "Information Design"

1. Constantinos K. Coursaris, Ph. D
Pierre-Majorique Léger, Ph.D.
Tech3Lab, HEC Montréal
Antoine Falconnet,
candidat à la M.Sc.
Tech3Lab, HEC Montréal
Corresponding author:
Constantinos.Coursaris@hec.ca
2021 HCI International, July 24-29
Improving User Experience
Through Recommendation
Message Design
A SYSTEMATIC LITERATURE REVIEW OF EXTANT LITERATURE
ON RECOMMENDER SYSTEMS AND MESSAGE DESIGN
Joerg Beringer
Blue Yonder, Coppell, TX

2. To further explore the antecedents
to effective recommender message design
by developing the theoretical grounding
of related concepts
RESEARCH MOTIVATION

3. © copyright Falconnet et al., 2021
Research questions
RQ1
What comprises the current
knowledge base of the antecedents
to effective RS message design?
RQ2
What statistically significant
results from past research can
inform current scholars and prac-
titioners of optimal RS message
design practices?
RQ3
What are the opportunities for
future research subsequently
potentially revealing guidelines
on how to optimize RS message
design in a managerial decision-
making context?

4. © copyright Falconnet et al., 2021
Research methodology (1/3)
A SYSTEMATIC LITERATURE REVIEW IN 4 STEPS
Searching for literature
in scientific databases
experiencing our
interface?
Reviewing and assessing
the search results
Analyzing
and synthesizing
the results
Reporting
the review
STEP
1
STEP
2
STEP
3
STEP
4

5. © copyright Falconnet et al., 2021
Research methodology (2/3)
INTERNATIONAL
DATABASES
Google Scholar, ABI/INFORM, ACM Digital Library,
ScienceDirect, SpringerLink, Information Systems
Journal, Information Systems Research, Journal of
Information Technology, Management Information
Systems Quarterly, Journal of Management Information
Systems
Recommender/recommendation systems, recommen-
der /recommendation system message, recommender/
recommendation systems user acceptance, recom-
mendation design, recommendation message design,
message design, message design acceptance, message
design guidelines, warning message design, persuasive
message design, information systems message design,
trust in recommender/recommendation systems, expla-
nation in recommender/recommendation systems.
2010-2020
Include papers about or published in...
▪ RS user acceptance
▪ RSs user-centric studies
▪ Message design
▪ Peer-reviewed conferences,
workshop, and journals
▪ English
▪ A year between 2010 to 2020
Exclude papers...
▪ Not addressing RS or message design
▪ Papers addressing RSs but centered
on methods and techniques
(algorithm, elicitation recommendation,
RS types, data mining etc.)
▪ Without empirical evidence
TIMEFRAME
KEYWORDS

6. © copyright Falconnet et al., 2021
Research methodology (3/3)
Citations (numerous and varied);
Clear and detailed presentation of the results and their
implication and contribution to the field;
Brings new knowledge and/or proposes relevant future research
to be carried out.
1
2
3
QUALITY CRITERIA RESULTS
132
papers collected
and analyzed
—
41
papers
preserved

7. © copyright Falconnet et al., 2021
Results – RQ1
RQ1 WHAT COMPRISES THE CURRENT KNOWLEDGE BASE
OF THE ANTECEDENTS TO EFFECTIVE RS MESSAGE DESIGN?
1
USERS PERCEPTIONS
AND BEHAVIORS
OF RECOMMENDER
SYSTEMS
▪ RS transparency
▪ Perceived usefulness
▪ Perceived ease of use
▪ Control
2
INFORMATION
MESSAGE
▪ Content more important
than the execution
3
RECOMMENDATION
MESSAGE CONTEN
AND FORMAT
▪ Contribute to user trust
▪ Impact the recommen-
dation’s acceptance
4
EXPLANATIONS IN
RECOMMENDATION
MESSAGE
▪ Lead to effective
recommendation
message

8. © copyright Falconnet et al., 2021
Results – RQ1
RQ1 WHAT COMPRISES THE CURRENT KNOWLEDGE BASE
OF THE ANTECEDENTS TO EFFECTIVE RS MESSAGE DESIGN?
RS transparency and perceived usefulness influence user’s trust and confidence, which is linked to the pur-
chase intention (Pu et al., 2011)
Control, perceived usefulness and perceived ease of use affect the user’s overall satisfaction, which is lin-
ked to the use intentions (Pu et al., 2011)
To create a transparent system, the recommendation must be accurate and contain explanations. The user feels
control if the interaction is adequate. The RS is perceived as easy to use due to an adequate interface and per-
ceived as useful if the recommendations are accurate, novel, diversify and contained sufficient information.
Message content is more important than its actual execution -> the information in each message will have to be
structured and adapted to the needs of the intended receivers and everything who will impair the understan-
ding of the recommendation system message must be banned (Pettersson, 2010, 2012 and 2014).
Effective recommendation messages are credible, transparent, accessible, accurate, valuable, contains suf-
ficient information, and contribute to user trust in the message, which drives the recommendation’s accep-
tance, and these factors can affect each others
The content and the format of recommendations have significant and varied impact on users’ evaluations of
recommendation message, therefore influence the user’s decision-making process.
User perceptions and behaviors
of Recommender Systems
Pu et al. (2011)
Information message
Pettersson (2010, 2012, 2014)
Recommendation message format
and content
Yoo and Gretzel (2012), Guanawardana &
Shani (2015), Ozok et al. (2010), Jameson et
al. (2015), Paniello et al. (2016)
Jameson et al. (2015)
Paniello et al. (2016)

9. © copyright Falconnet et al., 2021
Results – RQ1
Explanations in recommendation message
RQ1 WHAT COMPRISES THE CURRENT KNOWLEDGE BASE
OF THE ANTECEDENTS TO EFFECTIVE RS MESSAGE DESIGN?
Kunkel et al. (2019) The richness of explanations plays a pivotal role in trust-building processes and recommendation should be incorporate
explanatory component that imitate more closely the way humans exchange information. Recommend to combinate
different explanation styles in the recommendation, which lead to explanations with a higher perceived value and trust in.
McIenerney et al. (2018) But users respond to explanations differently according to their context and intent, so there is a need to jointly
optimize both recommendation (i.e., the solution) and explanation selection.
Nunes and Jannach (2017) Personalized explanations are often linked to improved transparency, persuasiveness and satisfaction,
when compared to non-personalized explanations.
Al-Taie and Kadry (2014) Long and strongly confident explanations can be more effective in the acceptance of interval forecasts.
Yoo and Gretzel (2012) Generating familiar recommendation with detailed information and explanations regarding the underlying logic
of how the recommendation was generated increase the users’ perceived credibility of the system.
Tintarev and Masthoff (2012)
Friedrich and Zanker (2011)
Explanations leads to satisfaction, transparency, confidence, perceived ease of use, perceived usefulness and
effective recommendation
Tintarev and Masthoff (2012)
Gedikli and Jannach (2011)
They can increase user acceptance of RS, help users make decisions more quickly, convince them to accept
the recommendation and develop users’ trust in the system as a whole
Tintarev and Masthoff (2011) Moreover, users have a preference for knowledgeable explanations and the recommender may be formulated along
the line of “You might (not) like Item A because...”
Friedrich and Zanker (2011) Encourage RS designers to excluded from an explanation all information and knowledges that are not relevant
for answering a request and structural characteristics can be used as additional dimensions.

10. © copyright Falconnet et al., 2021
Results – RQ2
RQ2 WHAT STATISTICALLY SIGNIFICANT RESULTS FROM PAST RESEARCH CAN INFORM
CURRENT SCHOLARS AND PRACTITIONERS OF OPTIMAL RS MESSAGE DESIGN PRACTICES?
1
ResQue MODEL &
COURSARISET AL. (2020)
REC. MSG. DESIGN STUDY
▪ Information specificity
▪ Information sufficiency
2
KNOWLEDGEABLE
EXPLANATIONS & DESCRIPTIONS
WITH A HIGH LEVEL OF DETAIL
AND STRONG INFORMATION
SCENT
▪ Strengthen persuasion
▪ Increase perceived
usefulness
3
TRANSPARENT RA
SOCIAL USER’S INFORMATION
FACT-BASED EXPLANATIONS
▪ Influence the intention
to use the system

11. © copyright Falconnet et al., 2021
Results – RQ2 RQ2 WHAT STATISTICALLY SIGNIFICANT RESULTS FROM PAST RESEARCH CAN INFORM
CURRENT SCHOLARS AND PRACTITIONERS OF OPTIMAL RS MESSAGE DESIGN PRACTICES?
Coursaris et al. (2020) Recommendation’s information specificity impacted information sufficiency and information transparency. Information
sufficiency (i.e., both problem and solution) influence the perceived usefulness and information transparency positi-
vely impacted recommendation confidence and perceived ease of use. Also, a higher specificity of the problem and the
solution information increase the user’s recommendation acceptance and the decision-making time while information
sequence problem to solution reduce the decision-making time.
Bigras et al.’s study (2019) Participants perceived RA credibility, decision quality, and satisfaction were positively affected by a transparent
RA and were not impacted by the cognitive effort needed to access and understand the explanations of a
transparent RA.
Schnabel et al. (2018) People have a preference for descriptions with a higher level of detail and strong information scent.
Berkovsky et al. (2017) The type of explanation to be used in a music RS depends on the desired effect on the user. persuasive explanation
is suited to support the competence facet of the RS, while displaying IMDb score will promote the honesty and
objectivity of the recommender.
Oechslein et al. (2014) Integration of social user’s information could improve the intention to use a recommender system.
Zanker and Schoberegger’s (2014) Indicate that fact-based explanations have a stronger impact on participants preference stability than
sentence-based explanations.
Zanker (2012) knowledgeable explanations significantly increase the perceived usefulness of a recommender system
Pu et al. (2011) Proposed the ResQue model after conducting a user-centric study

12. © copyright Falconnet et al., 2021
Results – RQ3
RQ3 WHAT ARE OPPORTUNITIES FOR FUTURE RESEARCH SUBSEQUENTLY POTENTIALLY REVEALING
GUIDELINES ON HOW TO OPTIMIZE RS MESSAGE DESIGN IN A MANAGERIAL DECISION-MAKING CONTEXT?
1
INFORMATION
MESSAGES
▪ Affective language
▪ Typographical cues
▪ Subjective language
▪ Explanation’s length
2
SOFTWARE UPDATE
WARNING MESSAGE
▪ Good design
▪ Scare tactics
▪ Simple language
3
NARRATIVE
MESSAGES
▪ More clear, enjoyable,
and involving
▪ Positive emotional and
cognitive responses
▪ Greater belief in claims

13. © copyright Falconnet et al., 2021
Results – RQ3 RQ3 WHAT ARE OPPORTUNITIES FOR FUTURE RESEARCH SUBSEQUENTLY POTENTIALLY
REVEALING GUIDELINES ON HOW TO OPTIMIZE RS MESSAGE DESIGN IN A MANAGERIAL
DECISION-MAKING CONTEXT?
Message content and format:
▪ Affective language in eliciting positive affective
responses from users (Makkan et al., 2020)
‒ Apologies, reward or praise (Li et al., 2017)
▪ Typographical cues in contexts other than scientific
Q&A forums (Zhang et al. 2019)
‒ Bold vs. Plain test in decision-making context
did not impact users’ perceptions and behaviors
(Coursaris et al., 2020)
‒ Italics, underlying, colour, etc.
▪ Multiple factors to reveal optimal combination(s)
‒ E.G. Problem information specificity, Solution
information specificity, and Information sequence
(Coursaris et al., 2020)
▪ Subjective vs. objective language on users’
perceptions of system, recommendation, intent
for acceptance and future usage.
‒ Tested for Virtual Agents (Matsui & Yamada, 2019)
▪ Length and details/precision of the
recommendation message
‒ Long explanations are more persuasive than short
explanations (Al-Taie and Kadry, 2014; Nunes &
Janach, 2017)
‒ Short and precise recommendations (Schreiner et al.,
2019; Harbach et al., 2013; Bavro-Lillo et al., 2011)

14. © copyright Falconnet et al., 2021
Results – RQ3 RQ3 WHAT ARE OPPORTUNITIES FOR FUTURE RESEARCH SUBSEQUENTLY POTENTIALLY
REVEALING GUIDELINES ON HOW TO OPTIMIZE RS MESSAGE DESIGN IN A MANAGERIAL
DECISION-MAKING CONTEXT?
Software update warning messages:
▪ Good design reduces hesitation to apply updates
▪ Better designs alleviates annoyance and confusion
while increasing importance and noticeability of
updates (Fagan et al. 2015)
‒ Good design reduces hesitation to apply updates
‒ Better designs alleviates annoyance and confu-
sion while increasing importance and noticeability of
updates (Fagan et al. 2015)
‒ More concise
‒ State what is being updated
‒ Use simple language
‒ Explain the reason(s) behind the update
‒ Explain the benefits
‒ Well-designed or placed buttons boost usability,
reduce confusion
‒ Short and precise sentences (Harbach et al., 2013)
‒ Avoid scare tactics (higher annoyance)
‒ Discourage users to read them and word defined as
hard have technical background or refer to unclear
concept
‒ Less technical
Discourages users to read them (Harbach et al., 2013))

15. © copyright Falconnet et al., 2021
Results – RQ3 RQ3 WHAT ARE OPPORTUNITIES FOR FUTURE RESEARCH SUBSEQUENTLY POTENTIALLY
REVEALING GUIDELINES ON HOW TO OPTIMIZE RS MESSAGE DESIGN IN A MANAGERIAL
DECISION-MAKING CONTEXT?
Narrative message
▪ More enjoyable and produce character involvement
(Moyé-Gusé et al., 2011; Moyé-Gusé and Nabi, 2011)
▪ Reduce or circumvent negative reactions to persua-
sion and counterarguing (Niederdeppe et al., 2011)
▪ Evoke emotional and cognitive responses
(Appel & Richter, 2010)
▪ Encourage a greater belief in the realism of claims
or the authenticity of the narrative world through
a suspension of disbelief (Weber & Wirth, 2014).
▪ More persuasive when the legitimacy burden is
greater because they mask the persuasive intent
(Niederdeppe et al., 2014; 2011).
▪ Convey information in ways that may reduce feelings
of being overloaded (Jensen et al., 2013).
▪ More explicit story structure, more understandable
and involves less information overload
(Barbour et al., 2015)

16. © copyright Falconnet et al., 2021
Results – RQ3
RQ3 WHAT ARE OPPORTUNITIES FOR
FUTURE RESEARCH SUBSEQUENTLY
POTENTIALLY REVEALING GUIDELINES
ON HOW TO OPTIMIZE RECOMMENDER
SYSTEM MESSAGE DESIGN IN A
MANAGERIAL DECISION-MAKING
CONTEXT?
Table: Concept matrix: opportunities for future research on how to optimize
RS message design in a managerial decision-making context.

17. Informed state-of-art of the current knowledge base
of the antecedents to effective RS message design
—
Compilation of guidelines and best practice from
significant past research for researchers and designers
—
Next steps: Conduct studies on identified opportunities
CONCLUSION

18. © copyright Falconnet et al., 2021
Thank you!