Slides from my keynote talk at the Digital Marketing Colloquium 2024, at the University of Bournemouth.

1. AI-Powered
Service
Interactions
Researching and shaping the
future of
Professor Ana Isabel Canhoto
University of Sussex
@canhoto
(c) Ana Isabel Canhoto www.anacanhoto.com 1

2. My colleague’s
question:
Is it possible that our
research project will
actually make life more
difficult for customers, or
create problems for
customer service staff? At
the end of the day, it’s
people we are talking
about, here.
(c) Ana Isabel Canhoto www.anacanhoto.com 2

3. The benefits: Highly personalised,
cost-efficient, and seamless service
experiences, through the the use of
AI technologies such as chatbots or
predictive analytics
(c) Ana Isabel Canhoto www.anacanhoto.com 3

4. Biased datasets
(c) Ana Isabel Canhoto www.anacanhoto.com 4
More: https://www.starlingbank.com/docs/reports-research/StarlingGenderRepresentationReport.pdf
e
f women
highlighted in
en as childlike,
dealing with their
ll the searches
ayed women
small amounts
d carefully or
the other hand,
cket pockets.
ey to be treasured,
women. In
showing men
ot only signals
ey as a very
Women pop pounds and pennies into
piggy banks while men stash wads of
cash in jacket pockets
Calculator
8%
Mobile
phone
Computer,
Tablet
Total Men & Money
Total Women & Money
In contrast, women were more likely than men to be
shown in a leisure setting. Twice the proportion of
women as men were pictured with a hot drink, either
in a café or at home. Women are more frequently
depicted next to pot plants than men, often with
flowers (if a man is pictured with a plant – it’s more
likely a cactus!). Professor Kanji told us that such
homely images further reinforce the messages of
fragility and nurturing – that women’s proper place
is in the home.
Women are more frequently depicted
next to pot plants than men
Summary Report: Gendered representations of money in visual media, a study 7
likely to have a visibly happy woman pictured (56%
women versus 40% men) – which could normalise
and sanitise indebtedness. The positive emotion
associated with women contrasts with the depiction
of men absorbing the strain. Only a few images show
women worrying about finance – and then mostly in
dramatic fashion, with comedically over-the-top facial
expressions commonplace.
Finally, the behaviour and role of men and women
in money images is intertwined with stereotypes.
Men are active decision makers and women are
passive onlookers, making basic decisions or no
decision at all.
Typically, men are shown paying bills, making
purchases, deals or even confidently flaunting their
money. Women’s behaviour is weighted towards
saving coins into a piggy bank.
In two of the three main image libraries, paper money
was missing from ‘women and money’ images and
the photographs in the image library that did picture
women with notes were comedic, with the women
sporting dramatic poses or shocked expressions.
In some cases, women simply watch on as a man
handles the finances. Sometimes they’re not even
engaged at all, suggesting ignorance or a lack of
interest in money matters.
Women are often delighted to secure
a loan, or patronisingly befuddled when
it comes to battling debt
Active vs passive money behaviour

5. (c) Ana Isabel Canhoto www.anacanhoto.com 5
edthree
npatterns
ionofwomen
themeshighlightedin
alofwomenaschildlike,
comestodealingwiththeir
nhalfofallthesearches
2%)portrayedwomen
ormen.
nhandlingsmallamounts
niesplacedcarefullyor
k.Menontheotherhand,
heirsuitjacketpockets.
cketmoneytobetreasured,
ealityforwomen.In
fimagesshowingmen
ofnotes,notonlysignals
ionsmoneyasavery
lives.
Womenpoppoundsandpenniesinto
piggybankswhilemenstashwadsof
cashinjacketpockets
Individual,
alone
56 1 14 4
69 3 3 0
e 75% 1% 19% 5%
e 92% 4% 4% 0%
Individual,
w/othersofthe
samegender
Individual,
w/othersofthe
differentgenders
Unclear*
seehands
entationsofmoneyinvisualmedia,astudy 5
Calculator
8%
Mobile
phone
Computer,
Tablet
TotalMen&Money
TotalWomen&Money
Incontrast,womenweremorelikelythanmentobe
showninaleisuresetting.Twicetheproportionof
womenasmenwerepicturedwithahotdrink,either
inacaféorathome.Womenaremorefrequently
depictednexttopotplantsthanmen,oftenwith
flowers(ifamanispicturedwithaplant–it’smore
likelyacactus!).ProfessorKanjitoldusthatsuch
homelyimagesfurtherreinforcethemessagesof
fragilityandnurturing–thatwomen’sproperplace
isinthehome.
Womenaremorefrequentlydepicted
nexttopotplantsthanmen
SummaryReport:Genderedrepresentationsofmoneyinvisualmedia,astudy 7
likelytohaveavisiblyhappywomanpictured(56%
womenversus40%men)–whichcouldnormalise
andsanitiseindebtedness. Thepositiveemotion
associatedwithwomencontrastswiththedepiction
ofmenabsorbingthestrain.Onlyafewimagesshow
womenworryingaboutfinance–andthenmostlyin
dramaticfashion,withcomedicallyover-the-topfacial
expressionscommonplace.
Finally,thebehaviourandroleofmenandwomen
inmoneyimagesisintertwinedwithstereotypes.
Menareactivedecisionmakersandwomenare
passiveonlookers,makingbasicdecisionsorno
decisionatall.
Typically,menareshownpayingbills,making
purchases,dealsorevenconfidentlyflauntingtheir
money.Women’sbehaviourisweightedtowards
savingcoinsintoapiggybank.
Intwoofthethreemainimagelibraries,papermoney
wasmissingfrom‘womenandmoney’imagesand
thephotographsintheimagelibrarythatdidpicture
womenwithnoteswerecomedic,withthewomen
sportingdramaticposesorshockedexpressions.
Insomecases,womensimplywatchonasaman
handlesthefinances.Sometimesthey’renoteven
engagedatall,suggestingignoranceoralackof
interestinmoneymatters.
Womenareoftendelightedtosecure
aloan,orpatronisinglybefuddledwhen
itcomestobattlingdebt
Activevspassivemoneybehaviour
SummaryReport:Genderedrepresentationsofmoneyinvisualmedia,astudy 6
Generative
AI
Source: https://www.jamaissanselles.fr/biais-intelligence-artificielle/

6. (c) Ana Isabel Canhoto www.anacanhoto.com 6
Larger datasets are not necessarily less biased
Source: Hofmann, V., Kalluri, P. R., Jurafsky, D., & King, S. (2024). Dialect prejudice predicts AI decisions about people's character,
employability, and criminality. arXiv preprint arXiv:2403.00742.
Hofmann et al (2024):

7. Our AI lives are ruled by a handful of datasets
• Developing training datasets is time- and labour-intensive, and requires
access to unique or privileged data (e.g., self-driving car logs)
• Koch et al (2021): Evidence of increasing concentration in fewer and fewer,
publicly available, training
datasets*, across various
task communities
*Often developed for a
different task
(c) Ana Isabel Canhoto www.anacanhoto.com 7
Source: Koch, B., Denton, E., Hanna, A., & Foster, J. G. (2021). Reduced,
reused and recycled: The life of a dataset in machine learning
research. arXiv preprint arXiv:2112.01716.

8. Rigidity of algorithms vs fluidity of human behaviour
- Report by the National Transportation
Safety Board re: the fatal crash by an
Uber self-driving vehicle:
- The car detected Herzberg with more than
enough time to stop.
- When the car first detected her presence, 5.6
seconds before impact, it classified her as a
vehicle, then other, then vehicle, then other,
then bicycle, then other, then back to bicycle.
- It never guessed Herzberg was on foot for a
simple, galling reason: Uber didn’t tell its car
to look for pedestrians outside of
crosswalks.
- “The system design did not include a
consideration for jaywalking pedestrians,”
(c) Ana Isabel Canhoto www.anacanhoto.com 8
Source: https://www.wired.com/story/ubers-self-driving-car-didnt-
know-pedestrians-could-jaywalk/

9. Hidden assumptions in algorithmic decision making
- Unless data are unambiguously
linked to the outcome, we need
to make assumptions
- Assumptions about the whole
become reality for the individual.
- E.g., A-levels algorithm*
- School’s past performance, class
size, …
(c) Ana Isabel Canhoto www.anacanhoto.com 9
Source:
https://assets.publishing.service.gov.uk/government/uploads/system/uplo
ads/attachment_data/file/909368/6656-
1_Awarding_GCSE__AS__A_level__advanced_extension_awards_and_exte
nded_project_qualifications_in_summer_2020_-_interim_report.pdf

10. Algorithms reflect biases in society
Hofmann et al (2024):
Dialect prejudice
predicts AI decisions
about people's
character, employability,
and criminality
(c) Ana Isabel Canhoto www.anacanhoto.com 10

11. Sensitivity to prompts
Krugel et al (2023):
Slight variations in
the phrasing of the
prompt produce
dramatically
different
interpretations of
the order, and
produce different
outcomes
(c) Ana Isabel Canhoto www.anacanhoto.com 11
Source: Krügel, S., Ostermaier, A., &
Uhl, M. (2023). ChatGPT’s inconsistent
moral advice influences users’
judgment. Scientific Reports, 13(1),
4569.

12. Concentration of AI investment
• AI is not open
• Cumulative market power (Keegan
et al. 2022)
• Performative power
• Claims about what AI can do
influences valuation and
stakeholders’ actions
• Sorting the wheat from the chaff
• AI vs data science (Keegan et al.
2022)
• Gen AI hype vs true innovation
(c) Ana Isabel Canhoto www.anacanhoto.com 12
Source: Keegan, B. J., Canhoto, A. I. & Yen, D. (2022). Power negotiation
on the tango dancefloor: The adoption of AI in B2B
marketing. Industrial Marketing Management, 100, 36-
48. https://doi.org/10.1016/j.indmarman.2021.11.001

13. The widening digital literacy gap
• Centre for Data Ethics and Innovation (CDEI)1: There is almost no
awareness of the use of algorithms (in the public sector)
• Gran et al (2021)2:
• Lack of awareness of algorithms as the new reinforced digital divide
• If people are not aware that algorithms are being used in a certain environment,
they can’t deploy the skills to navigate that environment
(c) Ana Isabel Canhoto www.anacanhoto.com 13
1. Source: https://thinksinsight.com/britainthinks-complete-transparency-complete-simplicity/
2. Source: Gran, A. B., Booth, P., & Bucher, T. (2021). To be or not to be algorithm aware: a question of a new digital divide?. Information,
Communication & Society, 24(12), 1779-1796.

14. (c) Ana Isabel Canhoto www.anacanhoto.com 14
Datasets
Algorithms Prompting
sensitivity
Big tech
AI literacy
Interdisciplinary
perspective

15. Leverage interdisciplinary perspectives
(c) Ana Isabel Canhoto www.anacanhoto.com 15
• Complement your knowledge gaps
• Computer science, medical sciences, human geography,
arts…
• Strengthen your own subject knowledge (Spiller et
al, 2015)
• Inform others’ work
Source: Spiller, K., Ball, K., Daniel, E., Dibb, S., Meadows, M. & Canhoto, A. I. (2015). Carnivalesque collaborations: reflections on ‘doing’ multi-
disciplinary research. Qualitative Research, 15(5), 551–567. DOI: 10.1177/1468794114548946

16. (c) Ana Isabel Canhoto www.anacanhoto.com 16
Datasets
Algorithms Prompting
sensitivity
Big tech
AI literacy
Interdisciplinary
perspective
Methods

17. Multiple methodologies
(c) Ana Isabel Canhoto www.anacanhoto.com 17
Dimoka (2010).
• Beyond the traditional social sciences
methods
• Brain science – e.g., Dimoka (2010)
• Creative methods – e.g., Kara (2020)
• Simulations – e.g., Fugener et al (2021)
• …
Sources:
• Dimoka, A. (2010). What Does the Brain Tell Us About Trust and Distrust? Evidence from a
Functional Neuroimaging Study. MIS Quarterly, 34(2), 373–396. https://doi.org/10.2307/20721433
• Fügener, A., Grahl, J., Gupta, A., & Ketter, W. (2021). Will humans-in-the-loop become borgs? Merits
and pitfalls of working with AI. Management Information Systems Quarterly (MISQ)-Vol, 45.
• Kara, H. (2020). Creative research methods: A practical guide. Policy Press.

18. (c) Ana Isabel Canhoto www.anacanhoto.com 18
Datasets
Algorithms Prompting
sensitivity
Big tech
AI literacy
Interdisciplinary
perspective
Methods
Behavioural
insight

19. We know about humans
(c) Ana Isabel Canhoto www.anacanhoto.com 19
Sources:
• Booth, S., Tompkin, J., Pfister, H., Waldo, J., Gajos, K., & Nagpal, R. (2017, March). Piggybacking robots: Human-robot overtrust in university
dormitory security. In Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (pp. 426-434).
• Canhoto, A.I., Keegan, B. J. & Ryzhikh, M. (2023) Snakes and Ladders: Unpacking the Personalisation-Privacy Paradox in the Context of AI-
Enabled Personalisation in the Physical Retail Environment. Information Systems Frontiers, DOI: 10.1007/s10796-023-10369-7
• Jago, A. S. (2019). Algorithms and authenticity. Academy of Management Discoveries, 5(1), 38-56.
Jago (2019)
Booth et al (2019)
Canhoto et al (2023)

20. (c) Ana Isabel Canhoto www.anacanhoto.com 20
Datasets
Algorithms Prompting
sensitivity
Big tech
AI literacy
Interdisciplinary
perspective
Methods
Behavioural
insight
Responsible
research

21. Responsible research
(c) Ana Isabel Canhoto www.anacanhoto.com 21
Sources:
• https://freakonomics.com/series/freakonomics-radio/
• Hannigan, T., McCarthy, I. P., & Spicer, A. (2023). Beware of botshit: How to manage the epistemic risks of generative chatbots. Business Horizons,
Forthcoming.
Hannigan et al (2023)
Freakonomics

22. (c) Ana Isabel Canhoto www.anacanhoto.com 22
Datasets
Algorithms Prompting
sensitivity
Big tech
AI literacy
Interdisciplinary
perspective
Methods
Behavioural
insight
Responsible
research
Engagement

23. Stakeholder engagement
(c) Ana Isabel Canhoto www.anacanhoto.com 23
Sources:
• Canhoto, A. I., Quinton, S., Jackson, P. & Dibb, S. (2016). The co-production of value in digital, university–industry R&D collaborative projects.
Industrial Marketing Management, 56(5), 86–96. DOI: 10.1016/j.indmarman.2016.03.010
• Spence, L. J., & du Gay, P. (2022). In praise of involvement. Business & Society, 61(4), 833-838.
Canhoto et al (2016)

24. AI-Powered
Service
Interactions
Researching and shaping the
future of
Professor Ana Isabel Canhoto
University of Sussex
@canhoto
(c) Ana Isabel Canhoto www.anacanhoto.com 24