CAN AI EFFECTIVELY
TAG FOR EMPATHY?
DAM Practitioners’ Summit
Director, Content Experience
A Discussion on Artificial Intelligence and
the Perception of Empathy.
TOPICS FOR TODAY
• Why Empathy?
• Tagging for Empathy: Is This Practical at Scale?
• Some Final Thoughts
1/30/2020 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 2
1/30/2020 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 3
Rebecca Schneider, Executive Director, Content
Organizer. Librarian. Gadget Lover. Owner of many, many pairs of
• Expert in taxonomies, metadata, and enterprise content
• Thought leader in taxonomy development and metadata.
• Key Clients: Total Wine & More, Verizon Wireless, Bank of
New York Mellon, Analog Devices
Helping you shape your content experience
1/30/2020 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 4
• I’d like to see empathy represented in tagging.
Rebecca Schneider, DAM Summit 2018
• Presentation: Tagging & Empathy
Rebecca Schneider, DAM Summit 2019
• OK, but how can we practically tag to scale? Is AI the solution?
Rebecca Schneider, October(ish) 2019
• This is a conversation, please contribute!
Why This Topic?
I’m a bit curious!
1/30/2020 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 6
Recognize what the other person is feeling.
Feel what the other person is feeling.
We want to help the other person deal with his/her situation and emotions.
1/30/2020 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 7
Being able to walk in another person’s shoes . . .
No matter how much those four-inch heels hurt you.
— Margaret Magnarelli
In Short . . .
Empathy is a skill that can be taught and learned.
1/30/2020 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 8
• Creates distinctive “in the moment”
experiences by providing highly relevant
• Increases brand loyalty, increasing
customer lifetime engagement
• Amplifies interest in the brand, beyond
initial customer base
Why is empathy important?
Brand and marketing perspective
I miss the personalization that Vegas was -
there were showroom captains and all the
dealers knew the gamblers by their first
— Wayne Newton
The Need for Context.
9/25/2019 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 10
What is your way of life?
Where did you go to school?
What’s your rough level of
How do you identify with others?
• Social Norms
What is acceptable in your world?
The Veldt by Ray Bradbury. a mother and father struggle with their
technologically advanced home taking over their role as parents, and
their children becoming uncooperative as a result of their lack of
9/25/2019 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 11
• Think in terms of an empathy map
Who is the user and their context?
• Audience (and context)
• Communication Goal
Empowerment, Understanding, etc.
• Emotional Mindset
Needs Validation, Got to Be First, Buy
and Be Done, etc.
9/25/2019 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 12
Empathy: Metadata Example
What would an empathy metadata structure look like?
• Context (and descriptive metadata)
Train, Travel, Snow
• Audience (and context)
Segments – Pre-Teens, Parents
Persona – Sally the Searcher
• Communication Goal
• Emotional Mindset
1/30/2020 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 13
• Walking the customer journey.
• Relaying customer stories.
• Using qualitative success measures.
Sentiment, focus groups, in-depth interviews, etc.
• Gathering customer support feedback.
• Leveraging sales team input (point of sale – B2C; direct sales – B2B).
Is This Practical at Scale?
1/30/2020 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 15
• AI and machine learning is already used for tagging assets.
• Many focus on particular verticals and associated objects (including people).
• Affective AI creates intelligence that responds to our facial expressions, vocal
undertones and other nonverbal cues.
But how does this help us tag assets using empathy?
AI & Metadata
1/30/2020 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 16
Interpreting Images is Complex
1/30/2020 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 17
• Reference datasets can vary
• Breadth vs. depth affects quality depending on coverage
• “Training” and review is paramount
Importance of Datasets
1/30/2020 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 18
• A ‘canonical’ training set, launched in
• Grew to 14 million images (including
those harvested from Google Images).
• Images were organized into over 20k
• Used for computer vision research.
• Exposed bias, issues with judgement.
Machines are only as unbiased as the
training sets they work with.
Datasets aren’t simply raw materials to
feed algorithms, but are political
interventions. As such, much of the
discussion around ‘bias’ in AI systems
misses the mark: there is no ‘neutral,’
‘natural,’ or ‘apolitical’ vantage point that
training data can be built upon.
— Kate Crawford and Trevor Paglen
Multiple Opportunities for Bias
9/25/2019 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 20
Remember Our Girls on the Train?
Some Final Thoughts
1/30/2020 | @RebeccaDeclares | © 2019 AvenueCX, LLC. All rights reserved. 22
• Now? No
• Future? Maybe
• It will require well-defined datasets.
• What if we consider bias as part of the context?
Mitigates risk (we know it is there)
Creates more data (potentially a bad or good thing)
Could provide more relevant experiences
Tagging for Empathy at Scale?
Thanks for your time today.
Today I want to explore how we can potentially tag for empathy at scale (instead of small one-off projects).
This is an open discussion, and I welcome any comments or questions during this presentation.
I work a lot with taxonomies and metadata with many of my clients.
I’m a content strategist that has worked with digital asset management projects.
So why this presentation?
This sprung from a comment I made during the 2018 DAM Summit in (I believe) this very same room. It was in response regarding the future direction of DAMs, I said that it would be nice to see more empathy represented in asset tagging.
Insight Exchange Network contacted me about speaking at the 2019 Summit – and I spoke on “Tagging and Empathy.”
I posed a question: can we tag to scale for empathy?
Then I had to pull the presentation together! After interviews, much research and thinking, here are my thoughts. This is an exploration and a start of a discussion.
So, again, feel free to contribute!
There are three types of empathy.
Cognitive. Emotional recognition; the perception and accurate identification of the ‘feeling states’ of others.
Emotional. Also called “Affective Empathy” is the mirroring of the ‘feeling states’ of others.
Compassionate. Feelings of sympathy, concern and compassion for another. Often considered to be a consequence of the first two forms of empathy. Typically this type of empathy is the most socially desirable.
A great video explaining empathy, by Brene Brown: https://youtu.be/1Evwgu369Jw
The key here, is that empathy is a skill that can be taught.
Quote: Margaret Magnarelli, https://contently.com/2017/11/03/marketing-buzzword-actually-care-about/
While empathy is considered to be good for people, what about organizations?
Keeping empathy in mind, helps to:
Creates distinctive content experiences that many consumers are expecting.
These distinctive content experiences lead to brand loyalty.
And they increase consumer interest.
So where does all of this talk about empathy get us? Let’s talk about context and think in terms of representing empathy when tagging assets.
This is particularly important for global brands, but can apply to all brands.
Culture – Way of life (which can differ within the same country). As someone born and raised in the Midwest and who has been living on the East coast for 20 years, I can certainly attest to this.
Education – Education level is key here, as well as associations with college or university experiences (especially in sports).
Income – For certain brands, understanding income is important. Is it affordable? Is it exclusive? Is it trendy? Will I look cool?
Ethnicity – Where do you feel at ‘home’? Who are your ‘people’? This can be small or large groups.
Social Norms – What is socially acceptable to you? Again, this could vary widely within a group or geographic area.
Keeping this context in mind, let’s talk metadata.
Now what about empathy metadata? I like to think in terms of an empathy map.
Who is the user and his/her context?
Audience – this is really where the context comes into play.
Segment - is the division of the market or population into subgroups with similar motivations. Segments can include: geographic, demographic, use of product, level of expertise
Persona - fictional characters (archetypes), which are created based upon research in order to represent the different user types that might use your service, product, site, or brand in a similar way.
Communication Goal – What does the company/brand want to communicate?
Emotional Mindset – What’s the mindset of the user? For example, in a retail context it could be: Needs Validation, Got to Be First/Early Adopter, Buy and Be Done with it, etc.
By incorporating these aspects into your metadata and tagging strategy allows you to delivery empathetic content to specific audiences.
Source for Emotional Mindset examples: Smith Report: Emotional Drivers of Purchase Decisions, https://smith.co/assets/docs/SMITH-POV-8-modes-of-shopping-report.pdf
Image: NN/g Nielsen Norman Group, https://www.nngroup.com/
Of course, there are additional types of metadata (photo color/treatment) and so on. This is just one slice.
Ultimately, you need to put yourself in your customer’s shoes.
Focus on qualitative methods, including:
Walking the customer journey(s)
Customer support/call center feedback
Sales team input, for B2C and B2B
This will help you direct your tagging strategy and help you to course correct when needed.
A bit more definition on AI:
Artificial intelligence can be loosely interpreted to mean incorporating human intelligence to machines. Whenever a machine completes tasks based on a set of stipulated rules that solve problems (algorithms), such an “intelligent” behavior is what is called artificial intelligence.
Machine learning can mean empowering computer systems with the ability to “learn”.
Deep learning is a subset of ML; in fact, it’s simply a technique for realizing machine learning. In other words, DL is the next evolution of machine learning.
Affective AI - aims to create artificial intelligence that recognizes and responds to our moods, emotions, facial expressions, vocal undertones and other nonverbal cues
Images can be slippery and unreliable. Full of contradictions.
The apple with a title of “This is Not an Apple.”
The water feature that could be mistaken for a syringe.
Interpretation of an emotional state: laughing so hard you are crying.
As humans, with our experience – we have a nuanced interpretation of these images. AI cannot always discern these differences.
Image (Girl): https://commons.wikimedia.org/wiki/File:When_was_the_last_time.jpg
Apple (This is Not and Apple, Rene Magritte): https://www.wikiart.org/en/rene-magritte/this-is-not-an-apple-1964
Water Feature: https://www.theverge.com/2019/7/19/20700481/ai-machine-learning-vision-system-naturally-occuring-adversarial-examples
Datasheets can vary in quality.
What were the decision criteria?
Who is accountable?
Breadth vs. Depth
More does not always equal better.
Current data may not be representative of future trends.
Historically underrepresented factors may increase bias.
Training and review is super important.
We need an ability to explain the outcomes of AI and related unintended consequences.
Organizations struggle to accept responsibilities for processes they cannot understand or control.
All of this increases cost and is a potential barrier to entry.
ImageNet contained a federated database of several datasets for AI – some more problematic than others.
Included objects as well as people.
The way it classified people was very problematic – as discovered as part of an art project: ImageNet Roulette.
ImageNet Roulette revealed ways in which people were tagged with racist and highly offensive terms: Flop, Kleptomaniac, Wanton, Tosser.
An images of a woman asleep in an airplane seat, right arm protectively around her pregnant stomach = snob.
Included many other misogynistic and racist terms, which I will not repeat here.
Many of these images (objects and people) were categorized by Amazon Mechanical Turk workers – low paid, crowed sourced labor, whose work ultimately affect the AI they were helping to create.
While some of the more problematic datasets were removed, it did not fully address the underlying problem. Lack of transparency, understanding of process and governance.
Excavating AI (on the ImageNet Roulette project): https://www.excavating.ai/
How AI Selfie App ImageNet Roulette Took the Internet by Storm: https://frieze.com/article/how-ai-selfie-app-imagenet-roulette-took-internet-storm
From Excavating AI
At the left, a categorization of cognitive biases, in four broad buckets.
What should we remember.
Too much information
Need to act fast.
Not enough meaning.
At the right, one section of the codex.
“We are drawn to details that confirm our own existing beliefs” – including:
Post-purchase rationalization (my favorite, all those shoes!)
And more . . .
Cognitive Bias Codex: https://en.wikipedia.org/wiki/List_of_cognitive_biases
Remember our girls?
Are they both girls?
What is your bias when look at this image?
Do you not like snow? Maybe it is ash? Or a ruined photograph?
Perhaps you don’t like children? Do you love children?
Love children, but hate cats?
Bias exists everywhere.
Right now, I don’t think tagging for empathy at scale is really practical. Unless there are a lot of resources behind this sort of project.
In the future – potentially.
Training datasets are critical and require transparency, understanding and rigor. AI system will also need the high level of transparency into process, decision-making into application of terms. Governances is so very important here.
In considering bias – I think we should own our bias and be transparent. Include it as part of our dataset and metadata. While we can be more neutral – we cannot be completely neutral. So, instead of pretending to be neutral – let’s just admit to our bias. It is now part of the context.