How Not to Destroy the World: Ethics in Design and Technology

How Not to Destroy the World
Smashing Conference Freiburg, 2018
by Morten Rand-Hendriksen

Can we use facial recognition to make
this user experience better?
Can we trust tech to not fork this up?

Define “better”.
Better, for whom?
Do we trust this technology?
Can we trust this technology?
What about implicit and explicit bias?
Who is (negatively) affected by this?
What precedence are we setting here?
How do we handle errors?
Who is responsible when something goes wrong?

I thought
we were
making
world
peace.
Act 1.

– Morten Rand-Hendriksen,
Smashing Conference Freiburg, 2018
When you move fast and
break things, you can end
up breaking people,
communities, even society
as a whole.

Technology is not value neutral.
Neither is design.

What is
right
and wrong.
Act 2.

This feels wrong,
but I can’t articulate exactly why…

ethics
From the Greek ethos, meaning “character” or “custom”.
: rules of behavior based on ideas about
what is morally good and bad.

From the Latin mores, meaning “customs”.
: concerning or relating to what is right and
wrong in human behavior.
morals

– Joan Donovan, media manipulation lead researcher at Data & Society
Platform companies really need to take a hard look at the ethics of
journalism, see why they developed standards, ethics, protocols, and
adopt those ethics and protocols into their algorithms as well as into
their protocols and their content moderation.

We need to talk about ethics;
what it is, and where it takes us.

design ethics
: rules of behavior based on ideas about
what design is morally good and bad.
: tools to help make and explain moral
judgements about design decisions.

How do we judge if what we do is
good or bad, right or wrong?

What virtues do you believe and promote?
What capabilities are you granting and enabling?
How you uphold your duties of care?
What are the consequences?

Can we use facial recognition to make
this user experience better?

"Age and gender detection can be adopted across a wide
range of use cases and markets including targeted offline
advertisement"

Facebook is somehow threatening
me that, if I do not buy into face
recognition, I will be in danger. It
goes completely against the
European law because it tries to
manipulate consent.
- Viviane Reding, former justice commissioner of the European Commission

For one week in January 2012, data
scientists skewed what almost
700,000 Facebook users saw when
they logged into its service. Some
people were shown content with a
preponderance of happy and positive
words; some were shown content
analyzed as sadder than average.

Chinese AI researchers
claimed they had trained a
face-recognition algorithm to
predict – with 90% accuracy
– whether someone was a
convicted criminal.

: the doctrine that the morality of an action is to
be judged solely by its consequences.
consequentialism

: the doctrine that actions are right if they are
useful or for the benefit of a majority.
utilitarianism

The rightness of an act is judged by how
well it promotes happiness.
The greatest happiness for the greatest
number should be the guiding principle of
conduct.

How consequentialism can help:

Consider the Unknown
Explore and project the consequences of every design
decision, both intended and accidental.

Consequentialism as a design tool:
• What are the consequences?
• Does this improve the common good of those affected?
• How do we measure utility?
• Who decides which users matter and why?

How do we uphold our duties of care?

duty ethics
: the normative ethical position that judges the
morality of an action based on rules.

“Act only in accordance with that maxim through
which you can at the same time will that it
become a universal law.”
categorical imperative

: Act in the same way you’d want every other
person to act in the same situation.
categorical imperative

The rightness of an act is judged by
whether it is done out of duty to the
principle of following moral rules.

Do the right thing because
and you believe
it is your duty to
your action
sets a precedence

Duty of Care
tl;dr: You are responsible not only for what
you put into the world, but what that does to
the world and its people.

Duty Ethics as a design tool:
• What norms are established?
• What duties of care do we have, and how do we uphold
them?
• Should every other person or company in this position to do
the same?

What Virtues do we
believe and promote?

- Andrew “Boz” Bosworth, Facebook Vice President
The ugly truth is that we
believe in connecting people
so deeply that anything that
allows us to connect more
people more often is *de
facto* good.

- Andrew “Boz” Bosworth, Facebook Vice President
In almost all of our work, we
have to answer hard
questions about what we
believe.

Values reflect what is acceptable in terms
of culture, but virtues reflect individual
human characteristics.

- Geoff Sheehan, Socrates and Zen
In Socrates’ eyes, a man
who claims to know what
bravery is but does not act
bravely would thereby prove
that he does not know what
bravery is.”

virtue ethics
: the normative ethical position that emphasizes
an individual actor’s character.

An action is right if it is the same as the
action of someone who has virtue.

• Courage
• Temperance
• Truthfulness
• Modesty
• Intelligence
• Logic
• Good sense
• Theoretical wisdom
• Practical wisdom
(Some of) Aristotle’s Virtues
• Honesty
• Self-Control
• Humility
• Justice
• Courage
• Empathy
• Care
• Civility
• Flexibility
• Perspective
• Magnanimity
• Technomoral Wisdom
Vallor’s Technomoral Virtues

• A designer is first and foremost a
human being.
• A designer is responsible for the work
they put into the world.
• A designer values impact over form.
• A designer owes the people who hire
them not just their labor, but their
counsel.
• A designer welcomes criticism.
• A designer strives to know their
audience.
• A designer does not believe in edge
cases.
• A designer is part of a professional
community.
• A designer welcomes a diverse and
competitive field.
• A designer takes time for self-
reflection.
A Designer’s Code of Ethics
by Mike Monteiro

Model Behavior
Act as someone who holds the virtues
you yourself aspire to.

Virtue Ethics as a design tool:
• What virtues do we believe and promote?
• What behaviors are we modeling
for others and for ourselves?
• What person / company do we become by doing this?

What Capabilities are we
granting and enabling?

- Jaron Lanier
as you have advertising, you
have this perverse incentive
to make it manipulative. You
can’t have a behavior-
modification machine with
advertisers and have
anything ethical; it’s not
possible.

capability approach
: theoretical framework that entails two core normative claims:
The freedom to achieve well-being is of primary moral importance,
that freedom to achieve well-being is to be understood in terms of
people's capabilities, that is, their real opportunities to do and be
what they have reason to value.

An action is right if it grants or enables
those acted upon capabilities in the form of
real opportunities to do and be what they
have reason to value.

Design Capabilities
Give everyone the capability to do and be
what they have reason to value.

Capability Approach as a design tool:
• What capabilities are we granting and enabling in
the end-user?
• Do these capabilities allow them the freedom to
achieve well-being?
• What future are we building?

How do they uphold
their Duties of Care?

Who virtues do they

What capabilities are they
granting and enabling?

Their ethics
granted us the
capability of trust.

There are lots of very smart
people doing fascinating work
on cryptographic voting
protocols. We should be
funding and encouraging
them, and doing all our
elections with paper ballots
until everyone currently
working in that field has
retired.

Our folks have mid-
level positions and most
can’t job hop.

I can’t just up and quit
my job because you
say something my
company does is
unethical and is
destroying our society.
I have a family.

The hard
problem of
design
ethics.
Act 4.

How do I include ethics
into my design practice?

What if my only option to
stay ethical is to quit?

How do I use ethics as a
design tool, not just a risk
mitigation tool?

How you uphold your Duties of Care?

at optimal range Sentry can
now identify threatening
aircraft about 95 percent of
the time.

In surveys conducted in
Syria, Hala found that
people need a minimum of 1
minute to seek adequate
shelter.
Sentry now averages a
warning time of eight
minutes.

Sentry is “like a mash-up of
ShotSpotter’s sensor
capabilities and Palantir’s
data analytics, but aimed at
markets that neither of those
companies would likely find
lucrative enough.”

What Capabilities do I want to
grant and enable?

What Virtues do I

How do I uphold my Duties of Care
and set a good precedence?

These are our materials.
UX, Content Strategy, AI,
Machine Learning, Algorithms,
Interaction Design, etc.

How
What
Why
Efficient cause Who
Final cause
Formal cause
Material cause

Design is political.
And so is technology.

We need a social contract
for design and technology.

Beyond User-centered Design,
let’s embrace
Capability-centered Design.

1.1
Contribute to society and to
human well-being,
acknowledging that all people
are stakeholders in
computing.

In this document, "harm"
means negative
consequences, especially
when those consequences
are significant and unjust.

Well-intended actions,
including those that
accomplish assigned duties,
may lead to harm. When that
harm is unintended, those
responsible are obliged to
undo or mitigate the harm as
much as possible.

Using ethics as a design tool
makes you aware of your desire to
shape the world to your vision.

Every design decision we make
carves a path our users will follow
into our shared future.

With every design decision we
build the future for our users
and ourselves.

The place to improve the world
is first in one's own heart and
head and hands,
and then work outward
from there.
- Robert M. Pirsig

How Not to Destroy the World: Ethics in Design and Technology

More Related Content

What's hot

Similar to How Not to Destroy the World: Ethics in Design and Technology

More from Morten Rand-Hendriksen

Recently uploaded

How Not to Destroy the World: Ethics in Design and Technology

Editor's Notes