Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How Not to Destroy the World: Ethics in Design and Technology

545 views

Published on

From Morten Rand-Hendriksen's Smashing Conference Freiburg 2018 talk.

Every decision we make is one made on behalf of your user. How do we know the decisions we make are the right ones? It is time we initiate a conversation: About where we are and where we want to go, about how we define and measure goodness and rightness in the digital realm, about responsibility, about decisions and consequences, about building something bigger than our own apps. It is time we talk about the ethics of design.

This talk introduces a method for ethical decision making in design and tech. Rather than a wet moralistic blanket covering the fires of creativity, ethics can be the hearth that makes our creative fires burn brighter without burning down the house.

https://smashingconf.com/speakers/morten-rand-hendriksen

Published in: Design
  • Be the first to comment

  • Be the first to like this

How Not to Destroy the World: Ethics in Design and Technology

  1. 1. How Not to Destroy the World Smashing Conference Freiburg, 2018 by Morten Rand-Hendriksen
  2. 2. Can we use facial recognition to make this user experience better? Can we trust tech to not fork this up?
  3. 3. Define “better”. Better, for whom? Do we trust this technology? Can we trust this technology? What about implicit and explicit bias? Who is (negatively) affected by this? What precedence are we setting here? How do we handle errors? Who is responsible when something goes wrong?
  4. 4. I thought we were making world peace. Act 1.
  5. 5. – Morten Rand-Hendriksen, Smashing Conference Freiburg, 2018 When you move fast and break things, you can end up breaking people, communities, even society as a whole.
  6. 6. Technology is not value neutral. Neither is design.
  7. 7. What is right and wrong. Act 2.
  8. 8. This feels wrong, but I can’t articulate exactly why…
  9. 9. ethics From the Greek ethos, meaning “character” or “custom”. : rules of behavior based on ideas about what is morally good and bad.
  10. 10. From the Latin mores, meaning “customs”. : concerning or relating to what is right and wrong in human behavior. morals
  11. 11. – Joan Donovan, media manipulation lead researcher at Data & Society Platform companies really need to take a hard look at the ethics of journalism, see why they developed standards, ethics, protocols, and adopt those ethics and protocols into their algorithms as well as into their protocols and their content moderation.
  12. 12. We need to talk about ethics; what it is, and where it takes us.
  13. 13. design ethics : rules of behavior based on ideas about what design is morally good and bad. : tools to help make and explain moral judgements about design decisions.
  14. 14. How do we judge if what we do is good or bad, right or wrong?
  15. 15. What virtues do you believe and promote? What capabilities are you granting and enabling? How you uphold your duties of care? What are the consequences?
  16. 16. The paths we choose. Act 3.
  17. 17. What are the consequences?
  18. 18. Can we use facial recognition to make this user experience better?
  19. 19. "Age and gender detection can be adopted across a wide range of use cases and markets including targeted offline advertisement"
  20. 20. Facebook is somehow threatening me that, if I do not buy into face recognition, I will be in danger. It goes completely against the European law because it tries to manipulate consent. - Viviane Reding, former justice commissioner of the European Commission
  21. 21. For one week in January 2012, data scientists skewed what almost 700,000 Facebook users saw when they logged into its service. Some people were shown content with a preponderance of happy and positive words; some were shown content analyzed as sadder than average.
  22. 22. Chinese AI researchers claimed they had trained a face-recognition algorithm to predict – with 90% accuracy – whether someone was a convicted criminal.
  23. 23. This feels wrong, but I can’t articulate exactly why…
  24. 24. : the doctrine that the morality of an action is to be judged solely by its consequences. consequentialism
  25. 25. : the doctrine that actions are right if they are useful or for the benefit of a majority. utilitarianism
  26. 26. The rightness of an act is judged by how well it promotes happiness. The greatest happiness for the greatest number should be the guiding principle of conduct.
  27. 27. 80/20
  28. 28. Design for the majority.
  29. 29. How decisions are made:
  30. 30. How consequentialism can help:
  31. 31. Consider the Unknown Explore and project the consequences of every design decision, both intended and accidental.
  32. 32. Consequentialism as a design tool: • What are the consequences? • Does this improve the common good of those affected? • How do we measure utility? • Who decides which users matter and why?
  33. 33. How do we uphold our duties of care?
  34. 34. This feels wrong, but I can’t articulate exactly why…
  35. 35. duty ethics : the normative ethical position that judges the morality of an action based on rules.
  36. 36. “Act only in accordance with that maxim through which you can at the same time will that it become a universal law.” categorical imperative
  37. 37. : Act in the same way you’d want every other person to act in the same situation. categorical imperative
  38. 38. The rightness of an act is judged by whether it is done out of duty to the principle of following moral rules.
  39. 39. Do the right thing because and you believe it is your duty to your action sets a precedence
  40. 40. Duty of Care tl;dr: You are responsible not only for what you put into the world, but what that does to the world and its people.
  41. 41. Duty Ethics as a design tool: • What norms are established? • What duties of care do we have, and how do we uphold them? • Should every other person or company in this position to do the same?
  42. 42. What Virtues do we believe and promote?
  43. 43. - Andrew “Boz” Bosworth, Facebook Vice President The ugly truth is that we believe in connecting people so deeply that anything that allows us to connect more people more often is *de facto* good.
  44. 44. - Andrew “Boz” Bosworth, Facebook Vice President In almost all of our work, we have to answer hard questions about what we believe.
  45. 45. This feels wrong, but I can’t articulate exactly why…
  46. 46. Values reflect what is acceptable in terms of culture, but virtues reflect individual human characteristics.
  47. 47. - Geoff Sheehan, Socrates and Zen In Socrates’ eyes, a man who claims to know what bravery is but does not act bravely would thereby prove that he does not know what bravery is.”
  48. 48. virtue ethics : the normative ethical position that emphasizes an individual actor’s character.
  49. 49. An action is right if it is the same as the action of someone who has virtue.
  50. 50. • Courage • Temperance • Truthfulness • Modesty • Intelligence • Logic • Good sense • Theoretical wisdom • Practical wisdom (Some of) Aristotle’s Virtues • Honesty • Self-Control • Humility • Justice • Courage • Empathy • Care • Civility • Flexibility • Perspective • Magnanimity • Technomoral Wisdom Vallor’s Technomoral Virtues
  51. 51. • A designer is first and foremost a human being. • A designer is responsible for the work they put into the world. • A designer values impact over form. • A designer owes the people who hire them not just their labor, but their counsel. • A designer welcomes criticism. • A designer strives to know their audience. • A designer does not believe in edge cases. • A designer is part of a professional community. • A designer welcomes a diverse and competitive field. • A designer takes time for self- reflection. A Designer’s Code of Ethics by Mike Monteiro
  52. 52. Model Behavior Act as someone who holds the virtues you yourself aspire to.
  53. 53. Virtue Ethics as a design tool: • What virtues do we believe and promote? • What behaviors are we modeling for others and for ourselves? • What person / company do we become by doing this?
  54. 54. What Capabilities are we granting and enabling?
  55. 55. - Jaron Lanier as you have advertising, you have this perverse incentive to make it manipulative. You can’t have a behavior- modification machine with advertisers and have anything ethical; it’s not possible.
  56. 56. capability approach : theoretical framework that entails two core normative claims: The freedom to achieve well-being is of primary moral importance, that freedom to achieve well-being is to be understood in terms of people's capabilities, that is, their real opportunities to do and be what they have reason to value.
  57. 57. An action is right if it grants or enables those acted upon capabilities in the form of real opportunities to do and be what they have reason to value.
  58. 58. Design Capabilities Give everyone the capability to do and be what they have reason to value.
  59. 59. Capability Approach as a design tool: • What capabilities are we granting and enabling in the end-user? • Do these capabilities allow them the freedom to achieve well-being? • What future are we building?
  60. 60. What are the consequences?
  61. 61. How do they uphold their Duties of Care?
  62. 62. Who virtues do they believe and promote?
  63. 63. What capabilities are they granting and enabling?
  64. 64. Their ethics granted us the capability of trust.
  65. 65. There are lots of very smart people doing fascinating work on cryptographic voting protocols. We should be funding and encouraging them, and doing all our elections with paper ballots until everyone currently working in that field has retired.
  66. 66. Our folks have mid- level positions and most can’t job hop.
  67. 67. I can’t just up and quit my job because you say something my company does is unethical and is destroying our society. I have a family.
  68. 68. The hard problem of design ethics. Act 4.
  69. 69. How do I include ethics into my design practice?
  70. 70. What if my only option to stay ethical is to quit?
  71. 71. How do I use ethics as a design tool, not just a risk mitigation tool?
  72. 72. What virtues do you believe and promote? What capabilities are you granting and enabling? How you uphold your Duties of Care? What are the consequences?
  73. 73. What virtues do you believe and promote? How you uphold your Duties of Care? What are the consequences? What capabilities are you granting and enabling?
  74. 74. at optimal range Sentry can now identify threatening aircraft about 95 percent of the time.
  75. 75. In surveys conducted in Syria, Hala found that people need a minimum of 1 minute to seek adequate shelter. Sentry now averages a warning time of eight minutes.
  76. 76. Sentry is “like a mash-up of ShotSpotter’s sensor capabilities and Palantir’s data analytics, but aimed at markets that neither of those companies would likely find lucrative enough.”
  77. 77. What Capabilities do I want to grant and enable?
  78. 78. What Virtues do I believe and promote?
  79. 79. How do I uphold my Duties of Care and set a good precedence?
  80. 80. What are the consequences?
  81. 81. Tension & Resolution Act 5.
  82. 82. These are our materials. UX, Content Strategy, AI, Machine Learning, Algorithms, Interaction Design, etc.
  83. 83. How What Why Efficient cause Who Final cause Formal cause Material cause
  84. 84. Technology is not value neutral. Neither is design.
  85. 85. Design is political. And so is technology.
  86. 86. We need a social contract for design and technology.
  87. 87. Beyond User-centered Design, let’s embrace Capability-centered Design.
  88. 88. 1.1 Contribute to society and to human well-being, acknowledging that all people are stakeholders in computing.
  89. 89. 1.2 Avoid harm.
  90. 90. In this document, "harm" means negative consequences, especially when those consequences are significant and unjust.
  91. 91. Well-intended actions, including those that accomplish assigned duties, may lead to harm. When that harm is unintended, those responsible are obliged to undo or mitigate the harm as much as possible.
  92. 92. Using ethics as a design tool makes you aware of your desire to shape the world to your vision.
  93. 93. Every design decision we make carves a path our users will follow into our shared future.
  94. 94. With every design decision we build the future for our users and ourselves.
  95. 95. build the future
  96. 96. The place to improve the world is first in one's own heart and head and hands, and then work outward from there. - Robert M. Pirsig
  97. 97. How Not to Destroy the World Smashing Conference Freiburg, 2018 by Morten Rand-Hendriksen

×