Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Ethical Edges We Build


Published on

Presented at University of Guelph November 30, 2018

Published in: Design
  • Be the first to comment

The Ethical Edges We Build

  1. 1. The Ethical Edges We Build Jess Mitchell, OCAD University @jesshmitchell I am an American, who as of today is eligible to begin my Canadian citizenship application. And I am standing on the land of the Anishinabek Nation and that makes me feel complicated: awesome, woeful, thoughtful, hopeful… Title: Rethink Everything, or at least some things Let’s talk about how to have tough conversations. It feels as though Open Dialogue is a great next step to confronting biases, assumptions, and exploring unmet needs. How can we do it though? How can we do it when we’re all so polarized? How can we do it and make sure it’s productive. What do we risk losing by not doing it? Let’s discuss…
  2. 2. We are all Designers of this world We contribute to and shape and are shaped by culture, but that’s a harder one to see the designs of… But we make decisions, organize our spaces (kitchen, living room) We choose to organize our thoughts uniquely (3 ring binder and those of us with post-its everywhere) We are motivated by emotions and feelings (in shopping — impulse buys — being attracted to nostalgic things, etc.) We walk through the world with thoughts, feelings, and actions — an in all of that we are deciding this and not that…
  3. 3. As Designers we make ethical decisions If I value this and not that (think of Oprah’s bookclub and the impact it has on books added to its list) if I choose this person and not that one (so many of these decisions aren’t based on reason or rationality and we know it — BIASES) if I choose this cultural norm and not that one (and if enough of us do this we are creating a cultural norm) Who gets to go to grad school?
 Who gets published? Add power to decisions and they quickly exert an ethic more widely and when I choose something (implicitly, explicitly, intentionally or unintentionally) I’m choosing not something else - this is ethics. How do we choose one at the expense of the other? 80/20 is an ethical decision
  4. 4. What is your Tolerance for Failure? One way to think about this question is to ask yourself as an ethicist, what is your tolerance for failure? Have you ever thought about this? What are your limits? Hiring is a good example or grading — what if we get it wrong? What if a judgement or determination we make effects the trajectory of someone’s life or next move at least and we got it wrong? This is the power of deciding… And I want to talk about the responsibility that I believe should come with that… I want us to tap back into the pit in our stomachs — the one that we should have about getting it wrong.
  5. 5. Many of the modern problems we’re confronting are ethical issues. and I would suggest we aren’t doing a very good job talking about them — let alone addressing them
  6. 6. who gets to be a Canadian? who gets to come to Canada? who gets in the door — literally and metaphorically whose family gets recognized (YMCAs and their family policy) who gets hired for the job who even applied for the job who gets the promotion or the raise who gets access who can use this app who can speak up who has power who is the decider who can afford childcare who gets an education who gets a bonus who has potable water? who do police watch? who looks suspicious?
  7. 7. Path And we know when it doesn’t work. We can see things and question them But we only question some things We leave some of the questions out… Why? Path in Park A concrete path ends with 4 steps and then grass in a garden.
  8. 8. Fail Bic pens for “Her” Conflation of two things that folks who take any women’s studies courses get disentangled immediately – sex vs. gender There are plenty of women who are feminine – and there are plenty that are masculine Who uses these pens? I wouldn’t, Madeline Albright? Angela Merkle? Christine LeGarde (managing director of the IMF)? My mom? I use pens that come in exactly these colours but they are gel pens marketed to designers – they look and feel like design tools, I can choose the 0.5mm which is VERY important to me. And they are the same pens my male colleagues use. And when we fail, we fail people! Here we’re reinforcing a problematic message about what it is to be a ‘her’ we all talk about wanting person-centred work, solutions, processes, services… but we’re failing people when we fail…
  9. 9. For Whom? My 75 year old father is a car guy — always has been. He showed up to buy a new car with his and my my mom’s favourite cd in hand… but the new cars don’t have cd players anymore. Who doesn’t have a computer to rip cds to a USB drive? Who doesn’t know what ripping is or how to get music onto a USB key? who was this infotainment system designed for? infotainment system in a car that is digital, colourful, and touch-screen sa=i&rct=j&q=&esrc=s&source=images&cd=&ved=2ahUKEwjIs8j08fXeAhWi24MKHdjmC7IQjRx6BAgBEAU& ws%2Fsubaru-launches-value-packed-2018-outback&psig=AOvVaw2hCveFjk1Ri2SljqPmgNGO&ust=1543453324306899
  10. 10. For Whom? the new student centre at Ryerson is an accessibility failure for those who are blind. who was this student centre designed for? David Lepofsky in his YouTube video showing the Ryerson student centre’s lack of accessiblity
  11. 11. For Whom? 2 men arrested for waiting for a business meeting in a Starbucks. who was Starbucks designed for? Donte Robinson and Rashon Nelson the two black men who were arrested in a Starbucks as they waited for a business meeting.
  12. 12. Things are changing: how we handle sexual harassment gay marriage the environment how we talk about Black Lives in 1989 Neil Young was singing about the ozone layer in Rockin in a Free World… We got a thousand points of light For the homeless man We got a kinder, gentler, Machine gun hand Got styrofoam boxes for the ozone layer Got fuel to burn, got roads to drive.
  13. 13. THINK •• FEEL •• DO comic showing a long line at a “simplistic solutions” booth and no one lined up at the “How to think for yourself” booth. But change is slow often. This is all of us. We are lazy. We take the quick fix. Feel uncomfortable or anxious, take a pill. Have conflict, avoid it. Don’t know something, don’t admit it. We are taught these ways of approaching problems. Look for an existing policy, look for an example of how it’s been done before (precedent), look for a scapegoat, look for a way to CYA. This is what we get from living in a litigious society - one where the calculation is often ‘can I get away with this’ not ‘is this the right thing to do.’ We THINK we can get away with it. Sub-prime mortgage mess Global financial crisis The environment and corporations/politicians — the vision is 4 years at most in the USA — talk about the aid in developing countries. we see the effects of this limited sight in Toronto too — just look at transportation!
  14. 14. feelings pr0n THINK •• FEEL •• DO Sarah McLachlan pet commercial Starving children in Africa fundraisers Telethons for kids with disabilities Worry about the homeless during holidays only Participate in a 5K for fill-in-the-blank but otherwise don’t contribute Disability Porn - Stella Young trotting someone out to tap into your emotional self to try to extract money/sympathy/fair treatment Do enough that we feel we’ve done something, but don’t think about it too much… Stressed Fearful/Threatened Overwhelmed Behind Anxious
  15. 15. When presented with a problem, what is the first thing you do? THINK •• FEEL •• DO When we are in the position of “designing’ through our decisions, we often fall short because of our approach: we approach problems this way we use form and function reflexively — We seldom talk about how form and function influence us. And they do – they influence how we think, what we do, and how we see the world. Rule followers
  16. 16. How we do things now •Start with a brief that explains our vision + purpose •Define the outcomes we hope to see •Find a method to apply to achieve results •Find a method to measure success use form w/o thinking of the impact of the form is irresponsible the rubric — oh, we’ll just adapt our rubric; this is what algorithm engineers say too — oh, it’s failing because we just need to train it on more data — to what end? when do we say UNCLE? or STOP, this isn’t working? the method — that’s the way we’ve always done it —2016 There could be a very serious problem with the past 15 years of research into human brain activity, with a new study suggesting that a bug in fMRI software could invalidate the results of some 40,000 papers the final report — and the appendix of variance — the Weinstein Company — now defunct — what did last year’s final report look like?
  17. 17. How we do things now •Start with a brief that explains our vision and purpose 
 Restate the vision and purpose •Define the outcomes we hope to see 
 Show the outcomes have occurred •Find a method to apply to achieve results 
 Show we’ve used the method •Find a method to measure success 
 Show measures of success
  18. 18. Edges Failures and Gaps Reveal where the opportunity is Show places for innovation THINK •• FEEL •• DO but we know that the failures are more important than the successes if we solve right away we don’t entertain the edges — and that’s where it’s at…
  19. 19. What is the role of capital? Whose voice is being amplified? Whose is being diminished? Who ultimately benefits? And we find the edges when we ask these questions. We also get transparency as an outcome if we ask these questions. 
 And these questions reveal the ethical edges.
  20. 20. Who has been architecting 
 our world? Beyond the simple and same – what we can do is undo the short-thinking of those who have been architecting our world by questioning and looking at how it has failed us Re-think how we ask the question Question the rules – who made them? 
 Why are you following them? 
 Who stands to benefit from them? 
 Do they make sense anymore? 
 Do they work?
  21. 21. If we cannot have a perfect predictor, what is the error rate that we as a society will tolerate? Who should be responsible for regulating and verifying this? - Julia Dressel; Dartmouth ‘17 Digitally, software engineers, developers, and designers are architecting our world. And they are often doing it alone (without ethics, without regulation, without checks and balances, without accountability). “We have, over the past few decades, unleashed technology on the world without fully comprehending its negative effects,” “Our research suggests that age and total number of previous convictions are the two most predictive criteria of recidivism used by COMPAS,” Dressel says. “The software is only moderately accurate so it makes lots of mistakes. “On a national scale, black people are more likely to have prior crimes on their record than white people, thus black defendants in general are more likely to be classified as medium or high risk. Mathematically, this means that the false positive rate is higher for black defendants than white defendants, leading to racial bias in recidivism prediction,” she says. “This is the perfect example of how technology can reinforce existing systematic inequalities.”
  22. 22. Who is 
 re-architecting our world? Beyond the simple and same – what we can do is undo the short-thinking of those who have been architecting our world by questioning and looking at how it has failed us Re-think how we ask the question Question the rules – who made them? 
 Why are you following them? 
 Who stands to benefit from them? 
 Do they make sense anymore? 
 Do they work?
  23. 23. The mob decides: It’s Twitter and the loudest, most angry voice is amplified. Re-architects Our tools for ostracizing each other are the same tools we use to create powerful community because all those digital tools, what they have in common is they’re magnifying the basic social instincts, good and bad, that are there in each of us. IRL — Mozilla podcast there was a story a few years ago about a little boy who ran into his aunt’s arms saying, “I love you, Auntie,” but the boy jumped so hard that he damaged her wrist. The aunt sued the little boy to pay for her hospital bills. Twitter responded loudly. because of this little weird American health insurance business in her deposition against the health insurance company who wouldn’t pay for the operation to get her wrist back in shape, the wording made it look as if she was including the child in her deposition. It was just legal wording to get the health company to pay for her operation So if we had waited two days, we wouldn’t have torn this woman to shreds because the information was wrong." "If the moderates stay silent, then who gets to be the spokespeople? It’s the people with the loudest voices."
  24. 24. Jian Ghomeshi Louis C.K. Harvey Weinstein Matt Damon Michelle Jones Innocent until proven guilty, right? And if proven innocent, then what? NY review of books article — backlash — a few who said they were glad it was published.
  25. 25. Outsourced to a PR company or lawyers make the decision Re-architects The formula seems to be: hire a PR firm and go out of the public spotlight immediately remain gone for at least 8 months do something positive feel out the pulse, attitude toward you… proceed with caution
  26. 26. You do the crime, 
 you pay the time we believe in this right? Thumbs up? Be suspicious folks — this is very black and white and we live in a grey world…
  27. 27. Michelle Jones after serving 20 years in prison for killing her 4 year old son, getting into Harvard’s History Graduate program, the president, provost, and deans of the graduate school reversed the decision out of concern that her background would cause a backlash among rejected applicants, conservative news outlets or parents of students some worried about the way she described her crime — maybe without the level of contrition they expected
  28. 28. You do the crime, 
 you pay forever who decides how long? If we put trust in an institution to decide, how do we make sure that institution is making good decisions? How many false positives are there?
  29. 29. Rehabilitation Reconciliation Forgiveness What do we lose if we don’t consider these things?
 Do we believe in them? 
 Can someone be rehabilitated? What does reconciliation look like? Who decides? How long does it take? What does forgiveness look like??
  30. 30. Who is accountable? Who is responsible? Where does the buck stop? What is YOUR moral responsibility?
  31. 31. Innocent until proven guilty The presumption of innocence from the UN Declaration of Human Rights this is the backbone of much law, it is an ethical edge we have socially decided on — and yet we know that courts gets it wrong. What is their tolerance for failure?? I’ve been obsessing about wrongful imprisonment cases lately. These are literally where we see the tolerance for failure.
  32. 32. ACLU What is our tolerance for failure? It isn’t fixed and I suspect it’s changing, flexing, expanding, and contracting through time and context. Go to Wikipedia and search for 2002 Brian Banks Rape of Wanetta Gibson* Long Beach, California 6 years in prison 5 years Yes Brian Banks was a student at Long Beach Polytechnic High School when a fellow student accused him of rape. He accepted a plea deal to avoid a lengthy sentence and ended up serving almost the entire sentence. The accuser was later recorded admitting that the sexual contact was consensual and that she made up the allegation so her mother wouldn't find out she was sexually active. Gibson's family had received a $1.5 million settlement from the school following Banks' guilty plea for failing to keep Wanetta safe.
  33. 33. What do we tolerate more? false-positives
 false-negatives are we consistent? are we fair?
 how do we decide? who decides? and is this changing??
  34. 34. Who has been architecting 
 our world? Who is 
 re-architecting it? Beyond the simple and same – what we can do is undo the short-thinking of those who have been architecting our world by questioning and looking at how it has failed us Re-think how we ask the question Question the rules – who made them? 
 Why are you following them? 
 Who stands to benefit from them? 
 Do they make sense anymore? 
 Do they work?
  35. 35. Powerful institutions corrupted often by the power antiquated in the way they do things
  36. 36. SHAME / BLAME Who is speaking out? Whose voices are we still not hearing?
  37. 37. Social Change how does it happen? glacially? radically? by revolution?
  38. 38. What is our tolerance for error? What is our demand for transparency? What about privacy and representation?
  39. 39. What would it look like to burn it down and start over? who is with me?! but we will throw out the baby with the bathwater. What would this new system look like though?
  40. 40. beware certainty beware completion The Unexamined Life uttered by Socrates at his trial for impiety and corrupting youth, for which he was subsequently sentenced to death was his life sentence a false-positive? did he really corrupt youth? was he guilty of impiety? are those really bad things?!
  41. 41. The Examined Life of 3s yes, no, maybe black, white, grey
  42. 42. Wake up / Stand up If you’re designing technology for society, and you don’t know anything about society, you’re unqualified. - Safiya Noble If you’re making any decisions that effect other people I would argue this should also apply…
  43. 43. Dr. Ian Malcolm, played by Jeff Goldblum in Jurassic Park quoted as saying, “Your scientists were so preoccupied with whether they could they didn’t stop to think if they should.”
  44. 44. @jesshmitchell My twitter handle: @jesshmitchell