Alice has a Blue Car: Beginning the Conversation Around Ethically Aware Decision Making
Report
ThoughtworksFollow
Nov. 18, 2019•0 likes•797 views
Alice has a Blue Car: Beginning the Conversation Around Ethically Aware Decision Making
Nov. 18, 2019•0 likes•797 views
Download to read offline
Report
Data & Analytics
Who we meet, what we choose to do, and the costs of our choices, are increasingly influenced and even driven by software systems, algorithms and infrastructure. The decisions underlying how those work are made by people just like you. This talk challenges the audience to consider the ongoing implications of decisions made early in the design process, and provides practical examples of translating moral standpoints into real-world implementations. Presented by ThoughtWorks Principle Data Engineer Simon Aubury and Project Manager Dr. Maia Sauren.
ThoughtworksFollow
Recommended
Security by default - Building continuous cyber-resilience.Thoughtworks
Do No Harm: Do Technologists Need a Code of Ethics?Thoughtworks
Hardware is hard(er)Thoughtworks
Technology Radar Webinar UK - Vol. 22Thoughtworks
Developer ExperienceThoughtworks
The layperson's guide to software architectureThoughtworks
More Related Content
What's hot
IoT Meetup Stockholm - Designing Connected ProductsMartin Charlier
Designers, Developers & DogsThoughtworks
Designing Connected Products - Web Directions 2015 SydneyMartin Charlier
Decision Intelligence: How AI and DI (and YOU) are Evolving to the Next LevelLorien Pratt
Prototyping Experiences for Connected ProductsMartin Charlier
Project Guidelinestipfutures2011
Similar to Alice has a Blue Car: Beginning the Conversation Around Ethically Aware Decision Making
Neo4j - Responsible AINeo4j
Présentation de Bruno Schroder au 20e #mforum (07/12/2016)Agence du Numérique (AdN)
AI Orange Belt - Session 2AI Black Belt
Creating Value With Artificial Intelligence, Rudradeb Mitra at Google DevelopersProvectus
Are You Underestimating the Value Within Your Data? A conversation about grap...Neo4j
Responsible AINeo4j
Similar to Alice has a Blue Car: Beginning the Conversation Around Ethically Aware Decision Making(20)
More from Thoughtworks
Design System as a ProductThoughtworks
Cloud-first for fast innovationThoughtworks
More impact with flexible teamsThoughtworks
Culture of InnovationThoughtworks
Dual-Track AgileThoughtworks
Amazon's Culture of InnovationThoughtworks
Recently uploaded
PLM 200.pdfMidhun Mohan
Quantum Karnaugh map in NV-center Quantum Computerssuserb645ae
Predictive HR Analytics_ Mastering the HR Metric ( PDFDrive ).pdfSanthosh Prabhu
apidays London 2023 - 7 pillars of an API Factory, Patrick Brosse, Amadeusapidays
OCTRI PSS Simulations in R Seminar.pdfssuser84c78e
apidays London 2023 - Revolutionising fitness and well-being, David Turner, V...apidays
Alice has a Blue Car: Beginning the Conversation Around Ethically Aware Decision Making
- 1. 1 Dr. Maia Sauren- Program and Project manager Simon Aubury- Principal Data Engineer
- 2. 2 Alice
- 3. 3 why do we care?
- 4. 4 the unintended consequences of not knowing your users
- 5. 5
- 6. 6 To understand our users we need data
- 7. 7© ThoughtWorks 2019 References: https://uxdesign.cc/designing-ethically-pt-2-535ac61e2992 “Data is not the new oil, data is the new asbestos”
- 8. 8
- 9. 9
- 10. 10 10 01 11
- 11. 11 10 01 11 ??
- 14. WHAT COULD POSSIBLY GO WRONG </sarcasm> 14
- 15. 15
- 16. the law is a blunt instrument for a reason 16
- 17. technology is a blunt instrument for the same reason 17
- 18. the user is the signal not the noise 18
- 19. design for high signal tolerance 19
- 20. which of your users provide the broadest definition of signal? 20
- 21. 21
- 22. 22
- 23. 23
- 24. 24
- 25. From theory to practice
- 26. 26 Product - we’re an insurance co
- 28. 28 What should Alice’s premiums be? A machine can do this Premium ($) = Risk (%) X Value ($) ● Features ● Algorithms
- 29. 29 UX ● Consent - meaningful consent ○ Gathering ○ Storage ○ Aligned to 3rd parties ● TOS - did you read it? What are our assumptions?
- 30. 30 UX - simple example What are our assumptions?
- 31. 31 UX - simple example What are our assumptions?
- 32. UX - protect the individual Randomized Response Technique
- 33. 33 Do you drive drunk? Collect useful insights across a population
- 34. 34 What data to calculate risk? What are our assumptions?
- 35. 35 Dev What are our assumptions?
- 36. 36 Dev What are our assumptions?
- 37. 37 Infra & security & storage What are our assumptions?
- 38. 38 Legal - Aust What are our assumptions?
- 39. 39 Legal - UK What are our assumptions?
- 42. 42 Back to our risk algorithm How’s this looking? Premium ($) = Risk (%) X Value ($) ● Features ○ Gender if ! EU ○ Car Model ○ Colour (maybe) ○ Postcode ○ Customer Name ○ Email
- 43. 43 What data to calculate risk? What are our assumptions? X X
- 44. 44 What data to calculate risk? What are our assumptions?
- 45. 45 Alice has a blue car Even if we don’t explicitly set out to capture the data; there’s implicit relationships in our feature set
- 46. 46 don’t reinvent the trolley problem other people have found solutions!
- 48. 48
- 49. 49
- 50. 50 EthicalOS
- 51. 51 Tarot cards of tech
- 57. 57 Game Access
- 58. 58 Glitch TOS
- 59. 59 Tenon.io
- 60. 60 rtl.wtf
- 61. 61
- 62. 62 Risk-Based Ethical Use of Data
- 63. which of your users provide the broadest definition of signal? 63
- 64. 64 Links http://ecee.colorado.edu/~ecen7438/ https://gameaccessblog.org.uk/ https://pair-code.github.io/what-if-tool/ https://theodi.org/wp-content/uploads/2019/07/ODI-Data-Ethics-Canvas-2019-05.pdf https://slideplayer.com/slide/4472170/ DrAfter123 / Vetta / Getty Images https://www.flickr.com/photos/spiderman/2061070613 https://broadlygenderphotos.vice.com/ https://www.monash.edu/__data/assets/pdf_file/0007/216475/An-investigation-into-the-relationship-between-vehicle-colour-and-crash-risk.pdf https://www.mdpi.com/journal/ijerph https://www.express.co.uk/life-style/cars/910479/car-insurance-email-address-gmail-hotmail https://gradientinstitute.org/news/Gradient%20Institute%20Submission%20-%20Australia's%20AI%20Ethics%20Framework%20Discussion%20Paper.pdf https://consult.industry.gov.au/strategic-policy/artificial-intelligence-ethics-framework/ https://incl.ca/wp-content/uploads/2018/05/accessibility-security.pdf http://rtl.wtf/tac/booboos/ http://tarotcardsoftech.com/ https://evapenzeymoog.com/ https://www.designethically.com/toolkit http://www.soc.univ.kiev.ua/sites/default/files/newsfiles/4_slides_rrt.pdf Tenon.io https://incl.ca/ https://glitch.com/legal/#tos
- 65. Thank you 65 Maia Sauren maia.sauren@thoughtworks.com @sauramaia Simon Aubury simon.aubury@thoughtworks.com @SimonAubury
Editor's Notes
- MC SPEAKING, Dr. Maia Sauren is a program and project manager at ThoughtWorks, with backgrounds in large-scale organisational transformation and healthcare applications. Maia’s previous lives were in biomedical engineering, science communication, and education. Simon is a Principal Data Engineer with expertise in enterprise system design for large data systems.
- MAIA SPEAKING Who is alice? What does Alice need, what is relevant ; and what are the consequence of our decisions? Tonight we want to engage you in a conversation .. what guidelines should you consider for your organisation and yourself when building solutions? What are the iimplications of decisions made early in the design process, and how this translates to real-world outcomes.
- Maia speaking Why do we care about alice, or her data? Why would anyone want that?
- Maia speaking Why do we care about alice, or her data? Why would anyone want that?
- MAIA SPEAKING Gender neutral ads for STEM jobs Women click on the ads more Ads become more expensive to show to women Women see fewer ads for jobs in STEM Algorithm aim: minimise spend
- Maia speaking Data needs to be Accurate Clean Diverse
- MAIA SPEAKING
- MAIA SPEAKING What is a signal? What is an error?
- MAIA SPEAKING What is a signal? What is an error?
- MAIA SPEAKING What is a signal? What is an error?
- MAIA SPEAKING What is a signal? What is an error?
- MAIA SPEAKING Noise as it applies to Within greater context Contextualised as relative harm, relative good, relative harm minimisation
- MAIA SPEAKING How do you teach that To a child To a machine
- MAIA SPEAKING How do you teach that To a child To a machine
- Bias laundering through technology - Gretchen Mccullock, Because internet
- MAIA SPEAKING
- MAIA SPEAKING So you have to give it blunt flexibility In the design stage
- MAIA SPEAKING
- MAIA SPEAKING So you have to give it blunt flexibility In the design stage
- MAIA SPEAKING
- Alice Saunders Refused airline ticket by British Aerospace
- MAIA SPEAKING
- MAIA SPEAKING https://broadlygenderphotos.vice.com/
- MAIA SPEAKING Frank Starmer Follow A large red stone (Uluru, south of Alice Springs) An adventure south of Alice Springs to visit Uluru. Here is a sunset photo. -- CC / https://www.flickr.com/photos/spiderman/2061070613
- SIMON SPEAKING Let’s illustrate data ethics with a scenario Can we describes the value judgements we make about Alice and approaches we make when generating, analysing and disseminating data about Alice Do we understand good practice in computing techniques, ethics and information assurance appreciative of relevant legislation, Guidelines - sensible defaults for individual teams Framework - translating values to behaviours Ethical position - organisational value statement And then Guidelines: Framework: Ethical position:
- SIMON SPEAKING We’re going to apply this lens by imaging the product decisions in a hypothical insurance co Congrats on the new job Think about the
- SIMON SPEAKING Alice has a need - an approariatly pricved insurance product What is the best thing for Alice? Let’s design a product
- SIMON SPEAKING Could have actuaries and risk tables People are dumb and lazy – we need robots to do the maths for them.
- SIMON SPEAKING Firstly - we need to engage with Alice. This is typically a sales funnel our business is based on data TOS - plain language; (Linkedin has a sentance that’s 91 words long) no dark patterns; accessible and meaningful (eg. language) organisational value statement
- SIMON SPEAKING Option 1 - poor Option 2 - better ; has choice An explanation of WHY; No word “Other” may make people feel like an after-thought And an explicit call out that it’s kept private Option 3 - best; don’t ask
- SIMON SPEAKING Credit decision based on non-compliance with instructions - blue pen users get a different weighting AFter much investigation, was discovered to be most correlated to the pen left in the office Data is not truth Humans create, generate, collect, capture, and extend data. The results are often incomplete, and the process of analyzing them can be messy. Data can be biased through what is included or excluded, how it is interpreted, and how it is presented. if the data is crappy, even the best algorithm won't help. Sometimes it's referred as "garbage in – garbage out".
- SIMON SPEAKING Respect privacy and the collective good Now : let’s discuss an approach for data capture for a sensitive topic. “Did you cheat on your partner in the last 12 months” “Dispose of waste illegally.” or “Lie on an insurance application” The randomized response technique (RRT) nearly 50 years can be used minimize the response bias from questions about sensitive issues. Protect the individual ; but collect accurate data in aggregate Respondents would be randomly assigned to answer either the sensitive question or the inverse question with a “yes” or “no” answer [8]. As is shown in Table 1, the assignment undergoes a randomization procedure, such as rolling a die
- SIMON SPEAKING Back to our example; how can we capture a meaningful response : do you drive drunk We want a represnatative answer; but if we ask Alice directly we’re unlikely to get an accurate anser How do you Respect privacy and the collective good ONLINE … without a dice? Randomized Response Technique can be applied online by a bit of A/B testing
- SIMON SPEAKING Imagine we’ve now got data that our users are happy to share; and we feel that it’s important to our business Or we have access to a partner data provider What can we derive from these insights? Basic example Basics of Features (Also known as parameters or variables) are the factors for a machine to look at. Those could be car mileage, user's gender, age, postcode; features are column names Algorithms - we might pick ML algoirtim (maybe a decision tree for classification, or clusstering model) that based on the data classifies users, and determines a likely risk
- SIMON SPEAKING Let’s discuss algoirthmic approaches to understanding Alice There;’s nothing new here; techniques here have been used for > 40 years To illustrate - lket’;s think of the simplify case where we have a corpus of relationships We can use algoirimit classifiers to associate frequence clusters of info with historric outcomes If historically we’ve seen high correlations between April and Aries ; perhaps we can automate this assumption as part of our business We could gloss this up as “AI” Just because AI can do something doesn’t mean that it should. What’s the effect on ALice is we apply this approach blindly?
- SIMON SPEAKING With only gender-neutral pronouns in languages like Malay, Google Translate says men are leaders, doctors, soldiers, and professors, while women are assistants, prostitutes, nurses, and maids. The results are similar in Hungarian. According to findings by Strawberry Bomb, Google Translate interprets men as intelligent, and women as stupid. It also suggests that women are unemployed, cook, and clean. Don’t presume the desirability of AI Just because AI can do something doesn’t mean that it should. When AI is incorporated into a design, designers should continually pay attention to whether people’s needs are changing, or an AI’s behavior is changing.
- SIMON SPEAKING Dev responsibilities Information handling Responsibilities for all parties; partners; 3rd parties providers; stakeholders PCI, PII, HIPA HIPPA alone - 51 pages in the overview document for one tech vendor Payments PCI - 71 pages for the same vendor
- SIMON SPEAKING notify the Australian Information Commissioner and individuals whose personal information has been subject to a data breach likely to result in serious harm data breach arises when the following three criteria are satisfied: There is unauthorised access to or unauthorised disclosure of personal information, This is likely to result in serious harm to one or more individuals; and The entity has not been able to prevent the likely risk of serious harm with remedial action. assessment within 30 days Respect privacy and the collective good While there are policies and laws that shape the governance, collection, and use of data, we must hold ourselves to a higher standard than “will we get sued?” Consider design, governance of data use for new purposes, and communication of how people’s data will be used. Call out downsides of holding data It’s a liability; Enduring
- SIMON SPEAKING Last line highlight
- SIMON SPEAKING Admiral admitted to charging drivers higher premiums for using a Hotmail address. It found some enquires using a Hotmail address up to £31.36 more expensive than those using a Gmail account.
- SIMON SPEAKING Car colour does in fact make a big difference to road safety - Silver, grey, red and black cars are most likely to be involved in accidents whereas white, orange and yellow are the safest colour choices
- SIMON SPEAKING So it’s getting a little harder to work out what features to use
- SIMON SPEAKING What do we now know about alice
- SIMON SPEAKING Baxck to our example ; did we pick good Features (Also known as parameters or variables) are the factors for a machine to look at. Those could be car mileage, user's gender, age, postcode; features are column names Algorithms - we might pick ML algoirtim (maybe a decision tree for classification, or clusstering model) that based on the data classifies users, and determines a likely risk DID we find or influence our algoithm in any other way?
- SIMON SPEAKING Even if we don’t explicitly set out to capture the data; there’s implicit relationships in our feature set Models can discover remarkable correlations Can we describes the value judgements we make about Alice and approaches we make when generating, analysing and disseminating data about Alice Do we understand good practice in computing techniques, ethics and information assurance appreciative of relevant legislation, Guidelines - sensible defaults for individual teams Framework - translating values to behaviours Ethical position - organisational value statement
- MAIA SPEAKING
- MAIA What If tool from Google inspect a machine learning model Test algorithmic fairness constraints Examine the effect of preset algorithmic fairness constraints, such as equality of opportunity in supervised learning.
- MAIA SPEAKING This is how we’re thinking internally ; a canvas Risk-Based Ethical Use of Data
- MAIA SPEAKING
- MAIA SPEAKING
- MAIA SPEAKING
- MAIA SPEAKING
- MAIA SPEAKING
- Moriel Schottlender
- MAIA SPEAKING This is how we’re thinking internally ; a canvas Risk-Based Ethical Use of Data
- MAIA SPEAKING
- Nobody speaking