Download as PDF, PPTX
Uploaded byEvan Estola
When recommendation systems go bad
Evan Estola discusses the risks associated with recommendation systems, emphasizing their influence on personal experiences and ethical concerns regarding data use. The presentation covers the need for interpretable models, ethical data practices, and the importance of diversity in testing to avoid systemic biases. Estola advocates for awareness and conversation around data ethics to prevent prejudice in machine learning algorithms.
More Related Content
Similar to When recommendation systems go bad
When recommendation systems go bad
- 1. When Recommendation Systems Go Bad EvanEstola 12/10/15
- 2. About Me ● EvanEstola ● Senior Machine Learning Engineer @ Meetup ● evan@meetup.com ● @estola
- 3. Agenda 1. Meetup 2. RecommendationSystems 3. What can go wrong? 4. How to prevent it
- 8. We want aworld full of real, local community. Women’s Veterans Meetup, San Antonio, TX
- 13.
- 15. You just wanteda kitchen scale, now Amazon thinks you’re a drug dealer
- 16. Recommendation Systems: RatingPrediction ● Netflix prize ● How many stars would user X give movie Y ● Boring
- 17. Recommendation Systems: LearningTo Rank ● Active area of research ● Use ML model to solve a ranking problem ● Pointwise: Logistic Regression on binary label, use output for ranking ● Listwise: Optimize entire list ● Performance Metrics ○ Mean Average Precision ○ P@K ○ Discounted Cumulative Gain
- 19. Data Science impacts lives ● News you’reexposed to ● Friend’s Activity/Facebook feed ● Job openings you find/get ● If you can get a loan ● Route you take to get somewhere ● If you can get a ride ● Movies you watch on Netflix ● Products you buy on Amazon ● Price you pay for things ● If a product is available to you at all ● Apps you download ● Ads you see
- 23. Ego ● Member/customer/user first ●Focus on building the best product, not on being the most clever data scientist ● Much harder to spin a positive user story than a story about how smart you are
- 24. ● ● Fake profiles,track ads ● Career coaching for “200k+” Executive jobs Ad ● Male group: 1852 impressions ● Female group: 318
- 25. ● “Black-sounding” names25% more likely to be served ad suggesting criminal record
- 26. Ethics We have acceptedthat Machine Learning can seem creepy, how do we prevent it from becoming immoral? We have an ethical obligation to not teach machines to be prejudiced.
- 27. Data Ethics Awareness ● Tell yourfriends ● Tell your coworkers ● Tell your boss
- 28. Awareness ● Start aconversation ○ Identify potentially marginalized user groups ○ Have an ethics strategy for evaluating whether to include sensitive features
- 30. Interpretable Models ● For simpleproblems, simple solutions are often worth a small concession in performance ● Inspectable models make it easier to debug problems in data collection, feature engineering etc. ● Only include features that work the way you want ● Don’t include feature interactions that you don’t want
- 31.
- 32. Feature Engineering andInteractions ● Good Feature: ○ Join! You’re interested in Tech x Meetup is about Tech ● Good Feature: ○ Don’t join! Group is intended only for Women x You are a Man ● Bad Feature: ○ Don’t join! Group is mostly Men x You are a Woman ● Horrible Feature: ○ Don’t join! Meetup is about Tech x You are a Woman Meetup is not interested in propagating gender stereotypes
- 33. Ensemble Models and Data segregation Ensemble Models:Combine outputs of several classifiers for increased accuracy If you have features that are useful but you’re worried about interaction (and your model does it automatically) use ensemble modeling to restrict the features to separate models.
- 34. Ensemble Model, DataSegregation Data: *Interests Searches Friends Location Data: *Gender Friends Location Data: Model1 Prediction Model2 Prediction Model1 Prediction Model2 Prediction Final Prediction
- 36. Diversity controlled test data ● Makesure product works for everybody ● Generate test data and evaluate your model against it to confirm no encapsulated prejudice
- 37. Diversity Controlled Testing ●CMU - AdFisher ○ Crawls ads with simulated user profiles ● Same technique can work to find bias in your own models! ○ Generate Test Data ■ Randomize sensitive feature in real data set ○ Run Model ■ Evaluate for unacceptable biased treatment ● Must identify what features are sensitive and what outcomes are unwanted
- 38. You know racistcomputers are a bad idea Don’t let your company invent racist computers