Making fashion recommendations with human-in-the-loop machine learning
•
6 likes•2,319 views
KDD 2016
Recommended
Recommended
More Related Content
What's hot
What's hot (10)
Viewers also liked
Viewers also liked (20)
Similar to Making fashion recommendations with human-in-the-loop machine learning
Similar to Making fashion recommendations with human-in-the-loop machine learning (20)
Recently uploaded
Recently uploaded (20)
Making fashion recommendations with human-in-the-loop machine learning
- 1. Making fashion recommendations with human-in-the-loop machine learning Brad Klingenberg, Stitch Fix brad@stitchfix.com Machine learning meets fashion KDD2016 | San Francisco | August 2016 Three lessons
- 2. Personal styling recommendations with humans in the loop Fashion with humans in the loop: It works really well, but it’s complicated
- 3. Personal styling recommendations with humans in the loop Fashion with humans in the loop: It works really well, but it’s complicated Lesson 1: You have more than one feedback loop Lesson 2: Human selection changes your objective function Lesson 3: Even humans need feature selection
- 4. Humans in the loop for fashion at Stitch Fix
- 5. Stitch Fix
- 6. Stitch Fix
- 7. Stitch Fix
- 8. Stitch Fix
- 9. Styling at Stitch Fix Personal styling Inventory
- 10. Styling at Stitch Fix: personalized recommendations Inventory Algorithmic recommendations Machine learning
- 11. Styling at Stitch Fix: expert human curation Human curation Algorithmic recommendations
- 12. Lesson 0: Humans and machines are a winning combination
- 13. Combining art & science Humans and machines are a winning combination.
- 14. Combining art & science Humans and machines are a winning combination. Human judgement ● helps leverage unstructured data
- 15. Combining art & science Humans and machines are a winning combination. Human judgement ● helps leverage unstructured data ● provides a human element to the process (empathy, creativity)
- 16. Combining art & science Humans and machines are a winning combination. Human judgement ● helps leverage unstructured data ● provides a human element to the process (empathy, creativity) ● frees the algorithm developer from edge cases
- 17. Lesson 1: You have more than one feedback loop
- 18. Traditional recommenders Learning through feedback
- 19. Humans in the loop Learning through feedback
- 20. Humans in the loop Learning through feedback Human interaction
- 21. Lesson 2: Human selection changes your objective function
- 22. Training a model What should you predict?
- 23. Training a model What should you predict? Historical shipment data is plentiful
- 24. Training a model What should you predict? Naive approach: ignore selection and train on historical shipment data Advantages ● “traditional” supervised problem ● simple historical data
- 25. Training a model Problem 1: selection can censor your data
- 26. Censoring through selection Problem: selection can censor your data
- 27. Censoring through selection Problem: selection can censor your data
- 28. Censoring through selection Problem: selection can censor your data Arms flaunted Success Yes No Yes No ? ? p 1-p
- 29. Success given selection Problem 2: success probabilities can make for terrible recommendations
- 30. Success given selection Problem: success probabilities can make for terrible recommendations
- 31. Success given selection Problem: success probabilities can make for terrible recommendations “Probability the stylist should choose this item”
- 32. Success given selection Problem: success probabilities can make for terrible recommendations “Probability the stylist should choose this item” Probability the stylist will be able to send this to the right client
- 33. Success given selection clients Y_i = 1 Y_i = 0 Example: A low-coverage item
- 34. Success given selection clients Y_i = 1 Y_i = 0 An edgy dress. Not many people will like it, but easy for a stylist to identify these clients Example: A low-coverage item
- 35. Success given selection clients Y_i = 1 Y_i = 0 High score! Example: A low-coverage item
- 36. Success given selection Example: A high-coverage item clients Y_i = 1 Y_i = 0
- 37. Success given selection clients Y_i = 1 Y_i = 0 More neutral item. Many clients will like it, but it is hard for stylists to identify these clients Example: A high-coverage item
- 38. Success given selection clients Y_i = 1 Y_i = 0 Average score! Example: A high-coverage item
- 39. A useful recommendation? High score Low score
- 40. The lesson Effective recommendations require understanding human selection In both of these cases,
- 41. Effective recommendations require understanding human selection Generally, selection data will be much larger and more complicated to collect and work with ● Negative cases: logging the set of things that was available to be selected but was not selected ● Presentation effects ○ similar to the way search engine results are studied
- 42. Lesson 3: Even humans need feature selection
- 43. Recall the styling process Human curation Algorithmic recommendations
- 44. Creating features for the human classifier Unstructured data can be overwhelming, even for humans
- 45. Creating features for the human classifier Derived feature 1 Derived feature 2 Feature engineering: creating useful summaries for human consumption
- 46. Creating features for the human classifier Derived feature 1 Derived feature 2
- 47. Creating features for the human classifier It is important to focus on ● Interpretability ● Evidence ● Orthogonality For fashion data especially, this is hard!
- 48. Creating features for the human classifier A / B Ultimately, this is an empirical question Run experiments with ● Production systems ● Simulations for human classifiers
- 49. Personal styling recommendations with humans in the loop Fashion with humans in the loop: It works really well, but it’s complicated Lesson 1: You have more than one feedback loop Lesson 2: Human selection changes your objective function Lesson 3: Even humans need feature selection