Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Bringing Science to the Art of Retail SM
Modeling Choice with
Limited Data
Vivek Farias, CTO @ Celect
Robert N. Noyce Prof...
An Important Question
©2015 Celect, Inc. All Rights Reserved.2
An Important Question
Comparisons are eas(ier) …
… but also, comparisons are everywhere.
©2015 Celect, Inc. All Rights Res...
Comparisons Are Everywhere
©2015 Celect, Inc. All Rights Reserved.4
Traditional Reviews: Peter prefers Tamarind to Cafe Bo...
Comparisons Are Everywhere
5
Click Stream
Data©2015 Celect, Inc. All Rights Reserved.
Comparisons Are Everywhere
6
Click Stream Data: Vinay is more in the mood for The
Expendables 3 vs. Cosmos or The Bank Job...
Comparisons Are Everywhere
©2015 Celect, Inc. All Rights Reserved.7
(Brick and Mortar) Retail Data
Comparisons Are Everywhere
8
(Brick and Mortar) Retail Data: I prefer Neutrogena to Dove,
and Aveeno, and …
©2015 Celect, ...
Two Applications
• Helping a department store build “hyper-local” experiences
‒ Fully data driven approach to modeling cho...
A Mental Picture Of A Choice Model
10
Representative Customer
©2015 Celect, Inc. All Rights Reserved.
A Generic Choice Model
11
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.
A Generic Choice Model
12
= 30+30 = 60%
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.
A Generic Choice Model
13
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.
A Generic Choice Model
14
= 30+30+20 = 80%
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.
A Generic Choice Model
15
$200
$300
$100
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.
A Generic Choice Model
16
$100 $100$200 $200 $300 $300
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.
A Generic Choice Model
17
30% 30% 7% 7% 20% 6%
$100 $100$200 $200 $300 $300
©2015 Celect, Inc. All Rights Reserved.
$200 $...
A Generic Choice Model
18
$200 $300$200 $200 $300 $300
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.
Revenue Estimation
19
?
Challenge: How to deal with sparse data & a high dim model?
Traditional Approach: Assume comes fro...
Revenue Estimation
20
Challenge: How to deal with sparse data & a high dim model?
Our Approach: Assume that comes from a s...
Our Approach
21
“Simple” “Conservative”
©2015 Celect, Inc. All Rights Reserved.
What do we know?
Case Study: Choice Modeling + Assortments
• Mid-Size Retailer (~$4B in revenue. Close to 300 stores)
• Task
‒ Optimize ass...
Case Study: Choice Modeling + Assortments
©2015 Celect, Inc. All Rights Reserved.23
Test Stores Control Stores
08/03 - 09/...
Case Study: Choice Modeling + Assortments
©2015 Celect, Inc. All Rights Reserved.24
Test Stores Control Stores
08/03 - 09/...
Case Study: Choice Modeling + Assortments
©2015 Celect, Inc. All Rights Reserved.25
5.71% -2.91%
-.54% -2.51%
Test Stores ...
Case Study: Choice Modeling + Assortments
©2015 Celect, Inc. All Rights Reserved.26
5.71% -2.91%
-.54% -2.51%
Test Stores ...
Case Study: Choice Modeling + Assortments
27
5.71% -2.91%
-.54% -2.51%
Test Stores Control Stores
08/03 - 09/27
06/15 - 07...
Case Study Summary
• Celect modifiers were applied to a limited portion of each test
store
• Adjusting against control sto...
Personalization
29 ©2015 Celect, Inc. All Rights Reserved.
“If you liked this then you will also like these other things”
Personalization
30
“If you liked this then you will also like these other things”
• What does it mean to personalize onlin...
Personalization
31
“If you liked this then you will also like these other things”
• What does it mean to personalize onlin...
Implementation at a Gigantic Retailer
• Retailer (~$36B in revenue. ~ 10 Monthly Uniques)
• Task
‒ Personalized offerings ...
Implementation at a Gigantic Retailer
©2015 Celect, Inc. All Rights Reserved.33
Category Incumbents Celect Increase
Applia...
In Summary
• Understanding choice is valuable
• We can model choice by learning from atomic comparisons
‒ Comparisons are ...
A Top Strategic Initiative
"Finally, our localization initiative is important to our growth plan as we look to offer
incre...
Thank you!
©2015 Celect, Inc. All Rights Reserved.36
Live Webcast – October 15th at 1pm ET
www.celect.com/screaming-data-w...
Upcoming SlideShare
Loading in …5
×

Modeling Choice with Limited Data - Celect at Strata + Hadoop World 2015

1,064 views

Published on

In these slides, we will walk through an innovative new approach to machine learning that seeks to model and learn customer choice patterns and preferences from sparse transactional data.

Published in: Technology
  • Login to see the comments

  • Be the first to like this

Modeling Choice with Limited Data - Celect at Strata + Hadoop World 2015

  1. 1. Bringing Science to the Art of Retail SM Modeling Choice with Limited Data Vivek Farias, CTO @ Celect Robert N. Noyce Professor @ MIT
  2. 2. An Important Question ©2015 Celect, Inc. All Rights Reserved.2
  3. 3. An Important Question Comparisons are eas(ier) … … but also, comparisons are everywhere. ©2015 Celect, Inc. All Rights Reserved.3
  4. 4. Comparisons Are Everywhere ©2015 Celect, Inc. All Rights Reserved.4 Traditional Reviews: Peter prefers Tamarind to Cafe Boulud Peter G. Peter G.’s Profile About Peter G. Peter G.
  5. 5. Comparisons Are Everywhere 5 Click Stream Data©2015 Celect, Inc. All Rights Reserved.
  6. 6. Comparisons Are Everywhere 6 Click Stream Data: Vinay is more in the mood for The Expendables 3 vs. Cosmos or The Bank Job or ... ? ©2015 Celect, Inc. All Rights Reserved.
  7. 7. Comparisons Are Everywhere ©2015 Celect, Inc. All Rights Reserved.7 (Brick and Mortar) Retail Data
  8. 8. Comparisons Are Everywhere 8 (Brick and Mortar) Retail Data: I prefer Neutrogena to Dove, and Aveeno, and … ©2015 Celect, Inc. All Rights Reserved.
  9. 9. Two Applications • Helping a department store build “hyper-local” experiences ‒ Fully data driven approach to modeling choice • Helping a top 10 US Retail website personalize to intent ‒ A new view of what personalization online means 9 “Harvesting and Learning From Comparisons” ©2015 Celect, Inc. All Rights Reserved.
  10. 10. A Mental Picture Of A Choice Model 10 Representative Customer ©2015 Celect, Inc. All Rights Reserved.
  11. 11. A Generic Choice Model 11 30% 30% 7% 7% 20% 6% ©2015 Celect, Inc. All Rights Reserved.
  12. 12. A Generic Choice Model 12 = 30+30 = 60% 30% 30% 7% 7% 20% 6% ©2015 Celect, Inc. All Rights Reserved.
  13. 13. A Generic Choice Model 13 30% 30% 7% 7% 20% 6% ©2015 Celect, Inc. All Rights Reserved.
  14. 14. A Generic Choice Model 14 = 30+30+20 = 80% 30% 30% 7% 7% 20% 6% ©2015 Celect, Inc. All Rights Reserved.
  15. 15. A Generic Choice Model 15 $200 $300 $100 30% 30% 7% 7% 20% 6% ©2015 Celect, Inc. All Rights Reserved.
  16. 16. A Generic Choice Model 16 $100 $100$200 $200 $300 $300 30% 30% 7% 7% 20% 6% ©2015 Celect, Inc. All Rights Reserved.
  17. 17. A Generic Choice Model 17 30% 30% 7% 7% 20% 6% $100 $100$200 $200 $300 $300 ©2015 Celect, Inc. All Rights Reserved. $200 $300
  18. 18. A Generic Choice Model 18 $200 $300$200 $200 $300 $300 30% 30% 7% 7% 20% 6% ©2015 Celect, Inc. All Rights Reserved.
  19. 19. Revenue Estimation 19 ? Challenge: How to deal with sparse data & a high dim model? Traditional Approach: Assume comes from a parametric family ©2015 Celect, Inc. All Rights Reserved. Generic Model Limited Data Predicted Rev Rate
  20. 20. Revenue Estimation 20 Challenge: How to deal with sparse data & a high dim model? Our Approach: Assume that comes from a sparse model ©2015 Celect, Inc. All Rights Reserved. Generic Model Limited Data ?Predicted Rev Rate
  21. 21. Our Approach 21 “Simple” “Conservative” ©2015 Celect, Inc. All Rights Reserved. What do we know?
  22. 22. Case Study: Choice Modeling + Assortments • Mid-Size Retailer (~$4B in revenue. Close to 300 stores) • Task ‒ Optimize assortment using our approach to learn choice patterns at individual stores • Data ‒ In-store inventory + transactions over time ‒ Online browse/ transactions • Controlled experiment at 10 representative stores (Q3 14) 22 ©2015 Celect, Inc. All Rights Reserved.
  23. 23. Case Study: Choice Modeling + Assortments ©2015 Celect, Inc. All Rights Reserved.23 Test Stores Control Stores 08/03 - 09/27 06/15 - 07/26 Test Period
  24. 24. Case Study: Choice Modeling + Assortments ©2015 Celect, Inc. All Rights Reserved.24 Test Stores Control Stores 08/03 - 09/27 06/15 - 07/26 Trailing 6 Weeks
  25. 25. Case Study: Choice Modeling + Assortments ©2015 Celect, Inc. All Rights Reserved.25 5.71% -2.91% -.54% -2.51% Test Stores Control Stores 08/03 - 09/27 06/15 - 07/26 Test Stores Net Relative 5.71 - -0.54 = 6.25%
  26. 26. Case Study: Choice Modeling + Assortments ©2015 Celect, Inc. All Rights Reserved.26 5.71% -2.91% -.54% -2.51% Test Stores Control Stores 08/03 - 09/27 06/15 - 07/26 Control Stores Net Relative -2.91 - -2.51 = -0.40%
  27. 27. Case Study: Choice Modeling + Assortments 27 5.71% -2.91% -.54% -2.51% Test Stores Control Stores 08/03 - 09/27 06/15 - 07/26 Adjusted Net Relative 6.25 - -0.40 = +6.65% ©2015 Celect, Inc. All Rights Reserved.
  28. 28. Case Study Summary • Celect modifiers were applied to a limited portion of each test store • Adjusting against control stores and for trailing 6-wk trends: ‒ Celect modified departments yielded an incremental 6.65% increase in revenue ‒ We predicted 6.43%, so our performance is predictable • Test stores in total yielded an incremental 2.98% increase in revenue potentially due to positive spillover effects of a better assorted store ©2015 Celect, Inc. All Rights Reserved.28
  29. 29. Personalization 29 ©2015 Celect, Inc. All Rights Reserved. “If you liked this then you will also like these other things”
  30. 30. Personalization 30 “If you liked this then you will also like these other things” • What does it mean to personalize online? • Explosion of ad-hoc heuristics -- tailor made to application • All have a ‘nearest neighbors’ flavor • How does choice play a role? ©2015 Celect, Inc. All Rights Reserved.
  31. 31. Personalization 31 “If you liked this then you will also like these other things” • What does it mean to personalize online? • Explosion of ad-hoc heuristics -- tailor made to application • All have a ‘nearest neighbors’ flavor • Our Approach: Build a choice model, and optimize assuming choice ©2015 Celect, Inc. All Rights Reserved.
  32. 32. Implementation at a Gigantic Retailer • Retailer (~$36B in revenue. ~ 10 Monthly Uniques) • Task ‒ Personalized offerings for search and checkout ‒ < 5 ms RTT tolerance • Data ‒ Online browse/ transactions ‒ CRM • Live on traffic ©2015 Celect, Inc. All Rights Reserved.32
  33. 33. Implementation at a Gigantic Retailer ©2015 Celect, Inc. All Rights Reserved.33 Category Incumbents Celect Increase Appliances $4.38 $5.57 27.17% Clothing $0.64 $0.77 20.31% Lawn & Garden $3.26 $4.50 38.04% Tools $1.36 $1.56 14.71% Toys $0.41 $0.67 63.41% Outdoor Living $4.49 $5.03 12.03% Total $2.86 $3.43 19.93% 20% increase in monetizing site traffic. Distinct incumbents in each category
  34. 34. In Summary • Understanding choice is valuable • We can model choice by learning from atomic comparisons ‒ Comparisons are everywhere • Data-Driven Choice modeling can re-shape offline retail functions ‒ Hyper-Localized Buying, Planning and Allocation • Data-Driven Choice modeling can re-shape personalization ‒ Understanding choice beyond nearest-neighbor algorithms 34 ©2015 Celect, Inc. All Rights Reserved.
  35. 35. A Top Strategic Initiative "Finally, our localization initiative is important to our growth plan as we look to offer increasingly local-centric assortments. We are testing new software called Celect, a predictive modeling tool that allows us to leverage customers' omnichannel purchase and browse behavior to identify merchandise localization opportunities. We believe that localization can allow us to further increase market share in our low-volume doors.” Kathy Bufano, CEO, Bon-Ton Stores BONT Earnings Call, May 21, 2015 ©2015 Celect, Inc. All Rights Reserved.35
  36. 36. Thank you! ©2015 Celect, Inc. All Rights Reserved.36 Live Webcast – October 15th at 1pm ET www.celect.com/screaming-data-webcast Visit blog.celect.com Follow us on Twitter! @celect

×