Macy's uses advanced analytics to improve the customer experience online and in stores. Their team builds predictive models to target customers with relevant products and information. Some of their projects include customer acquisition and retention modeling, personalized site recommendations, response propensity models, and using big data to better understand customer behavior. Their goal is to provide each customer with the best possible omnichannel shopping experience.

1. Macys.com Advanced Analytics
PAW for Business
San Francisco
May 14-18, 2016
Advanced Analytics at Macys.com

2. Agenda
 Macy’s customer centric strategy
 Big data analytics and traditional analytics
 Challenges and solutions of big data predictive
modeling
 Macy’s Advanced Analytics Team
 Our analytics projects
 Personalized site recommendations
 Response propensity models
 Best practices of analysts and modeling
2

3. Macy’s Customer Centric Strategy
Macy’s provides our customers the best omnichannel shopping
experience online and in stores
• Customers may shop online and pick up, return or exchange in stores
• Customers may also visit stores and for items not in stock, an associate
may help order online with free shipping
On our site and in our emails, we provide relevant product and
information to our customers
Macy’s makes extensive and comprehensive efforts to protect
customer privacy and identity information

4. Big data and new sciences
 Physicist Richard Feynman said, 30+ years ago, basically,
Social sciences are not a science. They got no Laws.
Social scientists may say food grown with organic fertilizer is better
for you. It may or may not be true. They don’t know.
1980: Social Sciences ≠ Science
 Now Big Data change every field in sciences, and even more in social
sciences, many times over
 We still have no laws in many instances, but we have a new paradigm
More data will make better predictions, by doing crowd sourcing
“All models are wrong, some are useful “ -- George Box
 All Laws are approximate, like stereotypes, have biases, but data never lie
 We now know a lot more in social sciences, and we can test it
 2016: Science Driven by Big Data IS the new science
Complex, Nonlinear, Large Dimensions, Sparse data, Probabilistic, Intuition
4

5. Modeling in the Big Data era
 Challenges:
 Modeling needs to scale
 Timeliness of models
 It takes time to integrate
 Test and Experimentation
 Solutions:
 Big data warehouse solutions
 Separation of concerns
 Scalable modeling tools
 Best practices in modeling
5

6. Separation of concerns
• Solution complexity
• Data complexity
• Variability of requirements
• Standard data mining algorithms
• Availability
• Reliability, scalability, latency
• CPU/GPU, RAM, NIC and
disk IO issues
6
Platform
Engineering
Data
Science

7. Best practice big data modeling
 Understand the problem domain deeply
 Understand how the data are collected, what data can
and cannot be collected
 Balance cost of collecting data and optimize modeling
 Model performance depends on quality of data
 Use automated, robust model building solutions
 Use feedback loop to test hypotheses
 Do simulations to see if changes are reasonable
 Good ideas are not necessarily complicated
 Focus on domain knowledge,
not just data mining tools
 Good models depend on test and optimization
 Willing to learn new things
7

8. Macy.com’s Advanced Analytics
 We are at the frontiers of Big Data science
 We improve customer experience at the customer level
 Our team members have very strong background in
 Quantitative fields, math, stat, physics, bioinformatics, decision
sciences, and computer science
 We collaborate with systems and IT teams internally as well as 3rd party
vendors like SAS Research, IBM Research…
 We use a wide range of tools Hadoop, SAS, SAP, H2O, R,
Mahout, and others
 We are data scientists with keen focus on domain problems
8

9. Customer acquisition and retention
 Targeting the right message to the right customer at the right
time
 Both about data and the customer, real time vs long term, sparse
data, seasonality and changing inventory
 Build predictive models of purchase behavior and identify
drivers
 Customer retention modeling and identify drivers
 Site recommendation algorithms
 Most work is in batch mode, expanding slowly into real time
 Rapid-prototyping and testing of algorithms and policies
 Output of the team’s work support other marketing teams to
identify, and reach best customers
9

10. Some other projects
 Data organization or data munging
 Data collections, individual and event level, 360 degrees, …
 Segmentation of customers
 Multiple channel attribution
 Customer Lifetime Value
 Monitor and optimize on long term value instead of session
conversions
 Customer Lifecycle Analysis
 New customer on boarding
 Forecast and optimization
 Prediction, simulation, and search and optimize
 Deep Learning modeling
 Image recognition
 Wide and Deep behavioral models
10

11. Macys.com’s Email and Site personalization
11

12. Customer segmentation
12
Demographic
Socio-economic
Behavioral
Values and styles
Channels
Brand

13. Product social network
13
Demographic
Style
Size
Brand
Price range
Season

14. Text mining for similarity
Augment data
Differentiation
Generalization
1
4

15. Visual Similarity from Deep Learning Modeling
Recommend visually similar items, generate product attributes by look

16. Who gets which email?
16

17. Propensity Models
17
We are building an expanding family of models, at
Category/Brand/Outfit…
If customer, by category, bought Online/Store, browsed,
A2B, Email open/clicks, gender, age, store proximity,
Personicx, recency/frequency/spent…
Predict buy
Men’s
Clothes ?
Next 4
weeksUp to two years observation

18. H2O Modeling Platform
• H2O in-memory map reduce and custom compressions
• H2O Demo: 100 million rows and 50 columns, logistic
regression on a 16 node cluster, finishes in 11 seconds
• A cluster of 20 machines
• Customer sample of 7 million, 4000+ columns
• Train 100 models, R interface, in two days
• Model accuracy by ROC AUC distributions, robustness
• Score all customers in minutes

19. Customer behavioral models
Insights and predictive models and decisions
Easy to model popular products
Hard to model long tail niche products
Balance trade offs between
increasing personalization
and reducing noise
Model decisioning using
business constraints
and linear programming
• Strength and Weakness

20. Model Complexity and Model Arbitration
Some predictive models may be more accurate, for example, customers
already buying in the same category
Customer lifetime value model, not only revenue but also expense, retention,
frequency of visits, engagement, omnichannel, devices, ratios, etc.
Causation, correlation and association
Model arbitration, decision rules under varying model accuracy
Contents and Levels of Personalization

21. Considerations of Recommendations
• Data sources
Data completeness, timeliness, data organization,
customer identification
• Define similarity
There are many ways to define similarity
• Automated, real time test and optimization
• Cross sell and up sell
• Repetition
Freshness vs burn into memory

22. Considerations of Recommendations
• Social needs, aspirations
• Trend, fashionable, hot, cool
• Be respected, accepted
• Explore, learn, serendipity
• Risk taking, find out self
• Less familiar
• Looking for guidance, willing to
invest and be influenced
• Trust, authority
• Acquisition modeling
• Hit and miss, nurture, branding
• Intrinsic needs
• Character, belief, value
• Habits, belong
• Style, long term taste, theme
• Familiar, known
• Looking for value, convenience,
deal, services
• Trust and service
• Easier to build a profile, preference
matching

23. Concluding thoughts
 Big data presents big opportunity and big challenges
 Data science is not about data, but domain solutions
 Modeling in Big Data era is different from traditional
practices
 Resolution, robustness, turn around time, timeliness
 Organizations need to adapt to model based decisions
 Data are not clean until thoroughly analyzed
 Scalable, efficient and automated modeling, deployment
and testing tools are essential
 There is no silver bullet, take a portfolio approach
 “If you want to have good ideas you must
have many ideas. Most of them will be wrong,
and what you have to learn is which ones
to throw away”
-- Linus Pauling
23