The document outlines a webinar presented by Keyrus and Alteryx focusing on leveraging machine learning and data analytics to enhance marketing ROI in retail and consumer packaged goods sectors. It covers topics including customer segmentation, marketing strategies, and the Alteryx analytics platform, emphasizing the importance of advanced analytics in optimizing marketing spend and improving decision-making. The presentation advocates for employing modern data science techniques over traditional methods to achieve better insight and results in marketing efforts.

2. © Copyright 2018 – Keyrus 2
Alter the Way You Feel About Analytics
Using Machine Learning to Understand and
Predict Marketing ROI
A Keyrus + Alteryx Webinar
Your Speakers:
Razvan Nistor, Head of Data Science, Keyrus US
razvan.nistor@keyrus.us
Scott Trauthen, Channel Marketing Director, Alteryx
strauthen@alteryx.com

3. © Copyright 2018 – Keyrus 3
AGENDA
• The Time is Now!!
• The Alteryx Analytics Platform
• Retail and CPG Overview
• Who are my customers?
• What’s the best way to reach them?
• How can we deliver the insights to the decision makers?

4. © Copyright 2018 – Keyrus 4
Our mission is to help customers
We pursue our mission by providing a full stack
of Data Intelligence services and solutions
Data Engineering
Big Data Solutions
Data Architecture
Real time ingestion
Data connectivity and integration
Master Data Management
Data Quality Management
Data Discovery
Enterprise BI
Exploration and
Visualization
Management Dashboards
KPIs and Scorecards
Self Service BI
Custom UI solutions
Data Science
Data Science consulting
Machine Learning
Predictive Analytics
Data Driven Innovation
Data Science boot camp
Products & Solutions
Quilliup
Rivery
Customer 360
Python Framework
Java Framework
Management &
Transformation
Strategy and innovation
Digital transformation
Performance management
Project support
Change management

5. © Copyright 2018 – Keyrus 5
THE BIG PICTURE
TIME IS NOW!!
STRATEGIC PROBLEM SOLVING
 150M web results for AI in Marketing

6. © Copyright 2018 – Keyrus 6
THE (TYPICAL) PLAN:

7. © Copyright 2018 – Keyrus 7
THE TIME IS NOW!!!

8. © Copyright 2018 – Keyrus 8
X
X
X
SOLVE SPECIFIC PROBLEMS!!!

9. © Copyright 2018 – Keyrus 9
• The time is now to deliver on the big-picture strategic initiatives
Process Optimization Return on InvestmentsConsumer Insights
• Supply Chain Optimization
• Better forecasting
• Inventory Management
• Manufacturing Analysis
• Process Optimization
• Who are my customers?
• Where are they located?
• Account targeting
• Industry trends
• What are they saying on
social media?
• Marketing campaign returns
• Trade marketing strategies
• Displays/Sampling impact
• Pricing strategy impact
BIG-PICTURE RETAIL/CPG QUESTIONS:

10. © Copyright 2018 – Keyrus 10
DATA SCIENCE OVERVIEW
WHAT IS DATA SCIENCE / MACHINE LEARNING / AI ?
DO YOU NEED ADVANCED MODELS!? THIS IS THE MILLION DOLLAR QUESTION
DATA SCIENCE PROJECTS AND WHO CAN DO THEM?

11. © Copyright 2018 – Keyrus 11
What is Data Science?
• Correlations & Statistics
• Forecasting
• Clustering & Segmentation
• Location Analytics & Demographics
• Natural Language Processing
• Optimization Algorithms
• Machine Learning
The art of using science to solve business problems

12. © Copyright 2018 – Keyrus 12
CATEGORIES OF ‘DATA SCIENCE’
Advanced
Analytics
Adv Statistics
Correlation Analysis
Regression
Forecasting
Clustering &
Segmentation
Similarity Analysis
Nearest Neighbors
/ Clusters
Demographics
Analysis
Machine
Learning
Supervised
Learning
Algorithms
Categorical
Classifiers
Advanced
Regression
Recommendation
Systems
Algorithms: Logistic Regression, Naïve Bayes,
Decision Trees, Random Forests, Boosted Trees,
Neural Networks and etc…
Optimization
Algorithms
Basic Simplex
Min/Max, Gradient
Descent
Stochastic Gradient,
Monte Carlo
Advanced Heuristic
Optimizers:
Simulated
Annealing or
Genetic Algorithm
Deep Learning
Convolutional
Neural Networks
Recurrent Neural
Networks
State of the art
Deep Learning
architectures
AI
Reinforcement
Learning Paradigms
Policy Gradients,
Actor-Critic, Q-
Learning
The Greatest IP of
our Time
Complexity & Hype
All of these methods result in a ‘model’
(method + fitted parameters) that takes
input data and outputs predictions
Today’s
Discussion

13. © Copyright 2018 – Keyrus 13
Advanced statistical algorithms that find the best way to map a set of inputs to a set of outputs
( This is what most people mean when they say ‘model’ )
WHAT IS MACHINE LEARNING?
• y ~ sqrt(x) ?
• y ~ x^(1/4) ?
• y ~ ln(x) + C ?
• ML will find the best way to describe
the relationship without doing the
symbolic regression… you just won’t
know what that function actually is
x
y

14. © Copyright 2018 – Keyrus 14
Data Science Project Workflow:
1. Problem Definition and Buy-In
2. Data Exploration and Project Design
3. Data ETL and cleansing
4. Data enhancement, imputation, feature engineering
5. Data training/testing/validation sets
6. Build Models on training data
7. Calibrate all models and methods
8. Validate model performance on out-of-sample data
9. Model analysis and feature impact
10. Model selection and productionization for predictions
11. Analysis framework development (App/API)
12. Scoring and insights generation
13. Communication, Training, and Adoption
14. Maintenance and refitting/retraining
#DoUml?WHO CAN DO MACHINE LEARNING?

15. © Copyright 2018 – Keyrus 15
WHAT DOES A DATA SCIENCE PROJECT LOOK LIKE IN REAL LIFE?

16. © Copyright 2018 – Keyrus 16
DATA SCIENCE DELIVERABLES: WHAT DO YOU GET?
• Depends on the Problem / People (roles) / Processes / Technologies in place
• Can be any number of ways of delivering insights in a way that integrates fully with existing systems
and ways of working in the organization:
Code Research Integrated
Dashboard
Integrated
Application
+ Enablement in the latest processes and tools

17. © Copyright 2018 – Keyrus 17
UNIFY YOUR ANALYTIC EXPERIENCE AND BREAK THE BARRIERS TO FASTER INSIGHTS
THE ALTERYX ANALYTICS PLATFORM
Unlock your analysts’ true potential

18. D I S C O V E R
+ S H A R E
P R E P +
B L E N D
A N A L Y Z E
+ M O D E L
D E P L O Y +
M A N A G E
D A T A S C I E N C E & A N A L Y T I C S C U L T U R E
C O M M U N I T Y
©2018 Alteryx, Inc. 18

19. the only quick-to-implement, self-service
data analytics platform that allows data
scientists & citizen users alike to break the
barriers to insight, so everyone can experience
the thrill of getting to the answer faster.
LikeNoOther.
Data Prep & Blending is the Foundation
For All Levels of Analytics
© 2017 Alteryx, Inc.

20. Oracle
Oracle
SAS
Input Output
Enrich
Prep and Blend Analyze Share
The Next GenerationAnalytics Platform Everyone isTalking About
CONNECT | DESIGNER | SERVER | GALLERY | PROMOTE

21. CODE-FREE ANALYTICS
for the citizen data scientist
Data Science for the Masses
• Broad range of preconfigured predictive models
• Complete toolset for spatial analytics
• Leverage models from data scientists
All Purpose Data Workbench
• Drag-and-drop UI for workflow creation
• Prep, blend and analyze for most any use case
• 250+ tools for wide array of data work
• Simple yet sophisticated tool configuration
• Global search for community support

22. CODE-FRIENDLY ANALYTICS
for the trained data scientist
High Performance for Big Data
• In-DB platform support
• Spark
Breadth of Algorithmic Support through
API
• R tool
• Python and Scala support
• Guide to creating R based Alteryx tools
© 2017 Alteryx, Inc.

23. © Copyright 2018 – Keyrus 23
THIS IS WHAT I CALL A TARGET RICH ENVIRONMENT (RICH WITH DATA, I MEAN)
THE RETAIL AND CPG SECTORS:

24. © Copyright 2018 – Keyrus 24
DATA ASSETS ALONG THE SUPPLY CHAIN TO THE CONSUMER
Manufacturer
• Creates or
Imports the
products
• Transactional
business
tracking
Shipments
Distributer
• Buys the goods
• Holds in
warehouses
• Delivers to
customers /
clients
• Service several
clients
Account
• Buys goods
and sells at
margin
• Products
compete for
shelf space
• Trade
marketing /
PoS items
Consumer
• Purchases and
consumes the
products
• Influenced by
Brand
Awareness
• Social media
• Consumer
Demographics
Sales
Databases
Operations
System
Finance
System
Manufacturing
Excels
Depletions
Competitor Advertising
Digital
Marketing
Trade
Marketing
Geographical
Demographics
Point of Sale
Scantrack
Recall
Brand Health
Social Media
Segmentation

25. © Copyright 2018 – Keyrus 25
• I’m a large scale CPG company competing in a very competitive sector
• I want to understand who my customers are and what marketing campaigns have the biggest
impact on sales returns
• Why? Because we spend +$100M on Advertisements and Promotions
• I need to start doing this now because my competitors have already started
• I will use the latest techniques to tackle this problem
Go for it, but be specific
SOLVE STRATEGIC,
BIG-PICTURE PROBLEMS:
 150M web results for AI in Marketing

26. © Copyright 2018 – Keyrus 26
CUSTOMER SEGMENTATION: WHO ARE MY CUSTOMERS? WHERE ARE THEY?
• Tie Demographics data to sales data
• Identify regions of highest growth / highest volume per capita / highest volume markets
• Profile these regions by MOSAIC Consumer Groups & Demographics
• Advertise to these regions and consumers

27. © Copyright 2018 – Keyrus 27
CUSTOMER SEGMENTATION: WHO ARE MY CUSTOMERS? WHERE ARE THEY?
Takeaways:
My customers span several Consumer Groups
growing at different rates:
• Aging of Aquarius: Maintain
• Steadfast Conventionalists: Win-back
• Striving Single Scene: Go-After
These groups are located across the country
in several targeted DMAs
Their Demographic makeup contains a large
portion of Hispanic / Latino Population

28. © Copyright 2018 – Keyrus 28
• Using a model to optimize marketing spends (ATL/BTL) and yield the highest returns
TotalSales
60%
10%
25%
Model
MARKETING MIX: WHAT’S THE BEST WAY TO REACH MY CUSTOMERS

29. © Copyright 2018 – Keyrus 29
MARKETING MIX: WHAT’S THE BEST WAY TO REACH MY CUSTOMERS
• Combine Marketing Spend with Sales data for markets of interest
• Build model(s) to attribute ROI to various marketing channels
• Perform What-If analysis or Optimize the Spend!

30. © Copyright 2018 – Keyrus 30
TRADITIONAL LINEAR MULTILEVEL MODELS
But real life is nonlinear
Spend
Return

31. © Copyright 2018 – Keyrus 31
MACHINE LEARNING REGRESSION MODELS:
Decision Trees: Random Forests:
Single (conditional) Tree:
Set of weighted if-else
statements
Ensemble of Trees:
Boostrap a collection of
trees and weight best
outcomes
Neural Networks:
Ensemble of computing nodes:
Interconnected elements that
process information by their
response to external inputs

32. © Copyright 2018 – Keyrus 32
How good is good enough? Do you really need bleeding-edge models?
(This is the Million dollar question)
ACTUAL VALUES ACTUAL VALUES
PREDICTEDVALUES
PREDICTEDVALUES
LINEAR MODEL NEURAL NETWORK

33. © Copyright 2018 – Keyrus 33
How good is good enough?
- Nonlinear effects are real
- LM under-estimates impact by $30M
- LM over-estimates spend by $2M
80%oftotalspend
Do you really need bleeding-edge models?
Yes. You should ML instead of LM.

34. © Copyright 2018 – Keyrus 34
USE CASE: MEDIA SPENDS + SYNDICATED SALES DATA
Main Dimensions
• Dates / Products / DMAs / Media_Types
• Mainly ATL Media: Cable, Network, Spanish
Measures
• Marketing: Media Spends & GRPs
• Sales: Distribution, ACV, Price, Volumes
• Contains competitors as well!
Target Variable
• $Volume

35. © Copyright 2018 – Keyrus 35
RESULTS AND NEXT STEPS:

36. © Copyright 2018 – Keyrus 36
MODEL OUTPUTS:

37. © Copyright 2018 – Keyrus 37
SALES PERCENTAGE ATTRIBUTION:

38. © Copyright 2018 – Keyrus 38
OPTIMIZED MARKETING SPEND – 13.4% INCREASE IN SALES:
• Use heuristic optimizers to find
optimum solution and spend
allocation
• Ex.: Monte Carlo / Simulated
Annealing / Genetic Algorithm
 Or simple grid search if it works!!
• Suited to optimizing complex
problems like Airline Scheduling
and Traveling Salesman
**Optimized for a given brand:

39. © Copyright 2018 – Keyrus 39
SUMMARY:
• Alteryx Data Pack allows for easy customer segmentation and market analysis
• Marketing Mix using ML Driven Regression Models are better able to capture nonlinearities in the data
• Model Optimization predicts increased sales by adjusting spend in marketing channels
• Scalable and re-useable ML engine:
 Easy to add additional features to input data: Add BTL & Trade Marketing channel spends
 Apply know-how to pricing analysis (what price vs competitor should I use?)
 Integrated into Marketing Team day-to-day process management

40. © Copyright 2018 – Keyrus 40
Q & A
THANK-YOU!!

41. © Copyright 2018 – Keyrus 41
FOCUSED ON UNLOCKING BUSINESS VALUE USING ADVANCED ANALYTICS
APPENDIX: OTHER RELEVANT INDUSTRY USE CASES:

42. 42© Copyright 2018 – Keyrus
Context
A large spirits company would like to understand how their products sell together to
offer up better value packs during key months
Approach and key success factors
● Use correlations and heatmaps to understand trends in time
● Integrate into the dashboards
Data Sources
● Internal sales data + VIP depletions
● NABCA market data
Challenge
Getting all the data in one place and slow IT
Benefits
● Help Marketing teams build value-add stories
● Useful insights into constructing value packs for different markets
● Suggest sales drivers in distributor calls
● New insights on product associations and positioning of brands
● Fully integrated into existing dashboards & systems
Big Data Analytics – Big Data Analytics to understand sales trends
Date
2017
Assignment length
1.5 FTEs for 1 month for
an integrated dashboard
Technolog(y)(ies)
Power BI, Python, R, AWS
CORRELATION&STATISTICALANALYSIS
SPIRITS MARKET ANALYSIS

43. 43© Copyright 2018 – Keyrus
Context
A large beer company would like to understand inventory levels at their
distributors: SHIPS - DEPS
Approach and key success factors
● Use the latest forecasting algorithms to give market managers the ability
to predict inventory levels (Ships – Deps) during key future sales months
● Integrate into the dashboards or provide a user interface
Data Sources
● Internal sales data + VIP depletions
Challenge
Providing a useable UI and UX app
Benefits
● Provide sales teams with visibility into inventory levels from ALL their
distributors and products of interest
● Monitor low stock situations and send alerts for re-ordering
● Use trends in time as discussion points in distributor conversations
● Provide easy to access app or interface
● Take inventory report direct to Distributor
Big Data Analytics – Understand inventory levels
Date
2016
Assignment length
1.5 FTEs for 3 months for
fully integrated solution
Technolog(y)(ies)
Alteryx, Qlik, R, AWS
SALESFORECASTING
SUPPLY CHAIN FORECASTING

44. 44© Copyright 2018 – Keyrus
Context
Connect with the consumer in order to better understand the customer
journey, customer preferences and sentiments. Drive product innovation based
on customer insights and big data analytics.
Approach and key success factors
● Sentiment analysis of social media, blogs and other online consumer
perceptions of specific ingredients of alcoholic or non- alcoholic beverages
● Dashboard and reporting of analysis to support daily operations (worldwide)
● Collect, structure and analyze external surveys to better define consumer
journey, and usage attitudes related to alcoholic and non-alcoholic occasions
Data Sources
● Social media feeds
● Industry specific websites & blogs
Challenge
Combining data discovery capabilities and analytical processing of large
amounts of multi-structured external and internal data (big data).
Benefits
● Deployment of an intuitive dashboard, facilitating adoption of analytics
amongst the business community
● Extended capabilities to explore new fields of data that will drive product
innovation and enhancement
● New insights on product associations and positioning of brands
Big Data Analytics – Understand consumer sentiment
Date
2016
Assignment length
3 FTEs for 6+ months
Technolog(y)(ies)
Tableau, Python, R,
AWS, Mechanical
Turk
SOCIALMEDIA/DEMOGRAPHICSANALYSIS
LARGE BREWER: SENTIMENT

45. 45© Copyright 2018 – Keyrus
Context
Use Experian Mosaic data, Census data, and Experian credit data to understand
WHO our customers are, and WHERE are they located?
Approach and key success factors
● Which accounts should we sell into that we are currently not accessing?
● Display on functional and intuitive dashboard
Data Sources
● Experian credit/demographics data
● Market data to understand competitor sales
● Internal sales data
Challenge
Combining the data in meaningful ways and not getting stuck in analysis-
paralysis by providing too much content on the user interface
Benefits
● Deployed intuitive location analytics
● Understand where competitors are focusing their sales efforts!
● Understand WHO are our ideal clients and where are they located
● Link our ideal client profiles to Mosaic consumer preference surveys
● Understand our customer
● Target areas where these customers are located
Big Data Analytics – Understand consumer Demographics
Date
2017
Assignment length
1.5 FTEs for 2 months
Technolog(y)(ies)
Carto, Tableau, SQL,
R, AWS
DEMOGRAPHICSANALYSIS
LARGE RETAILER DEMOGRAPHICS

46. 46© Copyright 2018 – Keyrus
Context
A large beer company wants to create a models that tie product sales to
marketing efforts to understand ROI
Approach and key success factors
● STOP using linear models from the 80’s
● Use advanced ML algorithms to capture nonlinearities and to easily combine
factors from different data sources
Data Sources
● Internal sales and depletions data
● Marketing spend data
● Finance data
Challenge
Combining the data in consolidated views and building accurate models
employing the latest Data Science techniques
Benefits
● Understand which campaigns yield the highest returns in given markets
● Model is able to support What-If analysis
● Use the latest optimization algorithms to find optimum spends
Big Data Analytics – Supply Chain Optimization using ML
Date
2016
Assignment length
1.5 FTEs for 3 months for
initial modeling
Technolog(y)(ies)
R, Python, Alteryx
AWS
MACHINELEARNING:MARKETINGROI
MARKETING-MIX MODELING

47. 47© Copyright 2018 – Keyrus
Context
A large dairy CPG company wants to create a model that predicts the cost to
serve a new product to market (ultimate what-if analysis)
Approach and key success factors
● Which product features impact the cost to serve the most?
● Provide a functional and useable system for analysts to re-use engines
Data Sources
● Internal sales and depletions data
● Operations data and packaging information + costs
● Finance data
Challenge
Combining the data in consolidated views and not overfitting the model.
Providing a USEABLE interface to excel-based analysts
Benefits
● Very quickly understand how components impact product costs
● Quickly provide Operations insights on what works and what doesn’t by
understanding cost implications for variety of pack options
● Understand best Routes-to-Market
● Bonus feature: Understand late shipments systematic issues
Big Data Analytics – Supply Chain Optimization using ML
Date
2017
Assignment length
1.5 FTEs for 3 months for
data consolidationg and
initial modeling
Technolog(y)(ies)
R, Python, Alteryx,
Tableau,
Microstrategy AWS
MACHINELEARNING:COST-TO-SERVE
PRODUCT COST-TO-SERVE

48. © Copyright 2018 – Keyrus 48
WHY KEYRUS
EXPERTISE BY
POSITIONING
Niche player with single focus on
DI and transformation
Deep understanding of customers’
pain points
Focus on embedding change in a
sustaining way
TECHNOLOGY
INDEPENDENT
Interest of the customer first,
but with privileged partnership with
all major and innovative vendors
END-TO-END
Integrated business and
technological service offering
with the ability to advice and
execute to ensure successful
outcomes
EXPERIENCE &
COMPETENCES
Balanced seniority mix
A proven track record combining
operational and consulting
experience for most consultants
PROVEN
APPROACH
Innovative & agile culture
Holistic methodological approach
capturing value across different
dimensions
Collaboration mode, co-
development and co-creation
OUR
AMBASSADORS
Proven track record of partnering
throughout our extensive satisfied
customer base
Reliable end-to-end partner that delivers

49. © Copyright 2018 – Keyrus 49
OUR GROUP'S DNA
& DIFFERENTIATORS
© Copyright 2017 – Keyrus 49
DATA “PURE PLAYER”
TECHNOLOGY
INDEPENDENT
OUR
AMBASSADORS
OUR
CONSULTANTS
LOCAL
PRESENCE
END-TO-END
CAPABILITIES
MAKING IT
HAPPEN
TOGETHER

50. © Copyright 2018 – Keyrus 50