In this keynote I will give you a business understanding of ML by going through key concepts and concrete use cases that illustrate its possibilities. I'll present new technology that makes ML more accessible, and I'll explain in simple terms the limitations to what can be achieved. Finally, I'll discuss pragmatic considerations of real-world applications and I'll give a sneak peak at the Machine Learning Canvas — a framework for describing a predictive system that uses ML to provide value to its end user.
14. –McKinsey & Co. (2011)
“A significant constraint on
realizing value from big data will
be a shortage of talent,
particularly of people with deep
expertise in statistics and machine
learning.”
34. –Katherine Barr, Partner at VC-firm MDV
"Pairing human workers with
machine learning and automation
will transform knowledge work
and unleash new levels of human
productivity and creativity."
83. • Who: SaaS company selling monthly subscription
• Question asked:“Is this customer going to leave
within 1 month?”
• Input: customer
• Output: no-churn (negative) or churn (positive)
• Data collection: history up until 1 month ago
• Baseline: if no usage for more than 15 days then
churn
85. Customer representation:
• basic info (age, income, etc.)
• usage of service (# times used app, avg time spent,
features used, etc.)
• interactions with customer support (how many,
topics of questions, satisfaction ratings)
86. Taking action to prevent churn:
• contact customers (in which order?)
• switch to different plan
• give special offer
• no action?
87. Measuring accuracy:
• #TP (we predict customer churns and he does)
• #FP (we predict customer churns but he doesn’t)
• #FN (we predict customer doesn’t churn but he does)
• Compare to baseline
88. Estimating Return On Investment:
• Taking action for #TP and #FP customers has a cost
• We earn #TP * success rate * revenue /cust. /month
• Compare to baseline
91. PREDICTIONS OBJECTIVES DATA
Context
Who will use the predictive system / who will be
affected by it? Provide some background.
Value Proposition
What are we trying to do? E.g. spend less time on
X, increase Y...
Data Sources
Where do/can we get data from? (internal
database, 3rd party API, etc.)
Problem
Question to predict answers to (in plain English)
Input (i.e. question "parameter")
Possible outputs (i.e. "answers")
Type of problem (e.g. classification, regression,
recommendation...)
Baseline
What is an alternative way of making predictions
(e.g. manual rules based on feature values)?
Performance evaluation
Domain-specific / bottom-line metrics for
monitoring performance in production
Prediction accuracy metrics (e.g. MSE if
regression; % accuracy, #FP for classification)
Offline performance evaluation method (e.g.
cross-validation or simple training/test split)
Dataset
How do we collect data (inputs and outputs)?
How many data points?
Features
Used to represent inputs and extracted from
data sources above. Group by types and
mention key features if too many to list all.
Using predictions
When do we make predictions and how many?
What is the time constraint for making those predictions?
How do we use predictions and confidence values?
Learning predictive models
When do we create/update models? With which data / how much?
What is the time constraint for creating a model?
Criteria for deploying model (e.g. minimum performance value — absolute,
relative to baseline or to previous model)
IDEASPECSDEPLOYMENT
94. PREDICTIONS OBJECTIVES DATA
BACKGROUND End-user Value prop Sources
ENGINE SPECS ML problem Perf eval Preparation
INTEGRATION Using pred Learning modelINTEGRATION Using pred Learning model
95. Why fill in ML canvas?
• Target the right problem for your company
• Choose right algorithm, infrastructure, or ML
solution
• Guide project management
• Improve team communication
98. • Need examples of inputs AND outputs
• Need enough examples
99. • ML to create value from data
• 2 phases: TRAIN and PREDICT
• Predictive APIs make it more accessible
• Good data is essential
• What do we do with predictions?
• Measure performance with accuracy, time and
bottom-line
• Also: deploy, maintain, improve…