SlideShare a Scribd company logo
Generalized	B2B	Machine	Learning	
at	Reten4on	Science	
	
	
Andrew	Waage	(andrew@reten*onscience.com)	
Co-founder	/	CTO
2012
Santa Monica
~50 (and hiring!)
AI Powered Marketing Automation Platform
Step 1
Collect Data
Step 2
Generate Predictions
Step 3
Automation Powers Intelligent
Campaigns Across Channels
Ecom / Retail Behavioral Custom Demographic
Email
Campaigns
On-Site
Display
Mobile Call
Center
What kind of Scale?
100’s Clients
210M+ Customers Tracked
1000+ Client-specific Models
2 Billion+
Predictions Daily10K+ Actions / second
“Generalized ML Platform”
What’s the
challenge?
The Challenge
•  Many Clients
•  Dirty Data
•  Sparse Datasets
•  Custom Attributes
•  Various Industries
Clean
PredictionsModel Layer!
C1	 C2	 C3	 C4
What Kind of Predictions?
Purchase Probability: High-likelihood
Lifecycle Stage: Ready to Buy
Churn Time: 300 days
Customer Future Value: $925
Contact Frequency: Every 3 days
Optimal Time to Engage: Thursday 7-9PM
Optimal Incentive/Discount: Dollars Off
Product Recommendations: Based on interest
Optimal Subject Line: Individual preference
Optimal Template: Individual preference
Our Approach & Learnings
1.  Robust Ingestion Pipeline
2.  Common Feature Engineering Layer
3.  “Plug-in” Architecture for Models
4.  Evaluation / AB Testing
5.  Robust Monitoring & Visualization
1. Robust Ingestion Pipeline
10K+ Actions Per Second
auto-scaling!
auto-scaling
lambdas!
•  Abstraction Layer: Data
Ingestion
•  Do not compromise for
clean data
•  Auto-scaling everywhere
•  High confidence in
upstream data
Flume
Kinesis
2. Common Feature Engineering Layer
•  Abstraction: Feature Layer
•  Allow custom features
•  Handle feature selection
•  Modelers know what to
expect
Raw	Data	
User	Behavior	
Features	
Product	
Features	
User	Sta4c	
Features	
Timing	Model	 CLV	Model	 Recommender
3. Model Plug-in Architecture
C6
C3
C4
C5
C7
• Plug-in Architecture
• Tune model hyper-parameters
• A/B test models per client
C1
C8 C2
Client’s Model Execution Plan
Recommender System
Multi-Layer Personalization:
•  Layer 1: ML / Algorithmic
•  ALS CF, Content-based, Item-Item
•  Layer 2: User-level Domain Logic
•  User-level predictions
(gender, location, shoe sizes)
•  Layer 3: Client-tuned Domain Logic / Controls
•  Rank by profit-margin
•  Increase discovery rate influxer
1. Algorithmic
2. User-Level
Domain Logic
3. Client-Level
Controls
4. Model Evaluation / Fast Feedback
A/B	Framework	
M1	 M2	 M3	
•  Start Simple
•  Collect feedback data
•  Skip long production cycle
•  Unbiased policy
generation is important
M1	
Campaign	Predic4ons
5. Robust Model Monitoring and Visualization
“Sauron”
(LOTR)!
Monitor,
monitor,
monitor!!
Monitor Recs: Distributions, Coverage, Diversity
REF:	hQp://sauron.rsci.co/advanced/sites/181/data_types/31
Monitoring Subject Line Bandit Models
Churn Rates, ROC Curve, Reliability Curve
Our Data Science Stack
Persistence! Pipeline / Process! Viz / Monitor! Code!
Takeaways
1.  Use abstraction layers
-  Clean / common interfaces
2.  Monitor, monitor, monitor
-  Fast feedback
3.  Start simple and keep iterating
Thank You!
andrew@retentionscience.com

More Related Content

What's hot

Snowplow at Sigfig
Snowplow at SigfigSnowplow at Sigfig
Snowplow at Sigfig
yalisassoon
 
Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...
Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...
Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...
Spark Summit
 
NextStep 2018 - OutSystems AI, #FutureOfCoding
NextStep 2018 - OutSystems AI, #FutureOfCodingNextStep 2018 - OutSystems AI, #FutureOfCoding
NextStep 2018 - OutSystems AI, #FutureOfCoding
OutSystems
 
Designing an automated and data-driven organization
Designing an automated and data-driven organizationDesigning an automated and data-driven organization
Designing an automated and data-driven organization
India Quotient
 
From Architecture to Analytics: A look at Simply Business’s data strategy
From Architecture to Analytics: A look at Simply Business’s data strategy From Architecture to Analytics: A look at Simply Business’s data strategy
From Architecture to Analytics: A look at Simply Business’s data strategy
Looker
 
Krish Swamy + Balaji Gopalakrishnan, Wells Fargo - Building a World Class Dat...
Krish Swamy + Balaji Gopalakrishnan, Wells Fargo - Building a World Class Dat...Krish Swamy + Balaji Gopalakrishnan, Wells Fargo - Building a World Class Dat...
Krish Swamy + Balaji Gopalakrishnan, Wells Fargo - Building a World Class Dat...
Sri Ambati
 
Pitch Deck - LabShare 2017
Pitch Deck - LabShare 2017Pitch Deck - LabShare 2017
Pitch Deck - LabShare 2017
LabSharegroup
 
Measure Camp Paris 2017 - Digital Analyst Job Description 2020
Measure Camp Paris 2017 - Digital Analyst Job Description 2020Measure Camp Paris 2017 - Digital Analyst Job Description 2020
Measure Camp Paris 2017 - Digital Analyst Job Description 2020
Optimal Ways
 
Tools for building the future
Tools for building the futureTools for building the future
Tools for building the future
Simon Belak
 
Image annotation for machine learning
Image annotation for machine learningImage annotation for machine learning
Image annotation for machine learning
Learning Spiral Pvt. Ltd
 
Doing data science with clojure
Doing data science with clojureDoing data science with clojure
Doing data science with clojure
Simon Belak
 
Strategy toolbox for startsups
Strategy toolbox for startsupsStrategy toolbox for startsups
Strategy toolbox for startsups
Asher Sterkin
 
Power to the People: A Stack to Empower Every User to Make Data-Driven Decisions
Power to the People: A Stack to Empower Every User to Make Data-Driven DecisionsPower to the People: A Stack to Empower Every User to Make Data-Driven Decisions
Power to the People: A Stack to Empower Every User to Make Data-Driven Decisions
Looker
 
Why is LEO CDP important for digital business ?
Why is LEO CDP important for digital business ?Why is LEO CDP important for digital business ?
Why is LEO CDP important for digital business ?
Trieu Nguyen
 
Exploratory analysis
Exploratory analysisExploratory analysis
Exploratory analysis
Simon Belak
 
GraphTour 2020 - Customer Journey with Neo4j Services
GraphTour 2020 - Customer Journey with Neo4j ServicesGraphTour 2020 - Customer Journey with Neo4j Services
GraphTour 2020 - Customer Journey with Neo4j Services
Neo4j
 
Storytelling & Interactive reporting with Power BI bookmarks and drillthrough
Storytelling & Interactive reporting with Power BI bookmarks and drillthroughStorytelling & Interactive reporting with Power BI bookmarks and drillthrough
Storytelling & Interactive reporting with Power BI bookmarks and drillthrough
Marc Lelijveld
 
Digitalisation from the back office to the factory floor
Digitalisation from the back office to the factory floorDigitalisation from the back office to the factory floor
Digitalisation from the back office to the factory floor
Stephanie Locke
 
Identifying Users Across Platforms with a Universal ID Webinar Slides
Identifying Users Across Platforms with a Universal ID Webinar SlidesIdentifying Users Across Platforms with a Universal ID Webinar Slides
Identifying Users Across Platforms with a Universal ID Webinar Slides
Looker
 
Tableau Conference 2018: Binging on Data - Enabling Analytics at Netflix
Tableau Conference 2018: Binging on Data - Enabling Analytics at NetflixTableau Conference 2018: Binging on Data - Enabling Analytics at Netflix
Tableau Conference 2018: Binging on Data - Enabling Analytics at Netflix
Blake Irvine
 

What's hot (20)

Snowplow at Sigfig
Snowplow at SigfigSnowplow at Sigfig
Snowplow at Sigfig
 
Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...
Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...
Artificial Intelligence: How Enterprises Can Crush It With Apache Spark: Keyn...
 
NextStep 2018 - OutSystems AI, #FutureOfCoding
NextStep 2018 - OutSystems AI, #FutureOfCodingNextStep 2018 - OutSystems AI, #FutureOfCoding
NextStep 2018 - OutSystems AI, #FutureOfCoding
 
Designing an automated and data-driven organization
Designing an automated and data-driven organizationDesigning an automated and data-driven organization
Designing an automated and data-driven organization
 
From Architecture to Analytics: A look at Simply Business’s data strategy
From Architecture to Analytics: A look at Simply Business’s data strategy From Architecture to Analytics: A look at Simply Business’s data strategy
From Architecture to Analytics: A look at Simply Business’s data strategy
 
Krish Swamy + Balaji Gopalakrishnan, Wells Fargo - Building a World Class Dat...
Krish Swamy + Balaji Gopalakrishnan, Wells Fargo - Building a World Class Dat...Krish Swamy + Balaji Gopalakrishnan, Wells Fargo - Building a World Class Dat...
Krish Swamy + Balaji Gopalakrishnan, Wells Fargo - Building a World Class Dat...
 
Pitch Deck - LabShare 2017
Pitch Deck - LabShare 2017Pitch Deck - LabShare 2017
Pitch Deck - LabShare 2017
 
Measure Camp Paris 2017 - Digital Analyst Job Description 2020
Measure Camp Paris 2017 - Digital Analyst Job Description 2020Measure Camp Paris 2017 - Digital Analyst Job Description 2020
Measure Camp Paris 2017 - Digital Analyst Job Description 2020
 
Tools for building the future
Tools for building the futureTools for building the future
Tools for building the future
 
Image annotation for machine learning
Image annotation for machine learningImage annotation for machine learning
Image annotation for machine learning
 
Doing data science with clojure
Doing data science with clojureDoing data science with clojure
Doing data science with clojure
 
Strategy toolbox for startsups
Strategy toolbox for startsupsStrategy toolbox for startsups
Strategy toolbox for startsups
 
Power to the People: A Stack to Empower Every User to Make Data-Driven Decisions
Power to the People: A Stack to Empower Every User to Make Data-Driven DecisionsPower to the People: A Stack to Empower Every User to Make Data-Driven Decisions
Power to the People: A Stack to Empower Every User to Make Data-Driven Decisions
 
Why is LEO CDP important for digital business ?
Why is LEO CDP important for digital business ?Why is LEO CDP important for digital business ?
Why is LEO CDP important for digital business ?
 
Exploratory analysis
Exploratory analysisExploratory analysis
Exploratory analysis
 
GraphTour 2020 - Customer Journey with Neo4j Services
GraphTour 2020 - Customer Journey with Neo4j ServicesGraphTour 2020 - Customer Journey with Neo4j Services
GraphTour 2020 - Customer Journey with Neo4j Services
 
Storytelling & Interactive reporting with Power BI bookmarks and drillthrough
Storytelling & Interactive reporting with Power BI bookmarks and drillthroughStorytelling & Interactive reporting with Power BI bookmarks and drillthrough
Storytelling & Interactive reporting with Power BI bookmarks and drillthrough
 
Digitalisation from the back office to the factory floor
Digitalisation from the back office to the factory floorDigitalisation from the back office to the factory floor
Digitalisation from the back office to the factory floor
 
Identifying Users Across Platforms with a Universal ID Webinar Slides
Identifying Users Across Platforms with a Universal ID Webinar SlidesIdentifying Users Across Platforms with a Universal ID Webinar Slides
Identifying Users Across Platforms with a Universal ID Webinar Slides
 
Tableau Conference 2018: Binging on Data - Enabling Analytics at Netflix
Tableau Conference 2018: Binging on Data - Enabling Analytics at NetflixTableau Conference 2018: Binging on Data - Enabling Analytics at Netflix
Tableau Conference 2018: Binging on Data - Enabling Analytics at Netflix
 

Viewers also liked

Running Business Critical Workloads on AWS – Nam Je Cho
Running Business Critical Workloads on AWS – Nam Je ChoRunning Business Critical Workloads on AWS – Nam Je Cho
Running Business Critical Workloads on AWS – Nam Je Cho
Amazon Web Services
 
Giip bp-giip connectivity1703
Giip bp-giip connectivity1703Giip bp-giip connectivity1703
Giip bp-giip connectivity1703
Lowy Shin
 
PaaS for Dummies
PaaS for DummiesPaaS for Dummies
PaaS for Dummies
bram hurkmans
 
Production testing and disaster recovery
Production testing and disaster recoveryProduction testing and disaster recovery
Production testing and disaster recovery
BizTalk360
 
The Biggest Lies That Digital Marketers Tell Themselves - 3XE Digital
The Biggest Lies That Digital Marketers Tell Themselves - 3XE DigitalThe Biggest Lies That Digital Marketers Tell Themselves - 3XE Digital
The Biggest Lies That Digital Marketers Tell Themselves - 3XE Digital
Eduardas Gricius
 
Cloud integration: what's in it for you? (Toon Vanhoutte & Massimo Crippa at ...
Cloud integration: what's in it for you? (Toon Vanhoutte & Massimo Crippa at ...Cloud integration: what's in it for you? (Toon Vanhoutte & Massimo Crippa at ...
Cloud integration: what's in it for you? (Toon Vanhoutte & Massimo Crippa at ...
Codit
 
Cassandra Talk: Austin JUG
Cassandra Talk: Austin JUGCassandra Talk: Austin JUG
Cassandra Talk: Austin JUG
Stu Hood
 
Elk Reporting Ii
Elk Reporting IiElk Reporting Ii
Elk Reporting Ii
mwmiller12
 
5733 a deep dive into IBM Watson Foundation for CSP (WFC)
5733   a deep dive into IBM Watson Foundation for CSP (WFC)5733   a deep dive into IBM Watson Foundation for CSP (WFC)
5733 a deep dive into IBM Watson Foundation for CSP (WFC)
Arvind Sathi
 
Open Source Monitoring Tools Shootout
Open Source Monitoring Tools ShootoutOpen Source Monitoring Tools Shootout
Open Source Monitoring Tools Shootout
tomdc
 
SPSNL17 - Securing Office 365 and Microsoft Azure like a rock star (or groupi...
SPSNL17 - Securing Office 365 and Microsoft Azure like a rock star (or groupi...SPSNL17 - Securing Office 365 and Microsoft Azure like a rock star (or groupi...
SPSNL17 - Securing Office 365 and Microsoft Azure like a rock star (or groupi...
DIWUG
 
Oracle integration cloud service (ICS) best practices learned from the field ...
Oracle integration cloud service (ICS) best practices learned from the field ...Oracle integration cloud service (ICS) best practices learned from the field ...
Oracle integration cloud service (ICS) best practices learned from the field ...
Phil Wilkins
 
Plan de transport 2014: le Brabant Flamand
Plan de transport 2014: le Brabant FlamandPlan de transport 2014: le Brabant Flamand
Plan de transport 2014: le Brabant Flamand
SNCB
 
Nato Constitution- & Laws. Chris Helweg
Nato Constitution-  &  Laws. Chris HelwegNato Constitution-  &  Laws. Chris Helweg
Nato Constitution- & Laws. Chris Helweg
Chris Helweg
 
Things you should know about Scalability!
Things you should know about Scalability!Things you should know about Scalability!
Things you should know about Scalability!
Robert Mederer
 
15 oefeningen schakelen van weerstanden
15 oefeningen schakelen van weerstanden15 oefeningen schakelen van weerstanden
15 oefeningen schakelen van weerstandenFreddy Van Eynde
 
The Loss of HMAS SYDNEY 2: Medical Aspects- Westphalen
The Loss of HMAS SYDNEY 2: Medical Aspects- WestphalenThe Loss of HMAS SYDNEY 2: Medical Aspects- Westphalen
The Loss of HMAS SYDNEY 2: Medical Aspects- Westphalen
Leishman Associates
 
AI = SE , giip system manage automation with A.I
AI = SE , giip system manage automation with A.IAI = SE , giip system manage automation with A.I
AI = SE , giip system manage automation with A.I
Lowy Shin
 
Evolving your automation with hybrid workers
Evolving your automation with hybrid workersEvolving your automation with hybrid workers
Evolving your automation with hybrid workers
kieranjacobsen
 
AWS Summit - Trends in Advanced Monitoring for AWS environments
AWS Summit - Trends in Advanced Monitoring for AWS environmentsAWS Summit - Trends in Advanced Monitoring for AWS environments
AWS Summit - Trends in Advanced Monitoring for AWS environments
Andreas Grabner
 

Viewers also liked (20)

Running Business Critical Workloads on AWS – Nam Je Cho
Running Business Critical Workloads on AWS – Nam Je ChoRunning Business Critical Workloads on AWS – Nam Je Cho
Running Business Critical Workloads on AWS – Nam Je Cho
 
Giip bp-giip connectivity1703
Giip bp-giip connectivity1703Giip bp-giip connectivity1703
Giip bp-giip connectivity1703
 
PaaS for Dummies
PaaS for DummiesPaaS for Dummies
PaaS for Dummies
 
Production testing and disaster recovery
Production testing and disaster recoveryProduction testing and disaster recovery
Production testing and disaster recovery
 
The Biggest Lies That Digital Marketers Tell Themselves - 3XE Digital
The Biggest Lies That Digital Marketers Tell Themselves - 3XE DigitalThe Biggest Lies That Digital Marketers Tell Themselves - 3XE Digital
The Biggest Lies That Digital Marketers Tell Themselves - 3XE Digital
 
Cloud integration: what's in it for you? (Toon Vanhoutte & Massimo Crippa at ...
Cloud integration: what's in it for you? (Toon Vanhoutte & Massimo Crippa at ...Cloud integration: what's in it for you? (Toon Vanhoutte & Massimo Crippa at ...
Cloud integration: what's in it for you? (Toon Vanhoutte & Massimo Crippa at ...
 
Cassandra Talk: Austin JUG
Cassandra Talk: Austin JUGCassandra Talk: Austin JUG
Cassandra Talk: Austin JUG
 
Elk Reporting Ii
Elk Reporting IiElk Reporting Ii
Elk Reporting Ii
 
5733 a deep dive into IBM Watson Foundation for CSP (WFC)
5733   a deep dive into IBM Watson Foundation for CSP (WFC)5733   a deep dive into IBM Watson Foundation for CSP (WFC)
5733 a deep dive into IBM Watson Foundation for CSP (WFC)
 
Open Source Monitoring Tools Shootout
Open Source Monitoring Tools ShootoutOpen Source Monitoring Tools Shootout
Open Source Monitoring Tools Shootout
 
SPSNL17 - Securing Office 365 and Microsoft Azure like a rock star (or groupi...
SPSNL17 - Securing Office 365 and Microsoft Azure like a rock star (or groupi...SPSNL17 - Securing Office 365 and Microsoft Azure like a rock star (or groupi...
SPSNL17 - Securing Office 365 and Microsoft Azure like a rock star (or groupi...
 
Oracle integration cloud service (ICS) best practices learned from the field ...
Oracle integration cloud service (ICS) best practices learned from the field ...Oracle integration cloud service (ICS) best practices learned from the field ...
Oracle integration cloud service (ICS) best practices learned from the field ...
 
Plan de transport 2014: le Brabant Flamand
Plan de transport 2014: le Brabant FlamandPlan de transport 2014: le Brabant Flamand
Plan de transport 2014: le Brabant Flamand
 
Nato Constitution- & Laws. Chris Helweg
Nato Constitution-  &  Laws. Chris HelwegNato Constitution-  &  Laws. Chris Helweg
Nato Constitution- & Laws. Chris Helweg
 
Things you should know about Scalability!
Things you should know about Scalability!Things you should know about Scalability!
Things you should know about Scalability!
 
15 oefeningen schakelen van weerstanden
15 oefeningen schakelen van weerstanden15 oefeningen schakelen van weerstanden
15 oefeningen schakelen van weerstanden
 
The Loss of HMAS SYDNEY 2: Medical Aspects- Westphalen
The Loss of HMAS SYDNEY 2: Medical Aspects- WestphalenThe Loss of HMAS SYDNEY 2: Medical Aspects- Westphalen
The Loss of HMAS SYDNEY 2: Medical Aspects- Westphalen
 
AI = SE , giip system manage automation with A.I
AI = SE , giip system manage automation with A.IAI = SE , giip system manage automation with A.I
AI = SE , giip system manage automation with A.I
 
Evolving your automation with hybrid workers
Evolving your automation with hybrid workersEvolving your automation with hybrid workers
Evolving your automation with hybrid workers
 
AWS Summit - Trends in Advanced Monitoring for AWS environments
AWS Summit - Trends in Advanced Monitoring for AWS environmentsAWS Summit - Trends in Advanced Monitoring for AWS environments
AWS Summit - Trends in Advanced Monitoring for AWS environments
 

Similar to Generalized B2B Machine Learning by Andrew Waage

ECIR Recommendation Challenges
ECIR Recommendation ChallengesECIR Recommendation Challenges
ECIR Recommendation Challenges
Daniel Kohlsdorf
 
Decision Matrix for IoT Product Development
Decision Matrix for IoT Product DevelopmentDecision Matrix for IoT Product Development
Decision Matrix for IoT Product Development
Alexey Pyshkin
 
Managing an Experimentation Platform by LinkedIn Product Leader
Managing an Experimentation Platform by LinkedIn Product LeaderManaging an Experimentation Platform by LinkedIn Product Leader
Managing an Experimentation Platform by LinkedIn Product Leader
Product School
 
Operationalizing Machine Learning—Managing Provenance from Raw Data to Predic...
Operationalizing Machine Learning—Managing Provenance from Raw Data to Predic...Operationalizing Machine Learning—Managing Provenance from Raw Data to Predic...
Operationalizing Machine Learning—Managing Provenance from Raw Data to Predic...
Databricks
 
MLSEV Virtual. One Platform to Rule Them All
MLSEV Virtual. One Platform to Rule Them AllMLSEV Virtual. One Platform to Rule Them All
MLSEV Virtual. One Platform to Rule Them All
BigML, Inc
 
AI projects - Lifecyle & Best Practices
AI projects - Lifecyle & Best PracticesAI projects - Lifecyle & Best Practices
AI projects - Lifecyle & Best Practices
Vincent de Stoecklin
 
Machine Learning and Industrie 4.0
Machine Learning and Industrie 4.0Machine Learning and Industrie 4.0
Machine Learning and Industrie 4.0
Peter Schleinitz
 
Cloud computing
Cloud computingCloud computing
Cloud computing
Amit Kumar
 
Machine Learning & Predictive Maintenance
Machine Learning &  Predictive MaintenanceMachine Learning &  Predictive Maintenance
Machine Learning & Predictive Maintenance
Arnab Biswas
 
ARC202:real world real time analytics
ARC202:real world real time analyticsARC202:real world real time analytics
ARC202:real world real time analytics
Sebastian Montini
 
Intelligent Digital Mesh Testing
Intelligent Digital Mesh TestingIntelligent Digital Mesh Testing
Intelligent Digital Mesh Testing
Nagarro
 
Citizen Data Science Training using KNIME
Citizen Data Science Training using KNIMECitizen Data Science Training using KNIME
Citizen Data Science Training using KNIME
Ali Raza Anjum
 
Barga Galvanize Sept 2015
Barga Galvanize Sept 2015Barga Galvanize Sept 2015
Barga Galvanize Sept 2015
Roger Barga
 
Incremental Queries and Transformations for Engineering Critical Systems
Incremental Queries and Transformations for Engineering Critical SystemsIncremental Queries and Transformations for Engineering Critical Systems
Incremental Queries and Transformations for Engineering Critical Systems
Ákos Horváth
 
How to Build an AI/ML Product and Sell it by SalesChoice CPO
How to Build an AI/ML Product and Sell it by SalesChoice CPOHow to Build an AI/ML Product and Sell it by SalesChoice CPO
How to Build an AI/ML Product and Sell it by SalesChoice CPO
Product School
 
MODEL-DRIVEN ENGINEERING (MDE) in Practice
MODEL-DRIVEN ENGINEERING (MDE) in PracticeMODEL-DRIVEN ENGINEERING (MDE) in Practice
MODEL-DRIVEN ENGINEERING (MDE) in Practice
Hussein Alshkhir
 
MLOps and Data Quality: Deploying Reliable ML Models in Production
MLOps and Data Quality: Deploying Reliable ML Models in ProductionMLOps and Data Quality: Deploying Reliable ML Models in Production
MLOps and Data Quality: Deploying Reliable ML Models in Production
Provectus
 
Managing AI Products
Managing AI ProductsManaging AI Products
Managing AI Products
Prasad Velamuri
 
Digital strategy overview
Digital strategy overviewDigital strategy overview
Digital strategy overview
Ashish Bhasin
 
Building a Real-Time Security Application Using Log Data and Machine Learning...
Building a Real-Time Security Application Using Log Data and Machine Learning...Building a Real-Time Security Application Using Log Data and Machine Learning...
Building a Real-Time Security Application Using Log Data and Machine Learning...
Sri Ambati
 

Similar to Generalized B2B Machine Learning by Andrew Waage (20)

ECIR Recommendation Challenges
ECIR Recommendation ChallengesECIR Recommendation Challenges
ECIR Recommendation Challenges
 
Decision Matrix for IoT Product Development
Decision Matrix for IoT Product DevelopmentDecision Matrix for IoT Product Development
Decision Matrix for IoT Product Development
 
Managing an Experimentation Platform by LinkedIn Product Leader
Managing an Experimentation Platform by LinkedIn Product LeaderManaging an Experimentation Platform by LinkedIn Product Leader
Managing an Experimentation Platform by LinkedIn Product Leader
 
Operationalizing Machine Learning—Managing Provenance from Raw Data to Predic...
Operationalizing Machine Learning—Managing Provenance from Raw Data to Predic...Operationalizing Machine Learning—Managing Provenance from Raw Data to Predic...
Operationalizing Machine Learning—Managing Provenance from Raw Data to Predic...
 
MLSEV Virtual. One Platform to Rule Them All
MLSEV Virtual. One Platform to Rule Them AllMLSEV Virtual. One Platform to Rule Them All
MLSEV Virtual. One Platform to Rule Them All
 
AI projects - Lifecyle & Best Practices
AI projects - Lifecyle & Best PracticesAI projects - Lifecyle & Best Practices
AI projects - Lifecyle & Best Practices
 
Machine Learning and Industrie 4.0
Machine Learning and Industrie 4.0Machine Learning and Industrie 4.0
Machine Learning and Industrie 4.0
 
Cloud computing
Cloud computingCloud computing
Cloud computing
 
Machine Learning & Predictive Maintenance
Machine Learning &  Predictive MaintenanceMachine Learning &  Predictive Maintenance
Machine Learning & Predictive Maintenance
 
ARC202:real world real time analytics
ARC202:real world real time analyticsARC202:real world real time analytics
ARC202:real world real time analytics
 
Intelligent Digital Mesh Testing
Intelligent Digital Mesh TestingIntelligent Digital Mesh Testing
Intelligent Digital Mesh Testing
 
Citizen Data Science Training using KNIME
Citizen Data Science Training using KNIMECitizen Data Science Training using KNIME
Citizen Data Science Training using KNIME
 
Barga Galvanize Sept 2015
Barga Galvanize Sept 2015Barga Galvanize Sept 2015
Barga Galvanize Sept 2015
 
Incremental Queries and Transformations for Engineering Critical Systems
Incremental Queries and Transformations for Engineering Critical SystemsIncremental Queries and Transformations for Engineering Critical Systems
Incremental Queries and Transformations for Engineering Critical Systems
 
How to Build an AI/ML Product and Sell it by SalesChoice CPO
How to Build an AI/ML Product and Sell it by SalesChoice CPOHow to Build an AI/ML Product and Sell it by SalesChoice CPO
How to Build an AI/ML Product and Sell it by SalesChoice CPO
 
MODEL-DRIVEN ENGINEERING (MDE) in Practice
MODEL-DRIVEN ENGINEERING (MDE) in PracticeMODEL-DRIVEN ENGINEERING (MDE) in Practice
MODEL-DRIVEN ENGINEERING (MDE) in Practice
 
MLOps and Data Quality: Deploying Reliable ML Models in Production
MLOps and Data Quality: Deploying Reliable ML Models in ProductionMLOps and Data Quality: Deploying Reliable ML Models in Production
MLOps and Data Quality: Deploying Reliable ML Models in Production
 
Managing AI Products
Managing AI ProductsManaging AI Products
Managing AI Products
 
Digital strategy overview
Digital strategy overviewDigital strategy overview
Digital strategy overview
 
Building a Real-Time Security Application Using Log Data and Machine Learning...
Building a Real-Time Security Application Using Log Data and Machine Learning...Building a Real-Time Security Application Using Log Data and Machine Learning...
Building a Real-Time Security Application Using Log Data and Machine Learning...
 

More from Data Con LA

Data Con LA 2022 Keynotes
Data Con LA 2022 KeynotesData Con LA 2022 Keynotes
Data Con LA 2022 Keynotes
Data Con LA
 
Data Con LA 2022 Keynotes
Data Con LA 2022 KeynotesData Con LA 2022 Keynotes
Data Con LA 2022 Keynotes
Data Con LA
 
Data Con LA 2022 Keynote
Data Con LA 2022 KeynoteData Con LA 2022 Keynote
Data Con LA 2022 Keynote
Data Con LA
 
Data Con LA 2022 - Startup Showcase
Data Con LA 2022 - Startup ShowcaseData Con LA 2022 - Startup Showcase
Data Con LA 2022 - Startup Showcase
Data Con LA
 
Data Con LA 2022 Keynote
Data Con LA 2022 KeynoteData Con LA 2022 Keynote
Data Con LA 2022 Keynote
Data Con LA
 
Data Con LA 2022 - Using Google trends data to build product recommendations
Data Con LA 2022 - Using Google trends data to build product recommendationsData Con LA 2022 - Using Google trends data to build product recommendations
Data Con LA 2022 - Using Google trends data to build product recommendations
Data Con LA
 
Data Con LA 2022 - AI Ethics
Data Con LA 2022 - AI EthicsData Con LA 2022 - AI Ethics
Data Con LA 2022 - AI Ethics
Data Con LA
 
Data Con LA 2022 - Improving disaster response with machine learning
Data Con LA 2022 - Improving disaster response with machine learningData Con LA 2022 - Improving disaster response with machine learning
Data Con LA 2022 - Improving disaster response with machine learning
Data Con LA
 
Data Con LA 2022 - What's new with MongoDB 6.0 and Atlas
Data Con LA 2022 - What's new with MongoDB 6.0 and AtlasData Con LA 2022 - What's new with MongoDB 6.0 and Atlas
Data Con LA 2022 - What's new with MongoDB 6.0 and Atlas
Data Con LA
 
Data Con LA 2022 - Real world consumer segmentation
Data Con LA 2022 - Real world consumer segmentationData Con LA 2022 - Real world consumer segmentation
Data Con LA 2022 - Real world consumer segmentation
Data Con LA
 
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Data Con LA
 
Data Con LA 2022 - Moving Data at Scale to AWS
Data Con LA 2022 - Moving Data at Scale to AWSData Con LA 2022 - Moving Data at Scale to AWS
Data Con LA 2022 - Moving Data at Scale to AWS
Data Con LA
 
Data Con LA 2022 - Collaborative Data Exploration using Conversational AI
Data Con LA 2022 - Collaborative Data Exploration using Conversational AIData Con LA 2022 - Collaborative Data Exploration using Conversational AI
Data Con LA 2022 - Collaborative Data Exploration using Conversational AI
Data Con LA
 
Data Con LA 2022 - Why Database Modernization Makes Your Data Decisions More ...
Data Con LA 2022 - Why Database Modernization Makes Your Data Decisions More ...Data Con LA 2022 - Why Database Modernization Makes Your Data Decisions More ...
Data Con LA 2022 - Why Database Modernization Makes Your Data Decisions More ...
Data Con LA
 
Data Con LA 2022 - Intro to Data Science
Data Con LA 2022 - Intro to Data ScienceData Con LA 2022 - Intro to Data Science
Data Con LA 2022 - Intro to Data Science
Data Con LA
 
Data Con LA 2022 - How are NFTs and DeFi Changing Entertainment
Data Con LA 2022 - How are NFTs and DeFi Changing EntertainmentData Con LA 2022 - How are NFTs and DeFi Changing Entertainment
Data Con LA 2022 - How are NFTs and DeFi Changing Entertainment
Data Con LA
 
Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automat...
Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automat...Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automat...
Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automat...
Data Con LA
 
Data Con LA 2022-Perfect Viral Ad prediction of Superbowl 2022 using Tease, T...
Data Con LA 2022-Perfect Viral Ad prediction of Superbowl 2022 using Tease, T...Data Con LA 2022-Perfect Viral Ad prediction of Superbowl 2022 using Tease, T...
Data Con LA 2022-Perfect Viral Ad prediction of Superbowl 2022 using Tease, T...
Data Con LA
 
Data Con LA 2022- Embedding medical journeys with machine learning to improve...
Data Con LA 2022- Embedding medical journeys with machine learning to improve...Data Con LA 2022- Embedding medical journeys with machine learning to improve...
Data Con LA 2022- Embedding medical journeys with machine learning to improve...
Data Con LA
 
Data Con LA 2022 - Data Streaming with Kafka
Data Con LA 2022 - Data Streaming with KafkaData Con LA 2022 - Data Streaming with Kafka
Data Con LA 2022 - Data Streaming with Kafka
Data Con LA
 

More from Data Con LA (20)

Data Con LA 2022 Keynotes
Data Con LA 2022 KeynotesData Con LA 2022 Keynotes
Data Con LA 2022 Keynotes
 
Data Con LA 2022 Keynotes
Data Con LA 2022 KeynotesData Con LA 2022 Keynotes
Data Con LA 2022 Keynotes
 
Data Con LA 2022 Keynote
Data Con LA 2022 KeynoteData Con LA 2022 Keynote
Data Con LA 2022 Keynote
 
Data Con LA 2022 - Startup Showcase
Data Con LA 2022 - Startup ShowcaseData Con LA 2022 - Startup Showcase
Data Con LA 2022 - Startup Showcase
 
Data Con LA 2022 Keynote
Data Con LA 2022 KeynoteData Con LA 2022 Keynote
Data Con LA 2022 Keynote
 
Data Con LA 2022 - Using Google trends data to build product recommendations
Data Con LA 2022 - Using Google trends data to build product recommendationsData Con LA 2022 - Using Google trends data to build product recommendations
Data Con LA 2022 - Using Google trends data to build product recommendations
 
Data Con LA 2022 - AI Ethics
Data Con LA 2022 - AI EthicsData Con LA 2022 - AI Ethics
Data Con LA 2022 - AI Ethics
 
Data Con LA 2022 - Improving disaster response with machine learning
Data Con LA 2022 - Improving disaster response with machine learningData Con LA 2022 - Improving disaster response with machine learning
Data Con LA 2022 - Improving disaster response with machine learning
 
Data Con LA 2022 - What's new with MongoDB 6.0 and Atlas
Data Con LA 2022 - What's new with MongoDB 6.0 and AtlasData Con LA 2022 - What's new with MongoDB 6.0 and Atlas
Data Con LA 2022 - What's new with MongoDB 6.0 and Atlas
 
Data Con LA 2022 - Real world consumer segmentation
Data Con LA 2022 - Real world consumer segmentationData Con LA 2022 - Real world consumer segmentation
Data Con LA 2022 - Real world consumer segmentation
 
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
 
Data Con LA 2022 - Moving Data at Scale to AWS
Data Con LA 2022 - Moving Data at Scale to AWSData Con LA 2022 - Moving Data at Scale to AWS
Data Con LA 2022 - Moving Data at Scale to AWS
 
Data Con LA 2022 - Collaborative Data Exploration using Conversational AI
Data Con LA 2022 - Collaborative Data Exploration using Conversational AIData Con LA 2022 - Collaborative Data Exploration using Conversational AI
Data Con LA 2022 - Collaborative Data Exploration using Conversational AI
 
Data Con LA 2022 - Why Database Modernization Makes Your Data Decisions More ...
Data Con LA 2022 - Why Database Modernization Makes Your Data Decisions More ...Data Con LA 2022 - Why Database Modernization Makes Your Data Decisions More ...
Data Con LA 2022 - Why Database Modernization Makes Your Data Decisions More ...
 
Data Con LA 2022 - Intro to Data Science
Data Con LA 2022 - Intro to Data ScienceData Con LA 2022 - Intro to Data Science
Data Con LA 2022 - Intro to Data Science
 
Data Con LA 2022 - How are NFTs and DeFi Changing Entertainment
Data Con LA 2022 - How are NFTs and DeFi Changing EntertainmentData Con LA 2022 - How are NFTs and DeFi Changing Entertainment
Data Con LA 2022 - How are NFTs and DeFi Changing Entertainment
 
Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automat...
Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automat...Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automat...
Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automat...
 
Data Con LA 2022-Perfect Viral Ad prediction of Superbowl 2022 using Tease, T...
Data Con LA 2022-Perfect Viral Ad prediction of Superbowl 2022 using Tease, T...Data Con LA 2022-Perfect Viral Ad prediction of Superbowl 2022 using Tease, T...
Data Con LA 2022-Perfect Viral Ad prediction of Superbowl 2022 using Tease, T...
 
Data Con LA 2022- Embedding medical journeys with machine learning to improve...
Data Con LA 2022- Embedding medical journeys with machine learning to improve...Data Con LA 2022- Embedding medical journeys with machine learning to improve...
Data Con LA 2022- Embedding medical journeys with machine learning to improve...
 
Data Con LA 2022 - Data Streaming with Kafka
Data Con LA 2022 - Data Streaming with KafkaData Con LA 2022 - Data Streaming with Kafka
Data Con LA 2022 - Data Streaming with Kafka
 

Recently uploaded

Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neo4j
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Neo4j
 
Data structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdfData structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdf
TIPNGVN2
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
James Anderson
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 

Recently uploaded (20)

Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
 
Data structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdfData structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdf
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 

Generalized B2B Machine Learning by Andrew Waage

  • 3. AI Powered Marketing Automation Platform Step 1 Collect Data Step 2 Generate Predictions Step 3 Automation Powers Intelligent Campaigns Across Channels Ecom / Retail Behavioral Custom Demographic Email Campaigns On-Site Display Mobile Call Center
  • 4. What kind of Scale? 100’s Clients 210M+ Customers Tracked 1000+ Client-specific Models 2 Billion+ Predictions Daily10K+ Actions / second
  • 6. The Challenge •  Many Clients •  Dirty Data •  Sparse Datasets •  Custom Attributes •  Various Industries Clean PredictionsModel Layer! C1 C2 C3 C4
  • 7. What Kind of Predictions? Purchase Probability: High-likelihood Lifecycle Stage: Ready to Buy Churn Time: 300 days Customer Future Value: $925 Contact Frequency: Every 3 days Optimal Time to Engage: Thursday 7-9PM Optimal Incentive/Discount: Dollars Off Product Recommendations: Based on interest Optimal Subject Line: Individual preference Optimal Template: Individual preference
  • 8. Our Approach & Learnings 1.  Robust Ingestion Pipeline 2.  Common Feature Engineering Layer 3.  “Plug-in” Architecture for Models 4.  Evaluation / AB Testing 5.  Robust Monitoring & Visualization
  • 9. 1. Robust Ingestion Pipeline 10K+ Actions Per Second auto-scaling! auto-scaling lambdas! •  Abstraction Layer: Data Ingestion •  Do not compromise for clean data •  Auto-scaling everywhere •  High confidence in upstream data Flume Kinesis
  • 10. 2. Common Feature Engineering Layer •  Abstraction: Feature Layer •  Allow custom features •  Handle feature selection •  Modelers know what to expect Raw Data User Behavior Features Product Features User Sta4c Features Timing Model CLV Model Recommender
  • 11. 3. Model Plug-in Architecture C6 C3 C4 C5 C7 • Plug-in Architecture • Tune model hyper-parameters • A/B test models per client C1 C8 C2 Client’s Model Execution Plan
  • 12. Recommender System Multi-Layer Personalization: •  Layer 1: ML / Algorithmic •  ALS CF, Content-based, Item-Item •  Layer 2: User-level Domain Logic •  User-level predictions (gender, location, shoe sizes) •  Layer 3: Client-tuned Domain Logic / Controls •  Rank by profit-margin •  Increase discovery rate influxer 1. Algorithmic 2. User-Level Domain Logic 3. Client-Level Controls
  • 13. 4. Model Evaluation / Fast Feedback A/B Framework M1 M2 M3 •  Start Simple •  Collect feedback data •  Skip long production cycle •  Unbiased policy generation is important M1 Campaign Predic4ons
  • 14. 5. Robust Model Monitoring and Visualization “Sauron” (LOTR)! Monitor, monitor, monitor!!
  • 15. Monitor Recs: Distributions, Coverage, Diversity REF: hQp://sauron.rsci.co/advanced/sites/181/data_types/31
  • 16. Monitoring Subject Line Bandit Models
  • 17. Churn Rates, ROC Curve, Reliability Curve
  • 18. Our Data Science Stack Persistence! Pipeline / Process! Viz / Monitor! Code!
  • 19. Takeaways 1.  Use abstraction layers -  Clean / common interfaces 2.  Monitor, monitor, monitor -  Fast feedback 3.  Start simple and keep iterating