SlideShare a Scribd company logo
Databricks Workshop - April 11th 2019
Case Study: Outreach
Andrew Brooks, Li Dong, Jiwei Cao
Content Overview
1. Introduction to Outreach
a. #1 Sales Engagement Platform
b. Data Science + Sales Engagement
2. Outreach with Databricks
a.Case Study - Production: Out-of-office Data Extraction
b.Case Study - Research: Intent Classification
2
Introduction to Outreach
The #1 Sales Engagement Platform
Wait, what is a Sales Engagement Platform?
4
It’s a book.
5
It’s also a new category of software.
Add content
6
Source: https://blog.topohq.com/sales-engagement-the-definitive-guide/
How about an Example?
7
Automates execution of some sales tasks:
Emails, Linkedin Messages etc.
Schedules and reminds the rep when it is the
right time to do the manual tasks (e.g. phone
call, custom manual email)
SEP Encodes and Automates Sales
Activities into Workflows/Pipelines
Data Science + Sales Engagement
Outreach ML Features:
- Automation - Information Extraction
- A/B testing
- Advanced Analytics (dashboard & reporting)
Optimization:
- Intent & Topic Detection
- Content & Action Recommendation
- Prioritization & Forecasting
8
Highlighted use case
Phone
Email
LinkedIn
Meetings
Data Sources:
Case Study: Out-of-office Data Extraction
Automation - Information Extraction
Highlighted use case: Out of Office return date and referral extraction
10
11
Production Architecture
Highlighted use case: OOO Information Extraction
12
Something to improve
- Separate stacks to develop ML model and deploy model
- jupyter notebooks
- databricks notebooks
- Docker, K8s(production)
- Lack of ML model life-cycle management
- model training
- experiment(alpha, beta)
- production to GA
- model iterations / releases
Production Architecture
Highlighted use case: OOO Information Extraction
13
Production (Next Step…)
Highlighted use case: OOO Information Extraction
Case Study: Intent Classification
Intent Classification: Provide More Value Metrics
15
Intent Classification: Problem Solving
Steps to solve it with NLP/Machine Learning:
1. Annotate some emails
2. Setup the Experiment Environment(Spark, NLP/ML-Packages)
3. Write Code and Running Experiments
4. Analyze Experiment Results
16
Goal: Classify email replies into 3 categories: positive, objection and unsubscription.
Doesn’t Look Complicated, But Painful
Pain Points:
● Difficult to setup and maintain a proper environment.
● Can’t run multiple Experiments at the same time
● Experiment Results are scattered in multiple files. Hard to navigate and analysis.
17
Intent Classification: Problem Solving
Steps to explore ideas with Databricks:
1. Annotate some emails
2. Setup the Experiment Environment(Spark, NLP/ML-Packages)
3. Write Code and Running Experiments
4. Analyze Experiment Results
5. Visualization the model prediction
18
Goal: Classify email replies into positive, objection and unsubscription.
Visualize the Model Prediction: t-SNE
19
Before Training After Training
How does Databricks help us
• Setup a Dedicated Environment with Less Effort
• Running Experiments at Scale
• Analyze Experiment Results at One Place
20
Q & A

More Related Content

Similar to 2019-04-11 Databricks Unified Analytics Workshop - Outreach Case Study

Isobar - Product & Ventures - Product Growth Management Playbook
Isobar - Product & Ventures -  Product Growth Management PlaybookIsobar - Product & Ventures -  Product Growth Management Playbook
Isobar - Product & Ventures - Product Growth Management Playbook
Nhat Tran
 
Thripp EME 2040 Slides on Microsoft Education Badges, PowerPoint Quiz, Comput...
Thripp EME 2040 Slides on Microsoft Education Badges, PowerPoint Quiz, Comput...Thripp EME 2040 Slides on Microsoft Education Badges, PowerPoint Quiz, Comput...
Thripp EME 2040 Slides on Microsoft Education Badges, PowerPoint Quiz, Comput...
Richard Thripp
 
How to become a software developer
How to become a software developerHow to become a software developer
How to become a software developer
Eyob Lube
 
RAPIDS
RAPIDSRAPIDS
RAPIDS
Teresa Liew
 
Learning Management Systems for Nonprofits – Net2van July 9 2019
Learning Management Systems for Nonprofits – Net2van July 9 2019Learning Management Systems for Nonprofits – Net2van July 9 2019
Learning Management Systems for Nonprofits – Net2van July 9 2019
NetSquared Vancouver
 
Supervised learning
Supervised learningSupervised learning
Supervised learning
ankit_ppt
 
Successful eLearning Implementation
Successful eLearning ImplementationSuccessful eLearning Implementation
Successful eLearning Implementation
LearningSO
 
data science and business analytics
data science and business analyticsdata science and business analytics
data science and business analytics
sunnypatil1778
 
Two simples and quick ways to Save and Share your company Knowledge
Two simples and quick ways to Save and Share your company Knowledge Two simples and quick ways to Save and Share your company Knowledge
Two simples and quick ways to Save and Share your company Knowledge
Jeff ANGAMA
 
Open learner models (OLMs) and learning analytics dashboards: A systematic re...
Open learner models (OLMs) and learning analytics dashboards: A systematic re...Open learner models (OLMs) and learning analytics dashboards: A systematic re...
Open learner models (OLMs) and learning analytics dashboards: A systematic re...
Daniel Davis
 
[MET] Software Recruiting 101
[MET] Software Recruiting 101[MET] Software Recruiting 101
[MET] Software Recruiting 101
ImaniSalazarNahle
 
NVIDIA RecSys Summit 2022 - EoR
NVIDIA RecSys Summit 2022 - EoRNVIDIA RecSys Summit 2022 - EoR
NVIDIA RecSys Summit 2022 - EoR
Bryan Bischof
 
CV2015
CV2015CV2015
Machine Learning Product Managers Meetup Event
Machine Learning Product Managers Meetup EventMachine Learning Product Managers Meetup Event
Machine Learning Product Managers Meetup Event
Benjamin Schulte
 
data-science-lifecycle-ebook.pdf
data-science-lifecycle-ebook.pdfdata-science-lifecycle-ebook.pdf
data-science-lifecycle-ebook.pdf
Danilo Cardona
 
On Machine Learning Readiness
On Machine Learning ReadinessOn Machine Learning Readiness
On Machine Learning Readiness
Anne-Marie Tousch
 
Recsys 2016
Recsys 2016Recsys 2016
Recsys 2016
Mindaugas Zickus
 
From DevOps to MLOps: practical steps for a smooth transition
From DevOps to MLOps: practical steps for a smooth transitionFrom DevOps to MLOps: practical steps for a smooth transition
From DevOps to MLOps: practical steps for a smooth transition
Anne-Marie Tousch
 
qualitative.ppt
qualitative.pptqualitative.ppt
qualitative.ppt
CityComputers3
 
Using Computer as a Research Assistant in Qualitative Research
Using Computer as a Research Assistant in Qualitative ResearchUsing Computer as a Research Assistant in Qualitative Research
Using Computer as a Research Assistant in Qualitative Research
JoshuaApolonio1
 

Similar to 2019-04-11 Databricks Unified Analytics Workshop - Outreach Case Study (20)

Isobar - Product & Ventures - Product Growth Management Playbook
Isobar - Product & Ventures -  Product Growth Management PlaybookIsobar - Product & Ventures -  Product Growth Management Playbook
Isobar - Product & Ventures - Product Growth Management Playbook
 
Thripp EME 2040 Slides on Microsoft Education Badges, PowerPoint Quiz, Comput...
Thripp EME 2040 Slides on Microsoft Education Badges, PowerPoint Quiz, Comput...Thripp EME 2040 Slides on Microsoft Education Badges, PowerPoint Quiz, Comput...
Thripp EME 2040 Slides on Microsoft Education Badges, PowerPoint Quiz, Comput...
 
How to become a software developer
How to become a software developerHow to become a software developer
How to become a software developer
 
RAPIDS
RAPIDSRAPIDS
RAPIDS
 
Learning Management Systems for Nonprofits – Net2van July 9 2019
Learning Management Systems for Nonprofits – Net2van July 9 2019Learning Management Systems for Nonprofits – Net2van July 9 2019
Learning Management Systems for Nonprofits – Net2van July 9 2019
 
Supervised learning
Supervised learningSupervised learning
Supervised learning
 
Successful eLearning Implementation
Successful eLearning ImplementationSuccessful eLearning Implementation
Successful eLearning Implementation
 
data science and business analytics
data science and business analyticsdata science and business analytics
data science and business analytics
 
Two simples and quick ways to Save and Share your company Knowledge
Two simples and quick ways to Save and Share your company Knowledge Two simples and quick ways to Save and Share your company Knowledge
Two simples and quick ways to Save and Share your company Knowledge
 
Open learner models (OLMs) and learning analytics dashboards: A systematic re...
Open learner models (OLMs) and learning analytics dashboards: A systematic re...Open learner models (OLMs) and learning analytics dashboards: A systematic re...
Open learner models (OLMs) and learning analytics dashboards: A systematic re...
 
[MET] Software Recruiting 101
[MET] Software Recruiting 101[MET] Software Recruiting 101
[MET] Software Recruiting 101
 
NVIDIA RecSys Summit 2022 - EoR
NVIDIA RecSys Summit 2022 - EoRNVIDIA RecSys Summit 2022 - EoR
NVIDIA RecSys Summit 2022 - EoR
 
CV2015
CV2015CV2015
CV2015
 
Machine Learning Product Managers Meetup Event
Machine Learning Product Managers Meetup EventMachine Learning Product Managers Meetup Event
Machine Learning Product Managers Meetup Event
 
data-science-lifecycle-ebook.pdf
data-science-lifecycle-ebook.pdfdata-science-lifecycle-ebook.pdf
data-science-lifecycle-ebook.pdf
 
On Machine Learning Readiness
On Machine Learning ReadinessOn Machine Learning Readiness
On Machine Learning Readiness
 
Recsys 2016
Recsys 2016Recsys 2016
Recsys 2016
 
From DevOps to MLOps: practical steps for a smooth transition
From DevOps to MLOps: practical steps for a smooth transitionFrom DevOps to MLOps: practical steps for a smooth transition
From DevOps to MLOps: practical steps for a smooth transition
 
qualitative.ppt
qualitative.pptqualitative.ppt
qualitative.ppt
 
Using Computer as a Research Assistant in Qualitative Research
Using Computer as a Research Assistant in Qualitative ResearchUsing Computer as a Research Assistant in Qualitative Research
Using Computer as a Research Assistant in Qualitative Research
 

Recently uploaded

My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.
rwarrenll
 
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
74nqk8xf
 
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
74nqk8xf
 
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Aggregage
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
u86oixdj
 
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
Social Samosa
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
vikram sood
 
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
mbawufebxi
 
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
v3tuleee
 
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
ahzuo
 
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
Timothy Spann
 
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
slg6lamcq
 
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
g4dpvqap0
 
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
slg6lamcq
 
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
nyfuhyz
 
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
ahzuo
 
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
g4dpvqap0
 
原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样
原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样
原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样
u86oixdj
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
manishkhaire30
 
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
v7oacc3l
 

Recently uploaded (20)

My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.
 
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
 
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
 
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
 
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
 
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
 
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
 
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
 
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
 
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
 
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
 
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
 
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
 
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
 
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
 
原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样
原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样
原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
 
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
 

2019-04-11 Databricks Unified Analytics Workshop - Outreach Case Study

  • 1. Databricks Workshop - April 11th 2019 Case Study: Outreach Andrew Brooks, Li Dong, Jiwei Cao
  • 2. Content Overview 1. Introduction to Outreach a. #1 Sales Engagement Platform b. Data Science + Sales Engagement 2. Outreach with Databricks a.Case Study - Production: Out-of-office Data Extraction b.Case Study - Research: Intent Classification 2
  • 3. Introduction to Outreach The #1 Sales Engagement Platform
  • 4. Wait, what is a Sales Engagement Platform? 4
  • 6. It’s also a new category of software. Add content 6 Source: https://blog.topohq.com/sales-engagement-the-definitive-guide/
  • 7. How about an Example? 7 Automates execution of some sales tasks: Emails, Linkedin Messages etc. Schedules and reminds the rep when it is the right time to do the manual tasks (e.g. phone call, custom manual email) SEP Encodes and Automates Sales Activities into Workflows/Pipelines
  • 8. Data Science + Sales Engagement Outreach ML Features: - Automation - Information Extraction - A/B testing - Advanced Analytics (dashboard & reporting) Optimization: - Intent & Topic Detection - Content & Action Recommendation - Prioritization & Forecasting 8 Highlighted use case Phone Email LinkedIn Meetings Data Sources:
  • 9. Case Study: Out-of-office Data Extraction
  • 10. Automation - Information Extraction Highlighted use case: Out of Office return date and referral extraction 10
  • 11. 11 Production Architecture Highlighted use case: OOO Information Extraction
  • 12. 12 Something to improve - Separate stacks to develop ML model and deploy model - jupyter notebooks - databricks notebooks - Docker, K8s(production) - Lack of ML model life-cycle management - model training - experiment(alpha, beta) - production to GA - model iterations / releases Production Architecture Highlighted use case: OOO Information Extraction
  • 13. 13 Production (Next Step…) Highlighted use case: OOO Information Extraction
  • 14. Case Study: Intent Classification
  • 15. Intent Classification: Provide More Value Metrics 15
  • 16. Intent Classification: Problem Solving Steps to solve it with NLP/Machine Learning: 1. Annotate some emails 2. Setup the Experiment Environment(Spark, NLP/ML-Packages) 3. Write Code and Running Experiments 4. Analyze Experiment Results 16 Goal: Classify email replies into 3 categories: positive, objection and unsubscription.
  • 17. Doesn’t Look Complicated, But Painful Pain Points: ● Difficult to setup and maintain a proper environment. ● Can’t run multiple Experiments at the same time ● Experiment Results are scattered in multiple files. Hard to navigate and analysis. 17
  • 18. Intent Classification: Problem Solving Steps to explore ideas with Databricks: 1. Annotate some emails 2. Setup the Experiment Environment(Spark, NLP/ML-Packages) 3. Write Code and Running Experiments 4. Analyze Experiment Results 5. Visualization the model prediction 18 Goal: Classify email replies into positive, objection and unsubscription.
  • 19. Visualize the Model Prediction: t-SNE 19 Before Training After Training
  • 20. How does Databricks help us • Setup a Dedicated Environment with Less Effort • Running Experiments at Scale • Analyze Experiment Results at One Place 20
  • 21. Q & A