SlideShare a Scribd company logo
Ham  or  Spam?  
Generic  Short  Text  
Classifier	
Romit Singhai
Some  Stats	
•  241 million active user accounts per month
•  5% are spam account
Some  More  Stats…	
•  130 million mobile users
•  Two thirds of mobile users received SMS spam
Data  Pipeline	
Acquisition
(TwiAer  
API)	
Cleaning  
and  
Munging	
Feature  
Engineering	
Feature  
Selection	
Feature  
Extraction
Text  Classification
Text  Classification
Confusion  Matrix
Feature  Engineering
Feature  Engineering
Generic  Classifier	
•  Added SMS data to Twitter Data
o  Tweet + SMS (AUC=0.91)
•  Just meta features
o  Random Forest Classifier(Mean Accuracy 0.70)
Benefits	
•  Reduce operational cost
•  Improve Security
•  Supports Revenue Model
Next  Steps…	
•  Incorporate data from other platforms
•  Analyze and Extract more meta features
•  “Catch All” Classifier as API
Thanks	
Romit Singhai
romits@gmail.com

More Related Content

Similar to Generic Short Text Classifier

Travis Wright - Complete it service management
Travis Wright - Complete it service managementTravis Wright - Complete it service management
Travis Wright - Complete it service management
Nordic Infrastructure Conference
 
It automation webinar presentation
It automation webinar presentationIt automation webinar presentation
It automation webinar presentation
Auxis Consulting & Outsourcing
 
It automation webinar presentation
It automation webinar presentation It automation webinar presentation
It automation webinar presentation
Auxis Consulting & Outsourcing
 
Customer case - Dynatrace Monitoring Redefined
Customer case - Dynatrace Monitoring RedefinedCustomer case - Dynatrace Monitoring Redefined
Customer case - Dynatrace Monitoring Redefined
Michel Duruel
 
Twitter sentiment analysis ppt
Twitter sentiment analysis pptTwitter sentiment analysis ppt
Twitter sentiment analysis ppt
AntaraBhattacharya12
 
Building data-driven microservices
Building data-driven microservicesBuilding data-driven microservices
Building data-driven microservices
Streamlio
 
Dreamforce 2017 - Advanced Logging Patterns with Platform Events
Dreamforce 2017 - Advanced Logging Patterns with Platform EventsDreamforce 2017 - Advanced Logging Patterns with Platform Events
Dreamforce 2017 - Advanced Logging Patterns with Platform Events
andyinthecloud
 
Prometheus Is Good for Your Small Startup - ShuttleCloud Corp. - 2016
Prometheus Is Good for Your Small Startup - ShuttleCloud Corp. - 2016Prometheus Is Good for Your Small Startup - ShuttleCloud Corp. - 2016
Prometheus Is Good for Your Small Startup - ShuttleCloud Corp. - 2016
ShuttleCloud
 
Best Practices for Streaming IoT Data with MQTT and Apache Kafka
Best Practices for Streaming IoT Data with MQTT and Apache KafkaBest Practices for Streaming IoT Data with MQTT and Apache Kafka
Best Practices for Streaming IoT Data with MQTT and Apache Kafka
Kai Wähner
 
Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...
Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...
Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...
confluent
 
Best Practices for Streaming IoT Data with MQTT and Apache Kafka®
Best Practices for Streaming IoT Data with MQTT and Apache Kafka®Best Practices for Streaming IoT Data with MQTT and Apache Kafka®
Best Practices for Streaming IoT Data with MQTT and Apache Kafka®
confluent
 
Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...
Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...
Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...
Amazon Web Services
 
API Management Microservices beyond HIP
API Management Microservices beyond HIPAPI Management Microservices beyond HIP
API Management Microservices beyond HIP
SmartWave
 
Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...
Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...
Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...
Computaris
 
Continuous Delivery Summit London '17
Continuous Delivery Summit London '17Continuous Delivery Summit London '17
Continuous Delivery Summit London '17
Sébastien Lemmens
 
ThirdEye - LinkedIn's Business-wide monitoring platform
ThirdEye - LinkedIn's Business-wide monitoring platformThirdEye - LinkedIn's Business-wide monitoring platform
ThirdEye - LinkedIn's Business-wide monitoring platform
Akshay Rai
 
Challenges of monitoring distributed systems
Challenges of monitoring distributed systemsChallenges of monitoring distributed systems
Challenges of monitoring distributed systems
Nenad Bozic
 
Paper Practical Itsm Transformation Qai V 1.0
Paper   Practical Itsm Transformation   Qai V 1.0Paper   Practical Itsm Transformation   Qai V 1.0
Paper Practical Itsm Transformation Qai V 1.0
Sukumar Daniel
 
Atlassian User Group NYC 092718 Event
Atlassian User Group NYC 092718 EventAtlassian User Group NYC 092718 Event
Atlassian User Group NYC 092718 Event
Marlon Palha
 
Microservices: Patterns & Practices
Microservices: Patterns & PracticesMicroservices: Patterns & Practices
Microservices: Patterns & Practices
Rafael Monteiro e Pereira
 

Similar to Generic Short Text Classifier (20)

Travis Wright - Complete it service management
Travis Wright - Complete it service managementTravis Wright - Complete it service management
Travis Wright - Complete it service management
 
It automation webinar presentation
It automation webinar presentationIt automation webinar presentation
It automation webinar presentation
 
It automation webinar presentation
It automation webinar presentation It automation webinar presentation
It automation webinar presentation
 
Customer case - Dynatrace Monitoring Redefined
Customer case - Dynatrace Monitoring RedefinedCustomer case - Dynatrace Monitoring Redefined
Customer case - Dynatrace Monitoring Redefined
 
Twitter sentiment analysis ppt
Twitter sentiment analysis pptTwitter sentiment analysis ppt
Twitter sentiment analysis ppt
 
Building data-driven microservices
Building data-driven microservicesBuilding data-driven microservices
Building data-driven microservices
 
Dreamforce 2017 - Advanced Logging Patterns with Platform Events
Dreamforce 2017 - Advanced Logging Patterns with Platform EventsDreamforce 2017 - Advanced Logging Patterns with Platform Events
Dreamforce 2017 - Advanced Logging Patterns with Platform Events
 
Prometheus Is Good for Your Small Startup - ShuttleCloud Corp. - 2016
Prometheus Is Good for Your Small Startup - ShuttleCloud Corp. - 2016Prometheus Is Good for Your Small Startup - ShuttleCloud Corp. - 2016
Prometheus Is Good for Your Small Startup - ShuttleCloud Corp. - 2016
 
Best Practices for Streaming IoT Data with MQTT and Apache Kafka
Best Practices for Streaming IoT Data with MQTT and Apache KafkaBest Practices for Streaming IoT Data with MQTT and Apache Kafka
Best Practices for Streaming IoT Data with MQTT and Apache Kafka
 
Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...
Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...
Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...
 
Best Practices for Streaming IoT Data with MQTT and Apache Kafka®
Best Practices for Streaming IoT Data with MQTT and Apache Kafka®Best Practices for Streaming IoT Data with MQTT and Apache Kafka®
Best Practices for Streaming IoT Data with MQTT and Apache Kafka®
 
Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...
Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...
Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...
 
API Management Microservices beyond HIP
API Management Microservices beyond HIPAPI Management Microservices beyond HIP
API Management Microservices beyond HIP
 
Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...
Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...
Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...
 
Continuous Delivery Summit London '17
Continuous Delivery Summit London '17Continuous Delivery Summit London '17
Continuous Delivery Summit London '17
 
ThirdEye - LinkedIn's Business-wide monitoring platform
ThirdEye - LinkedIn's Business-wide monitoring platformThirdEye - LinkedIn's Business-wide monitoring platform
ThirdEye - LinkedIn's Business-wide monitoring platform
 
Challenges of monitoring distributed systems
Challenges of monitoring distributed systemsChallenges of monitoring distributed systems
Challenges of monitoring distributed systems
 
Paper Practical Itsm Transformation Qai V 1.0
Paper   Practical Itsm Transformation   Qai V 1.0Paper   Practical Itsm Transformation   Qai V 1.0
Paper Practical Itsm Transformation Qai V 1.0
 
Atlassian User Group NYC 092718 Event
Atlassian User Group NYC 092718 EventAtlassian User Group NYC 092718 Event
Atlassian User Group NYC 092718 Event
 
Microservices: Patterns & Practices
Microservices: Patterns & PracticesMicroservices: Patterns & Practices
Microservices: Patterns & Practices
 

Recently uploaded

End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024
Lars Albertsson
 
Analysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performanceAnalysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performance
roli9797
 
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Kiwi Creative
 
A presentation that explain the Power BI Licensing
A presentation that explain the Power BI LicensingA presentation that explain the Power BI Licensing
A presentation that explain the Power BI Licensing
AlessioFois2
 
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
apvysm8
 
原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理
原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理
原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理
a9qfiubqu
 
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
mkkikqvo
 
Orchestrating the Future: Navigating Today's Data Workflow Challenges with Ai...
Orchestrating the Future: Navigating Today's Data Workflow Challenges with Ai...Orchestrating the Future: Navigating Today's Data Workflow Challenges with Ai...
Orchestrating the Future: Navigating Today's Data Workflow Challenges with Ai...
Kaxil Naik
 
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
nuttdpt
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
vikram sood
 
The Ipsos - AI - Monitor 2024 Report.pdf
The  Ipsos - AI - Monitor 2024 Report.pdfThe  Ipsos - AI - Monitor 2024 Report.pdf
The Ipsos - AI - Monitor 2024 Report.pdf
Social Samosa
 
Build applications with generative AI on Google Cloud
Build applications with generative AI on Google CloudBuild applications with generative AI on Google Cloud
Build applications with generative AI on Google Cloud
Márton Kodok
 
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
nyfuhyz
 
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
v7oacc3l
 
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
bopyb
 
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
Timothy Spann
 
writing report business partner b1+ .pdf
writing report business partner b1+ .pdfwriting report business partner b1+ .pdf
writing report business partner b1+ .pdf
VyNguyen709676
 
一比一原版(harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(harvard毕业证书)哈佛大学毕业证如何办理一比一原版(harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(harvard毕业证书)哈佛大学毕业证如何办理
taqyea
 
University of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma TranscriptUniversity of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma Transcript
soxrziqu
 
一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
aqzctr7x
 

Recently uploaded (20)

End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024
 
Analysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performanceAnalysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performance
 
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
 
A presentation that explain the Power BI Licensing
A presentation that explain the Power BI LicensingA presentation that explain the Power BI Licensing
A presentation that explain the Power BI Licensing
 
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
 
原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理
原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理
原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理
 
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
 
Orchestrating the Future: Navigating Today's Data Workflow Challenges with Ai...
Orchestrating the Future: Navigating Today's Data Workflow Challenges with Ai...Orchestrating the Future: Navigating Today's Data Workflow Challenges with Ai...
Orchestrating the Future: Navigating Today's Data Workflow Challenges with Ai...
 
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
 
The Ipsos - AI - Monitor 2024 Report.pdf
The  Ipsos - AI - Monitor 2024 Report.pdfThe  Ipsos - AI - Monitor 2024 Report.pdf
The Ipsos - AI - Monitor 2024 Report.pdf
 
Build applications with generative AI on Google Cloud
Build applications with generative AI on Google CloudBuild applications with generative AI on Google Cloud
Build applications with generative AI on Google Cloud
 
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
 
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
 
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
 
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
 
writing report business partner b1+ .pdf
writing report business partner b1+ .pdfwriting report business partner b1+ .pdf
writing report business partner b1+ .pdf
 
一比一原版(harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(harvard毕业证书)哈佛大学毕业证如何办理一比一原版(harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(harvard毕业证书)哈佛大学毕业证如何办理
 
University of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma TranscriptUniversity of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma Transcript
 
一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
 

Generic Short Text Classifier