Generic Short Text Classifier

1. Ham or Spam?
Generic Short Text
Classifier
Romit Singhai

2. Some Stats
• 241 million active user accounts per month
• 5% are spam account

3. Some More Stats…
• 130 million mobile users
• Two thirds of mobile users received SMS spam

4. Data Pipeline
Acquisition
(TwiAer
API)
Cleaning
and
Munging
Feature
Engineering
Feature
Selection
Feature
Extraction

5. Text Classification

6. Text Classification

7. Confusion Matrix

8. Feature Engineering

9. Feature Engineering

10. Generic Classifier
• Added SMS data to Twitter Data
o Tweet + SMS (AUC=0.91)
• Just meta features
o Random Forest Classifier(Mean Accuracy 0.70)

11. Benefits
• Reduce operational cost
• Improve Security
• Supports Revenue Model

12. Next Steps…
• Incorporate data from other platforms
• Analyze and Extract more meta features
• “Catch All” Classifier as API

13. Thanks
Romit Singhai
romits@gmail.com