Python as part of a production machine learning stack by Michael Manapat PyData SV 2014


Published on

Over the course of three years, we've built Stripe from scratch and scaled it to process billions of dollars of transaction volume a year by making it easy and painless for merchants to get set up and start accepting payments. While the vast majority of transactions facilitated by Stripe are honest, we do need to protect our merchants from rogue individuals and groups seeing to "test" or "cash" stolen credit cards. To combat this sort of activity, Stripe uses Python (together with Scala and Ruby) as part of its production machine learning pipeline to detect and block fraud in real time. In this talk, I'll go through the scikit-based modeling process for a sample data set that is derived from production data to illustrate how we train and validate our models. We'll also walk through how we deploy the models and monitor them in our production environment and how Python has allowed us to do this at scale.

Published in: Technology, Education

Python as part of a production machine learning stack by Michael Manapat PyData SV 2014

  1. 1. Python  as  part  of  a  produc0on   machine  learning  stack         Michael  Manapat   @mlmanapat   Stripe    
  2. 2. Outline     -­‐Why  we  need  ML  at  Stripe   -­‐Simple  models  with  sklearn   -­‐Pipelines  with  Luigi   -­‐Scoring  as  a  service    
  3. 3. Stripe  is  a  technology  company   focusing  on  making  payments  easy     -­‐Short  applica>on    
  4. 4. Tokeniza0on       Customer   browser   Stripe   Stripe.js   Token   Merchant   server   Stripe   API  call   Result  
  5. 5. API  Call     import stripe
  6. 6. Fraud  /  business  viola0ons     -­‐Terms  of  service  viola>ons  (weapons)   -­‐Merchant  fraud  (card  “cashers”)       -­‐Transac>on  fraud     -­‐No  machine  learning  a  year  ago  
  7. 7. Fraud  /  business  viola0ons     -­‐Terms  of  service  viola>ons     E-­‐cigareMes,  drugs,  weapons,  etc.     How  do  we  find  these  automa>cally?  
  8. 8. Merchant  sign  up  flow           Applica>on   submission   Website   scraped   Text  scored   Applica>on   reviewed  
  9. 9. Merchant  sign  up  flow           Applica>on   submission   Website   scraped   Text  scored   Applica>on   reviewed   Machine   learning   pipeline  and   service  
  10. 10. Building  a  classifier:  e-­‐cigareIes     data = pandas.from_pickle(‘ecigs’)
 text violator
 0 " please verify your age i am 21 years or older ... True
 1 coming soon toggle me drag me with your mouse ... False
 2 drink moscow mules cart 0 log in or create an ... False
 3 vapors electronic cigarette buy now insuper st... True
 4 t-shirts shorts hawaii about us silver coll... False
 [5 rows x 2 columns]  
  11. 11. Features  for  text  classifica0on     cv = CountVectorizer
 features = 
 Sparse  matrix  of  word  counts  from   input  text  (omiSng  feature  selec>on)  
  12. 12. Features  for  text  classifica0on   
 X_train, X_test, y_train, y_test = 
 features, data['violator'], 
 -­‐Avoid  leakage   -Other  cross-­‐valida>on  methods  
  13. 13. Training   
 model = LogisticRegression(), y_train)
 Serializer  reads  from   
  14. 14. Valida0on   
 probs = model.predict_proba(X_test)
 fpr, tpr, thresholds =
 roc_curve(y_test, probs[:, 1])
 matplotlib.pyplot(fpr, tpr)  
  15. 15. ROC:  Receiver  opera0ng  characteris0c   
  16. 16. Pipeline     -­‐Fetch  website  snapshots  from  S3   -­‐Fetch  classifica>ons  from  SQL/Impala   -­‐Sani>ze  text  (strip  HTML)   -­‐Run  feature  genera>on  and  selec>on   -­‐Train  and  serialize  model   -­‐Export  valida>on  sta>s>cs  
  17. 17. Luigi     class GetSnapshots(luigi.Task):
 def run(self):
 " "...
 class GenFeatures(luigi.Task):
 def requires(self):
 return GetSnapshots()"
  18. 18. Luigi  runs  tasks  on  Hadoop  cluster   "
  19. 19. Scoring  as  a  service     "Applica>on   submission   Website   scraped   Text  scored   Applica>on   reviewed   ThriO   RPC   Scoring   Service  
  20. 20. Scoring  as  a  service     struct ScoringRequest {
 1: string text
 2: optional string model_name
 struct ScoringResponse {
 1: double score" " "// Experiments?
 2: double request_duration
  21. 21. Why  a  service?     -­‐Same  code  base  for  training/scoring     -­‐Reduced  duplica>on/easier  deploys     -­‐Experimenta>on    
  22. 22. -­‐Log  requests    and  responses    (Parquet/Impala)     -­‐Centralized    monitoring    (Graphite)  
  23. 23. Summary     -­‐Simple  models  with  sklearn   -­‐Pipelines  with  Luigi   -­‐Scoring  as  a  service     Thanks!   @mlmanapat