Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Realtime predictive analytics using RabbitMQ & scikit-learn

3,185 views

Published on

In this talk, AWeber's Michael Becker describes how to deploy a predictive model in a production environment using RabbitMQ and scikit-learn. You'll see a realtime content classification system to demonstrate this design.

Published in: Technology, Education

Realtime predictive analytics using RabbitMQ & scikit-learn

  1. 1. Realtime Predictive Analytics Using scikit-learn and RabbitMQ Using scikit-learn and RabbitMQMichael Becker PyData Boston 2013
  2. 2. Who is this guy? Software Engineer @ AWeber Founder of the DataPhilly Meetup group @beckerfuffle beckerfuffle.com These slides and more @ github.com/mdbecker
  3. 3. What I'll cover •Model Distribution
  4. 4. What I'll cover •Model Distribution •Data flow
  5. 5. What I'll cover •Model Distribution •Data flow •RabbitMQ
  6. 6. What I'll cover •Model Distribution •Data flow •RabbitMQ •Demo
  7. 7. What I'll cover •Model Distribution •Data flow •RabbitMQ •Demo •Scalability
  8. 8. What I'll cover •Model Distribution •Data flow •RabbitMQ •Demo •Scalability •Other considerations
  9. 9. Supervised Learning
  10. 10. 38 top wikipedias Arabic ‫العربية‬ Bulgarian Български Catalan Català Czech Čeština Danish Dansk German Deutsch English English Spanish Español Estonian Eesti Basque Euskara Persian ‫فارسی‬ Finnish Suomi French Français Hebrew ‫עברית‬ Hindi िहिन्दी Croatian Hrvatski Hungarian Magyar Indonesian Bahasa Indonesia Italian Italiano Japanese 日本語 Kazakh Қазақша Korean 한국어 Lithuanian Lietuvių Malay Bahasa Melayu Dutch Nederlands Norwegian (Bokmål) Norsk (Bokmål) Polish Polski Portuguese Português Romanian Română Russian Русский Slovak Slovenčina Slovenian Slovenščina Serbian Српски / Srpski Swedish Svenska Turkish Türkçe Ukrainian Українська Vietnamese Tiếng Việt Waray-Waray Winaray
  11. 11. The model
  12. 12. Distributing the model
  13. 13. Data input
  14. 14. The client
  15. 15. Message loss
  16. 16. Enter RabbitMQ Reliability Flexible Routing Clustering HA Queues Many clients
  17. 17. AMQP Reliability Flexible Routing Clustering HA Queues Many clients
  18. 18. Data processing
  19. 19. The worker
  20. 20. The design
  21. 21. Demo time!
  22. 22. Demo time!
  23. 23. Demo time!
  24. 24. Scaling
  25. 25. Realtime vs batch
  26. 26. Monitoring
  27. 27. Load
  28. 28. Verify
  29. 29. Thank you API & Worker: Kelly O’Brien (linkedin.com/in/kellyobie) UI: Matt Parke (ordinaryrobot.com) Classifier: Michael Becker (github.com/mdbecker) Images: Wikipedia
  30. 30. My info Tweet me @beckerfuffle Find me at beckerfuffle.com These slides and more @ github.com/mdbecker

×