Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
ADVANCED SPARK
AND TENSORFLOW MEETUP
Updates and Announcements
August 4, 2016
advancedspark.com
MEETUP METRICS
• 4,000 Members in 1Year!?
• Github Repo: 600 Stars, 230 Forks
• DockerHub Repo: 5,400 Pulls!!
MEETUP AGENDA
• Deploying and Scaling ML Models with
PipelineIO Open Source (Chris Fregly)
• Fundamental Algorithms of Neu...
PIPELINE.IO
ExtendingYour ML Pipelines into Production
100% Open Source!
http://github.com/fluxcapacitor/pipeline
BRAINSTORMING AND VALIDATING
• Major Gaming Company
• Large Ride Sharing Service
• Popular Q & A Site
• Online Clothing Re...
PIPELINE.IO FOCUS
• Model Deploying andTesting
• Model Scaling and Serving
• Online ModelTraining
• Dynamic Model Optimizi...
MODEL DEPLOYING AND TESTING
Continuously Test
and Deploy Models
in Production!
DEMO!
Deploy Spark ML DecisionTree to Production
Deploy to Cloud
or On-Premise!
MODEL SCALING AND SERVING
DEMO!
Circuit Breakers and Request Batching
ONLINE MODEL TRAINING
• Continuous, Incremental, and Partial Training
• Kafka + Spark Streaming + Spark ML
• Real-time, Dy...
DEMO!
Real-time, Dynamic Recommendations
DYNAMIC MODEL OPTIMIZING
Generate Optimized
Code from Spark ML!
DEMO!
Dynamic Code Generation of DecisionTree
BECOME A CONTRIBUTOR!
WE’RE HIRING!!
• Kafka, Spark ML, and TensorFlow Contributors
• Systems Engineers
• GPU/CUDA Engineers
• C++, Java, Scala,...
SANTA CLARA WORKSHOP, AUG 6, 2016
Upcoming SlideShare
Loading in …5
×

Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline Deploy to Production

1,261 views

Published on

Empowering the Data Scientist with "1-Click" Production Deployment and Canary Testing of High-Performance and Highly-Scalable Spark ML and TensorFlow Models directly from Jupyter/iPython Notebooks using Docker, Kubernetes, Netflix OSS, Microservices, and Spinnaker.

With proper tooling and metrics, Data Scientists can directly deploy, analyze, A/B test, rollback, and scale out their Spark ML and TensorFlow model into live production serving with zero friction.

We will show you the open source tools that we've built based on Docker, Kubernetes, Netflix Open Source, Microservices, Spinnaker - and even Chaos Monkey!

Speaker: Chris Fregly @ PipelineIO, formerly Databricks and Netflix

Published in: Software
  • Be the first to comment

Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline Deploy to Production

  1. 1. ADVANCED SPARK AND TENSORFLOW MEETUP Updates and Announcements August 4, 2016 advancedspark.com
  2. 2. MEETUP METRICS • 4,000 Members in 1Year!? • Github Repo: 600 Stars, 230 Forks • DockerHub Repo: 5,400 Pulls!!
  3. 3. MEETUP AGENDA • Deploying and Scaling ML Models with PipelineIO Open Source (Chris Fregly) • Fundamental Algorithms of Neural Networks (Sam Abrahams)
  4. 4. PIPELINE.IO ExtendingYour ML Pipelines into Production 100% Open Source! http://github.com/fluxcapacitor/pipeline
  5. 5. BRAINSTORMING AND VALIDATING • Major Gaming Company • Large Ride Sharing Service • Popular Q & A Site • Online Clothing Retailer • DominantVideo Streaming
  6. 6. PIPELINE.IO FOCUS • Model Deploying andTesting • Model Scaling and Serving • Online ModelTraining • Dynamic Model Optimizing
  7. 7. MODEL DEPLOYING AND TESTING Continuously Test and Deploy Models in Production!
  8. 8. DEMO! Deploy Spark ML DecisionTree to Production Deploy to Cloud or On-Premise!
  9. 9. MODEL SCALING AND SERVING
  10. 10. DEMO! Circuit Breakers and Request Batching
  11. 11. ONLINE MODEL TRAINING • Continuous, Incremental, and Partial Training • Kafka + Spark Streaming + Spark ML • Real-time, Dynamic Recommendations
  12. 12. DEMO! Real-time, Dynamic Recommendations
  13. 13. DYNAMIC MODEL OPTIMIZING Generate Optimized Code from Spark ML!
  14. 14. DEMO! Dynamic Code Generation of DecisionTree
  15. 15. BECOME A CONTRIBUTOR!
  16. 16. WE’RE HIRING!! • Kafka, Spark ML, and TensorFlow Contributors • Systems Engineers • GPU/CUDA Engineers • C++, Java, Scala, Python ONLY NICE PEOPLE!!
  17. 17. SANTA CLARA WORKSHOP, AUG 6, 2016

×