Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep Learning Streaming Platform with Kafka Streams, TensorFlow, DeepLearning4J, H2O

17,966 views

Published on

Talk from JavaOne 2017: Apache Kafka + Kafka Streams for Scalable, Mission Critical Deep Learning.

Intelligent real time applications are a game changer in any industry. Deep Learning is one of the hottest buzzwords in this area. New technologies like GPUs combined with elastic cloud infrastructure enable the sophisticated usage of artificial neural networks to add business value in real world scenarios. Tech giants use it e.g. for image recognition and speech translation. This session discusses some real-world scenarios from different industries to explain when and how traditional companies can leverage deep learning in real time applications.

This session shows how to deploy Deep Learning models into real time applications to do predictions on new events. Apache Kafka will be used to execute analytic models in a highly scalable and performant way.

The first part introduces the use cases and concepts behind Deep Learning. It discusses how to build Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and Autoencoders leveraging open source frameworks like TensorFlow, DeepLearning4J or H2O.

The second part shows how to deploy the built analytic models to real time applications leveraging Apache Kafka as streaming platform and Apache Kafka’s Streams API to embed the intelligent business logic into any external application or microservice.

Some further material around Apache Kafka and Machine Learning:
- Blog Post: How to Build and Deploy Scalable Machine Learning in Production with Apache Kafka: https://www.confluent.io/blog/build-deploy-scalable-machine-learning-production-apache-kafka/
- Video: Build and Deploy Analytic Models with H2O.ai and Apache Kafka: https://www.youtube.com/watch?v=-q7CyIExBKM&feature=youtu.be
- Code: Github Examples using Apache Kafka, TensorFlow, H2O, DeepLearning4J: https://github.com/kaiwaehner/kafka-streams-machine-learning-examples

Published in: Technology

Deep Learning Streaming Platform with Kafka Streams, TensorFlow, DeepLearning4J, H2O

  1. 1. 1Confidential Kafka Streams + Deep Learning TensorFlow and H2O Applied to Real Time Streaming Applications Kai Waehner Technology Evangelist kontakt@kai-waehner.de LinkedIn @KaiWaehner www.kai-waehner.de
  2. 2. 2Apache Kafka and Deep Learning – Kai Waehner Agenda 1) Deep Learning and Real Time World 2) Deep Learning and the Apache Kafka Ecosystem 3) Building Neural Networks with TensorFlow and H2O 4) Deployment of Neural Networks with Kafka Streams
  3. 3. 3Apache Kafka and Deep Learning – Kai Waehner Agenda 1) Deep Learning and Real World Applications 2) Deep Learning and the Apache Kafka Ecosystem 3) Building Neural Networks with TensorFlow and H2O 4) Deployment of Neural Networks with Kafka Streams
  4. 4. 4Apache Kafka and Deep Learning – Kai Waehner Machine Learning ... allows computers to find hidden insights without being explicitly programmed where to look. Machine Learning • Decision Trees • Naïve Bayes • Clustering • Neural Networks • etc. Deep Learning • CNN • RNN • Autoencoder • etc.
  5. 5. 5Apache Kafka and Deep Learning – Kai Waehner Real World Examples of Deep Learning Language Translation https://recombu.com/mobile/article/google-instant-voice-translation-the-end-of-franglais_M11397.html http://wellroundedfashion.com/2015/01/13/trendy-tech-camouflage-from-facial-detection/ http://www.washington.edu/news/2017/07/11/lip-syncing-obama-new-tools-turn-audio-clips-into-realistic-video/ https://www.technologyreview.com/s/602796/starcraft-will-become-the-next-big-playground-for-ai/ Speech and Video Generation Picture Analysis (Face, Location, Items, …) Intelligent Decision Making
  6. 6. 6Apache Kafka and Deep Learning – Kai Waehner Deep Learning to Analyze and Act on Critical Business Moments Seconds Minutes Hours Price Optimization Predictive Maintenance Fraud Detection Cross Selling Transportation Rerouting Customer Service Inventory Management Windows of Opportunity
  7. 7. 7Apache Kafka and Deep Learning – Kai Waehner Live Demo Deep Learning in Action http://playground.tensorflow.org/
  8. 8. 8Apache Kafka and Deep Learning – Kai Waehner Agenda 1) Deep Learning and Real World Applications 2) Deep Learning and the Apache Kafka Ecosystem 3) Building Neural Networks with TensorFlow and H2O 4) Deployment of Neural Networks with Kafka Streams
  9. 9. 9Apache Kafka and Deep Learning – Kai Waehner Apache Kafka Ecosystem and Deep Learning Kafka Streams Kafka Connect Rest Proxy Schema Registry Go / .NET / Python Kafka Producer KSQL Kafka Streams
  10. 10. STREAMING PLATFORM BIG DATA ANALYTICS Oracle DB CoaP IoT Kafka Java Client ….. HP Vertica K A F K A H2O.ai, TensorFlow Batch Real Time Confluent REST Proxy MQTT IoT iPhone App Kafka Go Client C K O A N F N K E A C T H I V E Grafana Kafka Java EE Web App Spark C K O A N F N K E A C T Confluent Schema Registry Kafka Streams H2O.ai Mesos Kafka Streams TensorFlow Kubernetes Avro Avro 1) Data Producer 2) Analytics Platform 3) Streaming Platform 4) Data Consumer
  11. 11. 11Apache Kafka and Deep Learning – Kai Waehner Agenda 1) Deep Learning and Real World Applications 2) Deep Learning and the Apache Kafka Ecosystem 3) Building Neural Networks with TensorFlow and H2O 4) Deployment of Neural Networks with Kafka Streams
  12. 12. 12Apache Kafka and Deep Learning – Kai Waehner Languages, Frameworks and Tools for Deep Learning There is no Allrounder! Portable Format for Analytics (PFA)
  13. 13. 13Confidential Deep Learning with H2O.ai H2O Engine R / Python / Scala / Flow UI Java Code
  14. 14. 14Apache Kafka and Deep Learning – Kai Waehner Live Demo Use Case: Airline Flight Delay Prediction Machine Learning Algorithm: Deep Learning using Neural Networks Technology: H2O.ai, TensorFlow
  15. 15. 15Apache Kafka and Deep Learning – Kai Waehner H2O Deep Water (TensorFlow, MXNet, …) https://h2o-release.s3.amazonaws.com/h2o/rel-vapnik/1/ docs-website/h2o-docs/booklets/DeepWaterBooklet.pdf Deep Water (H2O + TensorFlow) Pre-Defined Networks + User-Defined Networks
  16. 16. 16Apache Kafka and Deep Learning – Kai Waehner Agenda 1) Deep Learning and Real World Applications 2) Deep Learning and the Apache Kafka Ecosystem 3) Building Neural Networks with TensorFlow and H2O 4) Deployment of Neural Networks with Kafka Streams
  17. 17. 17Apache Kafka and Deep Learning – Kai Waehner Stream Processing Data at Rest Data in Motion
  18. 18. 18Apache Kafka and Deep Learning – Kai Waehner Stream Processing Pipeline APIs Adapters / Channels Integration Messaging Stream Ingest Transformation Aggregation Enrichment Filtering Stream Preprocessing Process Management Analytics (Real Time) Applications & APIs Analytics / DW Reporting Stream Outcomes • Contextual Rules • Windowing • Patterns • Analytics • Machine Learning • … Stream Analytics Index / SearchNormalization Applying an Analytic Model is just a piece of the puzzle!
  19. 19. 19Apache Kafka and Deep Learning – Kai Waehner Kafka Streams (shipped with Apache Kafka) Map, filter, aggregate, apply analytic model, „any business logic“ Input Stream (Kafka Topic) Kafka Cluster Output Stream (Kafka Topic) Kafka Cluster Stream Processing Microservice (Kafka Streams) Deployed Anywhere Java App, Docker, Kubernetes, Mesos, “you-name-it”
  20. 20. 20Apache Kafka and Deep Learning – Kai Waehner When to use Kafka Streams for Stream Processing?
  21. 21. 21Apache Kafka and Deep Learning – Kai Waehner A complete streaming microservices, ready for production at large-scale Word Count App configuration Define processing (here: WordCount) Start processing
  22. 22. 22Apache Kafka and Deep Learning – Kai Waehner Use Case: Airline Flight Delay Prediction Machine Learning Algorithm: Neural Network built with H2O and TensorFlow Streaming Platform: Apache Kafka and Kafka Streams Live Demo
  23. 23. 23Apache Kafka and Deep Learning – Kai Waehner H2O.ai Model + Kafka Streams Filter Map 1) Create H2O DL model 2) Configure Kafka Streams Application 3) Apply H2O DL model to Streaming Data 4) Start Kafka Streams App
  24. 24. 24Apache Kafka and Deep Learning – Kai Waehner Github Examples: Kafka + Deep Learning https://github.com/kaiwaehner/kafka-streams-machine-learning-examples 1) git clone … 2) mvn clean package … 3) look at implementations and unit tests
  25. 25. 25Apache Kafka and Deep Learning – Kai Waehner Online Model Training with Apache Kafka and Kafka Streams How to improve models? 1. Manual Update 2. Automated Batch 3. Real Time Your choice… All possible with Kafka!
  26. 26. 26Apache Kafka and Deep Learning – Kai Waehner Caveats for Online Model Training • Processes and infrastructure not ready • Validation needed before production • Slows down the system • Only a few ML implementations à Build your own! • Many use cases do not need it à Do it only when really needed!
  27. 27. 27Apache Kafka and Deep Learning – Kai Waehner Key Take-Aways à Data Scientist and Developers have to work together continuously (org + tech!) à Mission critical, scalable production deployment is key for success of Machine Learning projects à Apache Kafka Ecosystem for Batch and Real Time Machine Learning (Training, Inference, Monitoring)
  28. 28. 28Apache Kafka and Deep Learning – Kai Waehner Confluent KSQL à www.confluent.io/product/ksql Use SQL-like queries to write stream processing applications with Kafka Streams Leverage Kafka and Kafka Streams from anywhere without coding • Streaming ETL • Anomaly Detection • Real Time Monitoring • etc. SELECT STREAM CEIL(timestamp TO HOUR) AS timeWindow, productId, COUNT(*) AS hourlyOrders, SUM(units) AS units FROM Orders GROUP BY CEIL(timestamp TO HOUR), productId; timeWindow | productId | hourlyOrders | units ------------+-----------+--------------+------- 08:00:00 | 10 | 2 | 5 08:00:00 | 20 | 1 | 8 09:00:00 | 10 | 4 | 22 09:00:00 | 40 | 1 | 45 ... | ... | ... | ... à … maybe also use it to apply analytic models via user defined functions? J
  29. 29. 29Apache Kafka and Deep Learning – Kai Waehner Kai Waehner Technology Evangelist kontakt@kai-waehner.de @KaiWaehner www.kai-waehner.de LinkedIn Questions? Feedback? Please contact me!

×