Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Data Pipelines With Streamsets

102 views

Published on

How to create data pipelines with Streamsets.

Published in: Technology
  • Be the first to comment

Data Pipelines With Streamsets

  1. 1. Data Pipelines With Streamsets Jowanza Joseph @jowanza
  2. 2. Agenda About me The Problem Space Streaming StreamSets Demo Questions
  3. 3. About Me Software Engineer at One ClickRetail Scala / Spark / Mesos / Kubernetes Author: Apache Spark Fieldbook Cyclist Husband and father
  4. 4. Retail Intelligence
  5. 5. Data Size
  6. 6. Real-Time
  7. 7. Operational Complexity
  8. 8. Batch Processing
  9. 9. What Are Data Pipelines?
  10. 10. What Problems Do They Solve?
  11. 11. Scalability Complexity Observability Extendability
  12. 12. Lambda Architecture
  13. 13. Kappa Architecture
  14. 14. Goals Data Provenance Guaranteed Delivery Configurable Extendable Multi-Protocol Support DAG Distribute
  15. 15. Based on Streams
  16. 16. Architecture
  17. 17. Running on Mesos
  18. 18. Analytics Data
  19. 19. Real-Time Data
  20. 20. Our Use Case
  21. 21. Demo

×