Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Brandon obrien streaming_data

177 views

Published on

Brandon O'Brien, Principal Software Engineer at Expedia, Inc

Published in: Engineering
  • Be the first to comment

Brandon obrien streaming_data

  1. 1. Streaming Data Ecosystems Brandon O’Brien @hakczar Expedia, Inc
  2. 2. Outline  Streaming Data Opportunities & Challenges  Streaming Data Ecosystem at Expedia  Use Cases  Questions?
  3. 3. Outline  Streaming Data Opportunities & Challenges  Streaming Data Ecosystem at Expedia  Use Cases  Questions?
  4. 4. Streaming Data: Opportunities  Data -> Insights -> Data Products -> $$$  Fresher data = better  However…
  5. 5. Streaming Data: Challenges  Today’s Challenge: System integration  N^2 integrations  Team roadmaps need to align
  6. 6. Streaming Data: Solutions  Solution:  Kafka with Streaming Data Ecosystem  Decoupled systems & roadmaps  Easy publish  Easy consume  Central data depot  Talk Focus:  Streaming Data Ecosystem that makes life easier for Kafka producers and consumers
  7. 7. Outline  Streaming Data Opportunities & Challenges  Streaming Data Ecosystem at Expedia  Use Cases  Questions?
  8. 8. Ecosystem Architecture
  9. 9. Streaming Data Ecosystem  Self-Service  Centralized Data Depot (logical)  Primer, CICD, Cloud  Elastic Components  Monitoring: Velocity, Lag
  10. 10. Streaming Data Ecosystem cont’d  Producers  Publish data to HTTP Ingestor endpoint  Ingestor -> Kafka  Ingestor -> S3  Consumers  Create skeleton apps with built-in Kafka integration  CICD pipeline for deployments
  11. 11. Outline  Streaming Data Opportunities & Challenges  Streaming Data Ecosystem at Expedia  Use Cases  Questions?
  12. 12. Use Case: Streaming Analytics
  13. 13. Streaming Data Ecosystems Brandon O’Brien @hakczar Expedia, Inc Questions?

×