Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Streaming Data Platforms & Building Data-Centric Products (w/ Brandon O'Brien)

189 views

Published on

Presented at Eastside DevOps Meetup Feb 20th at Expedia (https://www.meetup.com/Eastside-DevOps-Meetup/events/237421589/)

This presentation focused on reusable patterns for create a self-service data platform that facilitates data sharing across your organization, to make it easier to build data-powered products faster. Top patterns include making it easier for data producers to get data onto the platform, and to quickly spin up new data consumers using templated Apache Spark and Apache Storm apps.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Streaming Data Platforms & Building Data-Centric Products (w/ Brandon O'Brien)

  1. 1. Streaming Data Platforms Brandon O’Brien @hakczar Expedia, Inc
  2. 2. Outline  Use Cases  Streaming Data Opportunities & Challenges  Streaming Data Platform at Expedia  Questions?
  3. 3. Outline  Use Cases  Streaming Data Opportunities & Challenges  Streaming Data Platform at Expedia  Questions?
  4. 4. Use Case: Streaming Analytics
  5. 5. Outline  Use Cases  Streaming Data Opportunities & Challenges  Streaming Data Platform at Expedia  Questions?
  6. 6. Streaming Data: Opportunities  Data -> Insights -> Products -> $$$  Fresher data = better  However…
  7. 7. Streaming Data: Challenges  Today’s Challenge: System integration  System level: N^2 integrations  Team level: Roadmap alignment
  8. 8. Streaming Data: Solutions  Solution:  Kafka with Streaming Data Platform  Decoupled systems & roadmaps  Easy publish  Easy consume  Central data depot  Talk Focus:  Streaming Data Platform that makes life easier for Kafka producers and consumers
  9. 9. Outline  Use Cases  Streaming Data Opportunities & Challenges  Streaming Data Platform at Expedia  Questions?
  10. 10. Platform Architecture
  11. 11. Streaming Data Platform  Self-Service  Centralized Data Depot (logical)  DevOps: Primer, CICD, Cloud  Elastic Components  Monitoring: Velocity, Lag
  12. 12. Streaming Data Platform cont’d  Producers  Publish data to HTTP Ingestor endpoint  Ingestor -> Kafka  Ingestor -> S3  Consumers  Create skeleton apps with built-in Kafka integration  CICD pipeline for deployments  Changes the Conversation
  13. 13. Streaming Data Platforms Brandon O’Brien @hakczar Expedia, Inc Questions?

×