More Related Content Similar to Promoting a Data Driven Culture in a Microservices Environment Similar to Promoting a Data Driven Culture in a Microservices Environment (20) Promoting a Data Driven Culture in a Microservices Environment2. Overview
1. Introduction to Hudl
2. Hudl Data Journey
3. #DataProblems
4. Data Engineering
5. Data Analytics
6. Key Takeaways
7. Summary
7. Capture and
bring value to
every moment
in sports.
4.9 million
users
150 thousand
teams
4.5 billion
video views last 12 months
28. ● SQL
● Fully managed on AWS
● Reasonably priced
Amazon Redshift
29. ● SQL
● Fully managed on AWS
● Reasonably priced
Rob Story, Data Engineering
Architecture at Simple, PyData
Chicago
Amazon Redshift
30. For the Google Cloud User:
Google BigQuery
For the Do-it-yourself-er:
Hive / Impala / PrestoDB / Druid
For the Enterprise User:
Vertica / Teradata ?
Alternatives
57. ● Everyone has access -- 430+ Hudlies
● Lots of data
○ 24+ TB
○ 100B+ rows
Our needs
59. ● Open source (Python!)
● Query editor +
visualizations
● Hosted version or host
your own
re:dash
68. ● Relational Database Model
● Basic & intermediate SQL
● Table Familiarity
● Using re:dash
● Data Visualization
Certification Topics
73. “Find how many football teams
had 3 or more users watch video in
3 different months this year.”
81. September Stats
● 194 unique users executed a query
● 14,000 ad hoc queries executed
● 940 unique scheduled queries/week
83. ● Being Data-driven is a team sport
● Get the data architecture in place
● Make data and metrics accessible
● Be Flexible
Key Takeaways
84. Summary
1. Introduction to Hudl
2. Hudl Data Journey
3. #DataProblems
4. Data Engineering
5. Data Analytics
6. Key Takeaways
7. Summary
85. Tools we use
Summary
Jenkins Scheduling
Luigi Workflow management
Sqoop RDBMS Extraction
Spark Data transformation
AWS Lambda Event-driven processing
Redshift Data warehouse
re:dash Query interface + visualization