Snowflake LA Meetup Group - Hulu+Snowflake (Data Engineering Best Practices / Use cases) August Tuesday 20th

1. THE END GAME FOR
DATA
1
Oxana Urdaneta

2. Agenda
2
• Data at Hulu
• What about CONTENT Data?
• Use Cases
• Engineering Challenges
• What about SNOWFLAKE?
• How is it helping us?
• How is it helping our stakeholders?
• Recommendations in Snowflake
2

3. 3
Let’s Start with
the DATA

4. 10 Million Events/Minuteg
4
It all starts with the viewer...
35 PB of Data
>15M Unique
Devices per
Day
4
Data
Marts
Analytics
Third
Party
Targeting
Metrics
Data
Apps

5. 5
by user
by day
by hour
by device
by state
by Hulu product
by content
Data at Hulu

6. 6
What about
CONTENT Data?

7. Content Acquisition
How has the overall performance of my
content partners been?
How many users are engaging
with Kids content?
What other genres are Anime users
interested in?
What is the expected performance of this
new title?
How many users first engaged with any sports
network within their first day of signup?

8. Content Marketing
Was my content promoted to the right people?
Why did my content not perform the way
I expected it to?
Was the content efficient in attracting
new subscribers to the service?
What is the average age of users
watching The Handmaid’s Tale?

9. Content Finance & Planning
Did I pay the right price for this season?
Should I buy more of this content or
am I saturated?
What is the average cost per hour of
dramas?

10. 10
Engineering Challenges
Data Analysts
Data Scientists
Data Engineers
Data Modeling
Challenges
Data Availability
Challenges
DATA GOVERNANCE
DATA QUALITY
FIND OPTIMAL AND CREATIVE SOLUTIONS SO OUR DATA PIPELINES
SCALE ALONG WITH THE GROWTH OF OUR DATA

11. 11
Recommendations to overcome Data Modeling challenges
● Data Profiling
● Understand our data
● Partition/Clustering
● Consistency in our designs
● Flexibility vs Performance
● Capacity planning
● Data Governance

12. 12
Recommendations to overcome Data Availability challenges
● Technology Selection
● Be Data Driven
● Data Quality
● Query plans

13. 13
What about
SNOWFLAKE ?

14. 14
How is Snowflake helping us
● Transactional capabilities
● Data Warehouse principles
● Parallel Execution
● Multi-dimensional cubes

15. 15
How is Snowflake helping our Stakeholders
● PARTNERS
○ Getting their data on time
○ Taking advantage of Snowflake’s
Data Sharing capability (future)
● HULUGANS
○ We know there’s a cost
associated to it, but Snowflake
allows our Analysts to generate
insights in the blink of an eye
○ Centralized place to get data
Snowflake is Reliable

16. 16
Recommendations in
● Cluster your tables
○ When creating big tables
○ Check Query Profile for Pruning
● Pick the right warehouse
○ Query Time: warehouse size
○ Concurrency: Clusters
○ As you need

17. 17
● Make use of Query History
○ Duration: Wait vs Execution
○ Bytes scanned: Local vs Remote
● Leverage query_history table
○ snowflake.account_usage.query_history
Recommendations in
(cont’d)

18. 18

19. THANK YOU
19