Understanding and Optimizing the Database’s Role in Streaming Entertainment
Behind every seamless and engaging viewing experience is a great database design. But what this actually means varies wildly. Databases are used for a range of media streaming use cases – each with distinctly different workloads and requirements.
And then there are the challenges that span use cases. Pressure to keep costs and latencies down as usage rises. Dealing with unpredictable intra-day traffic fluctuations as well as major events with hundreds of millions of users. And removing dependencies on databases that were easy to adopt, but not sustainable at scale.
Join us as we explore these topics using real-world examples. You’ll learn:
- The role databases play in common media streaming use cases
- The core challenges faced in media streaming, along with different options for addressing them
- What’s changing in the media streaming landscape and how this might impact databases
2. + For data-intensive applications that require high
throughput and predictable low latencies
+ Close-to-the-metal design takes full advantage of
modern infrastructure
+ >5x higher throughput
+ >20x lower latency
+ >75% TCO savings
+ Compatible with Apache Cassandra and Amazon
DynamoDB
+ DBaaS/Cloud, Enterprise and Open Source
solutions
The Database for Gamechangers
2
“ScyllaDB stands apart...It’s the rare product
that exceeds my expectations.”
– Martin Heller, InfoWorld contributing editor and reviewer
“For 99.9% of applications, ScyllaDB delivers all the
power a customer will ever need, on workloads that other
databases can’t touch – and at a fraction of the cost of
an in-memory solution.”
– Adrian Bridgewater, Forbes senior contributor
3. 3
+400 Gamechangers Leverage ScyllaDB
Seamless experiences
across content + devices
Digital experiences at
massive scale
Corporate fleet
management
Real-time analytics 2,000,000 SKU -commerce
management
Video recommendation
management
Threat intelligence service
using JanusGraph
Real time fraud detection
across 6M transactions/day
Uber scale, mission critical
chat & messaging app
Network security threat
detection
Power ~50M X1 DVRs with
billions of reqs/day
Precision healthcare via
Edison AI
Inventory hub for retail
operations
Property listings and
updates
Unified ML feature store
across the business
Cryptocurrency exchange
app
Geography-based
recommendations
Global operations- Avon,
Body Shop + more
Predictable performance for
on sale surges
GPS-based exercise
tracking
Serving dynamic live
streams at scale
Powering India's top
social media platform
Personalized
advertising to players
Distribution of game
assets in Unreal Engine
4. Introductions
Felipe Mendes, Solution Architect at ScyllaDB
+ Years of experience with Linux and other distributed systems
+ An open source enthusiast
+ Passion towards helping businesses to achieve their most challenging
goals
Noelly Medina, Technical Engineer Team Lead at ScyllaDB
+ Years of experience in customer-facing roles
+ Customer’s satisfaction is my satisfaction
+ Always seeking for new challenges
5. Agenda
+ After all … What is Streaming?
+ A glance at Media Streaming
+ Challenges
+ How can ScyllaDB help?
+ Use Cases
+ The Future of Streaming Media
7. Seems Accurate?
streaming noun [U] (INTERNET)
the activity of listening to or watching sound or video directly from the internet
streaming adjective [ not gradable ]
playing continuously as sound or moving images are sent in a flow directly from the Internet to a computer
streaming noun [ U ] IT, INTERNET, COMMUNICATIONS
the act of sending sound or video to a computer, mobile phone, etc. directly from the internet so that it does not need to be
downloaded and saved first
stream·ing noun
a method of transmitting or receiving data (especially video and audio material) over a computer network as a steady, continuous
flow, allowing playback to start while the rest of the data is still being received.
Cambridge Dictionary
Oxford Languages
8. Seems Accurate?
streaming noun [U] (INTERNET)
the activity of listening to or watching sound or video directly from the internet
streaming adjective [ not gradable ]
playing continuously as sound or moving images are sent in a flow directly from the Internet to a computer
streaming noun [ U ] IT, INTERNET, COMMUNICATIONS
the act of sending sound or video to a computer, mobile phone, etc. directly from the internet so that it does not need to be
downloaded and saved first
stream·ing noun
a method of transmitting or receiving data (especially video and audio material) over a computer network as a steady, continuous
flow, allowing playback to start while the rest of the data is still being received.
Cambridge Dictionary
Oxford Languages
9. Tuning It Up A Little Bit
+ a method of transmitting or receiving data in motion over a computer network,
+ as a steady, continuous flow
+ with one or more data processors along the way
+ allowing for fast consumption while the rest of the data is still being received
+ backed up by a storage layer for data persistence and consistency
stream·ing noun
12. Are We There Yet?
stream·ing noun
a method of transmitting or receiving data (especially video and audio material) over a
computer network as a steady, continuous flow
14. Media Streaming Challenges
+ Low Latency
Critical to any streaming type of
use case. No circling wheels?
+ High Throughput
Support ultra-fast data ingestion
and lookups at scale
+ Geo-Replication
Enhanced user experience, and
smart replication
+ High Availability
ZERO downtime, support all peak
(SuperBowl?) traffic
+ Without Breaking the Bank
Reduce OpEx and CapEx as a
whole
15. Too Good To Be True?
Disney+ Hotstar replaced ElasticSearch and Redis with ScyllaDB
16. Too Good To Be True?
Tubi latencies went from ~300ms to ~10ms on their ML Feature Store
17. Too Good To Be True?
Comcast reduced their C* footprint from >950 nodes to 78 ScyllaDB nodes!
Cassandra
Node Count
ScyllaDB
Node Count
Recordings Ring 432 18
Reminders Ring 96 18
Recordings Secondary Ring 70 18
History Ring 96 6
Instruction and Lookup Ring 268 18
Total 962 78
Pre ScyllaDB
Node Count
962
Post ScyllaDB
Node Count
78
18. ScyllaDB Database Architecture
Horizontal & Vertical Scaling
Built in C++
(no Java overhead)
System and Data
Center Aware
Sharding Per Core Shard-Aware Drivers
Auto-Performance
Tuning
Network
Processor NUMA
Storage
Unique Close-to-Metal Architecture
19. ScyllaDB Database Architecture
Built in C++
(no Java overhead)
System and Data
Center Aware
Sharding Per Core Shard-Aware Drivers
Auto-Performance
Tuning
21. Brainstorming…
stream·ing noun
a method of transmitting or receiving data (especially video and audio material) over a
computer network as a steady, continuous flow
+ Open the app
+ Validate whether subscription is active
+ Verify device data (security, sw level, telemetry)
+ Check device location (regulations, personalization)
+ On playback: Number of concurrent streams per account
+ Endless possibilities!
22. ScyllaDB and Streaming Media Use Cases
Lookup endpoint where the actual
medium is stored for playback and
serving back to the end user
Recommendations
High throughput
volume
Extreme low latency OLTP & OLAP
Geo Replication Lower Costs
Stream Metadata Store
Provide a personalized view of
which content to consume next
based on its previous interactions
Catalog
Stores all relevant media data for
what is currently available within
the services' catalog
Playhead Tracking
Ability to know at which point in
time an user stopped consuming a
given media, so it can be
resumable later
ML Feature Store
A central store empowers data
scientists to run models under a
single place with recent data