NYC* 2013 — "Using Cassandra for DVR Scheduling at Comcast"
Upcoming SlideShare
Loading in...5
×
 

NYC* 2013 — "Using Cassandra for DVR Scheduling at Comcast"

on

  • 2,831 views

Comcast is developing a highly scalable cloud DVR scheduling system on top of Cassandra. The system is responsible for managing all DVR data and scheduling logic for devices on the X1 platform. This ...

Comcast is developing a highly scalable cloud DVR scheduling system on top of Cassandra. The system is responsible for managing all DVR data and scheduling logic for devices on the X1 platform. This talk will cover the overall architecture of the scheduling system, data model, message queue and notification software that have been developed as part of this ambitious project. We'll take a deep dive into the details of our data model and review the implementation of Comcast's open-source, Cassandra-based clones of Amazon SQS and SNS.

Statistics

Views

Total Views
2,831
Views on SlideShare
2,752
Embed Views
79

Actions

Likes
2
Downloads
33
Comments
0

2 Embeds 79

http://tspace.web.att.com 78
http://localhost 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Split the backplane2x4 planesInternal controller drives HDD @ Raid 1LSI controller drives SSD @ Raid 0
  • Pending writes prevent ability to delete rowNormal compaction will not clean up rowMajor compaction can not clean up if there a pending writes to the row

NYC* 2013 — "Using Cassandra for DVR Scheduling at Comcast" NYC* 2013 — "Using Cassandra for DVR Scheduling at Comcast" Presentation Transcript

  • Using Cassandra for DVR Scheduling at ComcastVERSION 1.0DATE 3/20/213AUTHOR John McCann | @dangermccann Using Cassandra for DVR Scheduling at Comcast 4/9/2013 1
  • The X1 Platform XFINITY On Demand Choose from thousands of movies and shows. More than a back button See the last nine shows and channels youve watched with a single tap. Using Cassandra for DVR Scheduling at Comcast 4/9/2013 2
  • The X1 Platform The ultimate scoreboard Track multiple games at once and check the latest scores, standings, and schedules. Apps right on your TV Now your TV is for more than just watching. Using Cassandra for DVR Scheduling at Comcast 4/9/2013 3
  • The X1 Platform• Next generation of the XFINITY TV experience• New line of set-top boxes and technologies that lets XFINITY TV run in the cloud• Premium content viewing experience, with access to XFINITY On Demand, DVR and Pay-Per-View video• De-coupled, service-based architecture where services are developed and deployed independently Using Cassandra for DVR Scheduling at Comcast 4/9/2013 4
  • What Comcast Stores in Cassandra• Customer viewing history• Resume points for DVR and XFINITY On Demand content• Device and account preferences• Pay-Per-View purchases• DVR recordings and scheduling instructions Using Cassandra for DVR Scheduling at Comcast 4/9/2013 5
  • Why Comcast Uses Cassandra• Multi-data center (active / active)• Performance of write-heavy use cases – DVR status updates – Remembering resume points during content playback• Developers (not DBAs) model and manage the data• Open source encourages in-house expertise Using Cassandra for DVR Scheduling at Comcast 4/9/2013 6
  • Redesigned DVR Using Cassandra for DVR Scheduling at Comcast 4/9/2013 7
  • DVR System ArchitectureEast Data Center West Data Center API Server Worker API Server Worker Worker Queue Worker Queue API Server Worker API Server Worker API Server Worker API Server Worker Cassandra Cassandra Using Cassandra for DVR Scheduling at Comcast 4/9/2013 8
  • Cassandra Configuration• 4-node rings in 2 data centers• NetworkTopologyStrategy replication strategy• Replication factor is 3 (per data center)• LOCAL_QUORUM consistency level for most operations Using Cassandra for DVR Scheduling at Comcast 4/9/2013 9
  • Comcast Message Bushttps://github.com/Comcast/cmb CQS Endpoint• Horizontally scalable queuing and notification service• Compatible with Amazon SQS and SNS Cassandra Redis• Cassandra used to persist messages• Redis used as caching layer• Open source (Apache license) CNS Endpoint CNS Worker Using Cassandra for DVR Scheduling at Comcast 4/9/2013 10
  • Use Case ExplorationScheduling a Series Recording Using Cassandra for DVR Scheduling at Comcast 4/9/2013 11
  • User Interaction Set-top boxUser UI Server Using Cassandra for DVR Scheduling at Comcast 4/9/2013 12
  • User Interaction Set-top box API Server Worker Worker QueueUser API Server Worker UI Server API Server Worker Cassandra Memcached Using Cassandra for DVR Scheduling at Comcast 4/9/2013 13
  • Recording Schedule Update API Server Worker Worker Queue API Server Worker API Server Worker Cassandra Using Cassandra for DVR Scheduling at Comcast 4/9/2013 14
  • Recording Schedule Update Set-top box API Server Worker Worker Queue API Server Worker API Server Worker Cassandra Using Cassandra for DVR Scheduling at Comcast 4/9/2013 15
  • Recording Schedule Update Set-top box API Server Worker Worker Queue API Server Worker API Server Worker Cassandra Using Cassandra for DVR Scheduling at Comcast 4/9/2013 16
  • UI Update API Server Worker Worker Queue API Server Worker UI Server API Server Worker Cassandra Using Cassandra for DVR Scheduling at Comcast 4/9/2013 17
  • UI Update API Server Worker Worker Queue API Server Worker UI Server API Server Worker Cassandra Memcached Using Cassandra for DVR Scheduling at Comcast 4/9/2013 18
  • System Diagram Set-top box API Server Worker Worker QueueUser API Server Worker UI Server API Server Worker Cassandra Memcached Using Cassandra for DVR Scheduling at Comcast 4/9/2013 19
  • Data Model Analysis1. Recording Instructions2. Recording Schedule3. Completed Recordings Using Cassandra for DVR Scheduling at Comcast 4/9/2013 20
  • Recording Instructions Schema• Stores the instructions used to produce the recording schedule in the order that the instructions were generated.ScheduleInstruction AccountID/DeviceID TimeUUID : [instruction] TimeUUID : [instruction] … Composite row key Instructions sorted by creation time using TimeUUID as the column name. The column value contains the instruction data. Using Cassandra for DVR Scheduling at Comcast 4/9/2013 21
  • Recording Schedule Schema• Stores the data for the most recent recording schedule that has been delivered to the Recorder on the STB.DvrSchedule AccountID/DeviceID Properties of each recording stored as separate columns using composite column names: recording_ID/[property_name] Using Cassandra for DVR Scheduling at Comcast 4/9/2013 22
  • Completed Recordings Schema• Stores the data for all of the completed recordings on the STB.Recording AccountID/DeviceID Properties of each recording stored as separate columns using composite column names: recording_ID/[property_name] Using Cassandra for DVR Scheduling at Comcast 4/9/2013 23
  • Hardware Configuration• HP DL360 G8• 64GB RAM• 2 x 600 GB SATA Hard Drive• RAID 1 for OS Partition• LSI controller• 2 x Samsung 840 512GB SSD (room for up to 4 SSDs) Using Cassandra for DVR Scheduling at Comcast 4/9/2013 24
  • Solid State DrivesSamsung 840 512GB• 50% cost increase per node• 400% capacity increase per node• Smaller rings, better read performance• Read performance consistency (flat 99th percentiles for reads) Using Cassandra for DVR Scheduling at Comcast 4/9/2013 25
  • Lessons Learned• If youre using Hector, pay close attention to CassandraHostConfigurator.maxActive.• Don’t enable the row cache if you need to perform slice queries.• Don’t delete a row if you plan to write columns to the same row later (better in 1.2).• Don’t run Cassandra on shared storage if you can avoid it. Using Cassandra for DVR Scheduling at Comcast 4/9/2013 26
  • Q&AThanks for your attention!http://xfinity.comcast.net/x1John McCann@dangermccann Using Cassandra for DVR Scheduling at Comcast 4/9/2013 27