Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How Sky Uses Cassandra Video Platform


Published on


Published in: Technology
  • Be the first to comment

  • Be the first to like this

How Sky Uses Cassandra Video Platform

  1. 1. How Sky Uses Cassandra Video Platform Dr. Strange
  2. 2. A Little About Me Richard Ellingham • Part of the Site Reliability Engineering team at Sky UK within the OTT SWE Department - OTT refers to content from a third party that is delivered to an end- user, with the ISP simply transporting IP packets • I’ve been involved with Cassandra at Sky for c. 3 years • Run a small number of core teams that form a Centre of Excellence • Specialise in areas of Performance Testing, Reliability and Persistence Engineering – specifically Cassandra
  3. 3. So What? Online Video Platform (OVP) Talk: What critical services does Cassandra underpin that allow customers to stream content to their devices? Primarily OVP provides playout services for the OTT Client Devices, this includes • Authorisation of content entitlements for a given user/device) - UMV • Policing Device and Stream Concurrency – DCM • Enforcing Stream Start Parental PIN checks – DCM • Providing URLs for Streaming and Download – TM, LCM, DM • Granting and Issuing DRM Tokens - DRM • Managing Content Bookmarks - BM
  4. 4. A Little About Cassandra Estate Estate • Circa 100 production nodes – total about 500 nodes across all environments • Run a combination of Apache Cassandra and Datastax Cassandra • Circa 10 clusters - Combination of multi-tenant and dedicated • Largest cluster is c. 35 nodes holding 4TB, smallest 6 nodes • Active/Active across multiple DCs • We can support up to 3M concurrent users streaming content
  5. 5. Cassandra Cluster Hemel A Little About Cassandra Estate Rack1 Rack2 Rack3 Node3 Node2 Node1 Node3 Node2 Node1 Node3 Node2 Node1 Slough Rack1 Rack2 Rack3 Node3 Node2 Node1 Node3 Node2 Node1 Node3 Node2 Node1 • Majority of estate is DSE Cassandra – 4.8.x • We only run Cassandra workloads for DSE • CentOS 7.2 (12 vCPUs) • Java 1.8 - G1GC (32GB Heaps) • 500GB data mount • Minimum if 2 DCs with 3 Racks per DC • Minimum of 2 nodes per Rack • Clients read/write local_quorum • Use a Downgrade Policy • 1 Seed defined per Rack
  6. 6. In the Beginning - MPOD Challenges - Monolithic System • Release cycle was slow – c. 6 weeks • Hard to scale • Persistence layer scaling was limited – only so far you can scale up • Data layer was running an Oracle RAC cluster cross DC using Active Dataguard • Data footprint was tricky to manage. • Scheduling changes was tricky • Availability – We weren’t getting the uptimes we wanted. • Hard to make truely active/active - Bi Directional Goldengate (nightmare!)
  7. 7. Some challenges we needed to address Availability & Reliability • Multi-DC – Truly Active/Active • 100% service uptime • Distributed Architecture Scalability • Use micro-service architecture - Decoupled services from monolithic architecture • Scale out architecture Build New Capabilities • Needed quick release cycles to introduce new features and ways of consuming content International Presence • Need to be able to offer propositions quickly to other territories
  8. 8. Some challenges we needed to address Typical Load Profile • High profile events see c. 700k concurrent users (eg GoT, Football) • Sharp ramp up • Sharp rises just before start of programme • Sharp rises just before end of programme
  9. 9. OVP – Online Video Platform / Streaming Platform Service Video Platform (SPS) Client UMV API TM LCM Identity AC TM API LCM API DCM API DRM API DM API DCM UMV DM Content Discovery Gonzales Commerce MCS WIG Headend DRM License Issuers Playready Cisco VGC Fairplay Marlin Oogway CRM API CBS IDS CRM IDS 1 2 3 54 6 8 CDN Akamai Level3 9 1 0 Content Discover/ Browse Returns: OAUTH Token Input: Credentials Input: OAUTH Token Returns: User Token Heartbeat Input: DRM Token Returns: DRM License Stream Content Retrieve Entlmnts Confirm: OAUTH Token Cache EntlmntsRetrieve Metadata Save Events Browse Events Royalties Fraud Call Centre Persist Metadata Return: DRM Token Concrncy Check 1 2 1 3 1 4 7 1 1 1 5 1 6 1 7