Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Alex Smith, ASEAN Media Solutions Architect, AWS...
The Problem
Your Consumers Have Changed
More content choices
More devices
More delivery methods
New experiences
Your Business Has Changed
More data
Faster go to market
Leaner teams
Innovation and failure
What to Expect from the Session
Content is king
Building from scratch
Battle-hardened lessons – Verizon onCue
Clarifications
OTT, OVP, AVOD, TVOD, SVOD
It doesn’t matter
Microservices
Buzzword?
Specific scope, interoperable services...
Over the Top Platform
Content
Production
Content
Storage
Processing &
Management
Content
Production
Content
Storage
Processing &
Management
Content
Production
Content
Distribution
Content
Storage
Processing &
Management
Content
Production
Content
Distribution
Content
Storage
Content
Consumption
Processing &
Management
Shared IT Services
NetworkInfrastructure OperationsSecurity
Content
Production
Content
Distributio...
Processing &
Management
Content
Production
Content
Distribution
Content
Storage
Content
Consumption
Your Existing Physical Requirements
HD-SDI
Satellite transmission
Local SAN
HW transcoder
Your OTT Approach
FTP / Accelerator
Large NAS
DAM / Workflow System
Storage and Ingest — Serverless
AWS Import/ Export
S3 Ingest
Content
provider
Storage and Ingest — Serverless
AWS Import/ Export
S3 Ingest
Lifecycle
policies
S3 bucket
Amazon Glacier
Content
provider
...
Storage and Ingest — Serverless
AWS Import/ Export
S3 Ingest
S3
notification
AWS Lambda
Lifecycle
policies
S3 bucket
Amazo...
Storage and Ingest — Serverless
AWS Import/ Export
S3 Ingest
S3
notification
AWS Lambda
Metadata
extract & persist
Amazon
...
Storage and Ingest — Serverless
AWS Import/ Export
S3 Ingest
For source, renditions, metadata sidecar files
S3
notificatio...
Storage and Ingest — Serverless
AWS Import/ Export
S3 Ingest
For source, renditions, metadata sidecar files
API Gateway
Cl...
Processing &
Management
Content
Production
Content
Distribution
Content
Storage
Content
Consumption
Encoding, Packaging, Encrypting
Challenges
Multiple client devices
Higher quality content
Parallel, complex workflow
Uneve...
Content Processing Pipeline (Using Lambda)
AWS Import/ Export
S3 Ingest
Ingest
Content Processing Pipeline (Using Lambda)
AWS Import/ Export
S3 Ingest
S3 notification
Ingest
Content Processing Pipeline (Using Lambda)
AWS Import/ Export
S3 Ingest
S3 as backend storage for content files accessible...
Content Processing Pipeline (Using Lambda)
AWS Import/ Export
S3 Ingest
S3 as backend storage for content files accessible...
Content Processing Pipeline (Using Lambda)
AWS Import/ Export
S3 Ingest
S3 as backend storage for content files accessible...
Content Processing Pipeline (Using Lambda)
AWS Import/ Export
S3 Ingest
S3 as backend storage for content files accessible...
What About Packaging?
All moving towards HTTP –
RTMP is hard to scale
Battle of the standards –
HLS, HDS, SmoothStream,
MP...
And Encryption / DRM?
Still largely driven by studio
requirements
Just-in-time encryption (hard)
Reusability across packag...
Pitfalls of Content Prep
Betting big on closed
standards
Technologies in vogue
Adoption is device-driven
Avoiding Pitfalls of Content Prep
Keep your mezzanine / masters (S3 IA, Glacier)
Mix and match your encoding
Contradictory...
Top Tip
CPU-based encoding
c4.8xlarge / m4.10xlarge
P-State, C-State configuration
Varies with particular encoding
librari...
Processing &
Management
Content
Production
Content
Distribution
Content
Storage
Content
Consumption
Live
RTMP ingest to origin
Repackaged and encrypted on
the fly
DRM less common
WebRTC for 1-to-few, then
RTMP/HTTP when tr...
HTTP Live Streaming (HDS & HLS)
HLS was pretty close to de facto
Space becoming disrupted again
Network considerations in ...
HLS and HDS
Uses “parent / child / chunk”
model
HLS: Playlist contains
chunklists, which contain
chunks
HDS: Manifest cont...
HTTP Dynamic Streaming — HDS
> GET /live/channel1.abst HTTP/1.1
< Seg1-Frag55
< Seg1-Frag56
[…]
> GET /live/channel1.abst ...
HTTP Dynamic Streaming — HDS
Use fragment name alignment in the event of a republish
Ensure that fragments are aligned acr...
Top Tip
HDS bootstraps are binary
To view, you must decode
them first
https://bitly.com/abstdecoder
(Thanks SwiftServe!)
HTTP Live Streaming — HLS
> GET /live/channel1.m3u8 HTTP/1.1
< HTTP/1.1 200 OK
< Date: Tue, 15 Sep 2015 13:37:56 GMT
< Ser...
HTTP Live Streaming — HLS
> GET /live/channel1.m3u8 HTTP/1.1
< HTTP/1.1 200 OK
< Date: Tue, 15 Sep 2015 13:37:56 GMT
< Ser...
HTTP Live Streaming
Add a no, or short age, cache header
Use segment name randomisation
Use Player metrics to detect probl...
Multi-CDN
CDN Selector
Build vs. buy
Excellent offerings on the
market
Metadata often tightly coupled
with platform, so tricky to us...
CDN Selection Methods
DNS-based
Geo / latency / intelligent routing at DNS level
Asset sharding
50% of assets on CDN A, 50...
CDN-aware Asset Info Service
AWS Elastic
Beanstalk
CDN
Authentication
Authorisation
Asset info
request
CDN-aware Asset Info Service
> GET /asset/31337 HTTP/1.1
< assetUrl: "http://cdn-a.alexjs.im/vod/31337.m3u8"
< adProvider:...
CDN-aware Asset Info Service
S3 Ingest
Ingest
Amazon
DynamoDB
Media processing
Amazon Elastic
Transcoder
Processing
CDN-aware Asset Info Service
S3 Ingest
Ingest
AWS Elastic
Beanstalk
Amazon
DynamoDB
Media processing
Amazon Elastic
Transc...
CDN-aware Asset Info Service
S3 IngestAuthoritative Source – for multi-CDN
Ingest
AWS Elastic
Beanstalk
CDN
Amazon
DynamoD...
S3 IngestAuthoritative Source – for multi-CDN
Ingest
API
Gateway
CDN
Amazon
DynamoDB
Media processing
Amazon Elastic
Trans...
Processing &
Management
Content
Production
Content
Distribution
Content
Storage
Content
Consumption
Concurrency Management (and
More)
Concurrency
Problem
Studio mandate on stream
concurrency
Cross-account sharing
Approach
Player heartbeat
Subscription-base...
Subscriber Concurrency Management
Heartbeat sent from every
player at regular intervals
Supports both native and
web-based...
Subscriber Concurrency Management
{
assetId: "d6f9fe" // Programme ID
userId: "33114220875dc" // Token / User
ID timeStamp...
Subscriber Concurrency Management
Heartbeat received by Amazon
Kinesis
Data fed into DynamoDB
DRM licence server reads
Dyn...
Subscriber Concurrency Management
Amazon
Kinesis
Lambda
Track drop-off
Highlight popular content
Feed back into content
de...
Subscriber Concurrency Management
Amazon
Kinesis
Lambda
Syndicate data to ads server
Recommendations-based
demographic
Dyn...
Subscriber Concurrency Management
Amazon
Kinesis
Lambda
Cross-device
Play / pause / resume
No servers needed
Client-side “...
Go90 — Verizon
Go90 go90.com, #go90
Processing &
Management
Content
Production
Content
Distribution
Content
Storage
Content
Consumption
Go90 — from 30,000 ft
100+ micro services/systems
10+ deployments per week
Multiple clients, large target user base
High r...
APIs — Edge Services
Typical Service Stack
Storage
Logging Service Client
Monitoring Configuration
Domain Logic
NGINX
https://github.com/oncue
Governor
Accounts
Log Upload
Search
Profile Image
Discovery
Event Q
DRM
Metadata
Processing
Metadata
Ingest
Content
Ops
An...
STB Maestro
Content
Ingest
CDN
Governor
Accounts
Log Upload
Search
Profile Image
Discovery
Event Q
DRM
Analytic
s
Favorites
Cert
Prov
AWS
HSM
Ads
Offers and
Packa...
Accounts
Log Upload
Search
Profile Image
Discovery
Event Q
DRM
Favorites
Cert
Prov
AWS
HSM
Ads
Offers and
Packages
DAI
bac...
Governor
Accounts
Log Upload
Search
Profile Image
Discovery
Event Q
DRM
Metadata
Processing
Metadata
Ingest
Content
Ops
An...
Metadata — Ingest and Storage
Courtesy: IMDB
Nile
Linear
(Airings)
VODLive
Nile
MMS
(content ops)
Linear Ingest
Provisioning
Workflow
Content
Types
Input
Process
Nile
...
Linear
(Airings)
VODLive
Content
Types
Package Info
Amazon EC2
Linear
(Airings)
VODLive
MMS
(content ops)
Linear Ingest
Provisioning
Workflow
Content
Types
Input
Process
Package Info
Am...
Linear
(Airings)
VODLive
Nile
MMS
(content ops)
Linear Ingest
Provisioning
Workflow
Content
Types
Input
Process
Nile
Workf...
Nile
Linear
(Airings)
VODLive
Nile
MMS
(content ops)
Linear Ingest
Provisioning
Workflow
Content
Types
Input
Process
Nile
...
Nile
Linear
(Airings)
VODLive
Nile
MMS
(content ops)
Linear Ingest
Provisioning
Workflow
Content
Types
Input
Process
Nile
...
Nile
Linear
(Airings)
VODLive
Nile
MMS
(content ops)
Linear Ingest
Provisioning
Workflow
Content
Types
Input
Process
Nile
...
Product Intelligence
Data
Collection
Camus
(Consume
events)
Amazon S3
Real-time
Processing
Amazon
S3
Amazon Redshift
Data
Warehouse
Batch
Proce...
Best Practices
Avoid a priori optimization
Deploy often
Keep simple and separate systems
Don’t always give in to “right to...
Summary
Summary
• Avoid a priori optimization
• Deploy often
• Keep simple and separate
systems
• Don’t always give in to “right
t...
Summary
• Serverless by default
• Avoid lock-in
• Be cautious of the outside
• Reuse data for good
• Avoid a priori optimi...
Summary
• Serverless by default
• Avoid lock-in
• Be cautious of the outside
• Reuse data for good
• Avoid a priori optimi...
Thank you!
Remember to complete
your evaluations!
Upcoming SlideShare
Loading in …5
×

(ARC303) Pure Play Video OTT: A Microservices Architecture

4,182 views

Published on

"An end-to-end, over-the-top (OTT) video system is built of many interdependent architectural tiers, ranging from content preparation, content delivery, and subscriber and entitlement management, to analytics and recommendations. This talk will provide a detailed exploration of how to architect a media platform that allows for growth, scalability, security, and business changes at each tier, based on real-world experiences delivering over 100 Gbps of concurrent video traffic with 24/7/365 linear TV requirements. Finally, learn how Verizon uses AWS, including Amazon Redshift and Amazon Elastic MapReduce, to power its recently launched mobile video application Go90.

Using a mixture of AWS services and native applications, we address the following scaling challenges:


Content ingest, preparation, and distribution
Operation of a 24x7x365 Linear OTT Playout Platform
Common pitfalls with transcode and content preperation
Multi-DRM and packaging to allow cross platform playback
Efficient delivery and multi-CDN methodology to allow for a perfect experience globally
Kinesis as a dual purpose system for both analytics and concurrency access management
Integration with Machine Learning for an adaptive recommendation system, with real time integration between content history and advertising data
User, entitlement, and content management
General best practices for ‘Cloud Architectures’ and their integration with Amazon Web Services; Infrastructure as Code, Disposable and immutable infrastructure, code deployment & release management, DevOps and Microservices Architectures


This session is great for architects, engineers, and CTOs within media and entertainment or others simply interested in decoupled architectures."

Published in: Technology

(ARC303) Pure Play Video OTT: A Microservices Architecture

  1. 1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Alex Smith, ASEAN Media Solutions Architect, AWS Vidhya Narayanan, Director, Verizon onCue October 2015 Pure Play Video, Over-The-Top A Microservices Architecture in the Cloud ARC303
  2. 2. The Problem
  3. 3. Your Consumers Have Changed More content choices More devices More delivery methods New experiences
  4. 4. Your Business Has Changed More data Faster go to market Leaner teams Innovation and failure
  5. 5. What to Expect from the Session Content is king Building from scratch Battle-hardened lessons – Verizon onCue
  6. 6. Clarifications OTT, OVP, AVOD, TVOD, SVOD It doesn’t matter Microservices Buzzword? Specific scope, interoperable services Allow rapid innovation
  7. 7. Over the Top Platform
  8. 8. Content Production Content Storage
  9. 9. Processing & Management Content Production Content Storage
  10. 10. Processing & Management Content Production Content Distribution Content Storage
  11. 11. Processing & Management Content Production Content Distribution Content Storage Content Consumption
  12. 12. Processing & Management Shared IT Services NetworkInfrastructure OperationsSecurity Content Production Content Distribution Content Storage Content Consumption
  13. 13. Processing & Management Content Production Content Distribution Content Storage Content Consumption
  14. 14. Your Existing Physical Requirements HD-SDI Satellite transmission Local SAN HW transcoder
  15. 15. Your OTT Approach FTP / Accelerator Large NAS DAM / Workflow System
  16. 16. Storage and Ingest — Serverless AWS Import/ Export S3 Ingest Content provider
  17. 17. Storage and Ingest — Serverless AWS Import/ Export S3 Ingest Lifecycle policies S3 bucket Amazon Glacier Content provider Lifecycle policies S3 IA
  18. 18. Storage and Ingest — Serverless AWS Import/ Export S3 Ingest S3 notification AWS Lambda Lifecycle policies S3 bucket Amazon Glacier Content provider Lifecycle policies S3 IA
  19. 19. Storage and Ingest — Serverless AWS Import/ Export S3 Ingest S3 notification AWS Lambda Metadata extract & persist Amazon DynamoDB Amazon CloudSearch S3 bucket Amazon Glacier Content provider Lifecycle policies Lifecycle policies S3 IA
  20. 20. Storage and Ingest — Serverless AWS Import/ Export S3 Ingest For source, renditions, metadata sidecar files S3 notification Amazon Elastic Transcoder proxy / thumbnail generation AWS Lambda Metadata extract & persist Amazon DynamoDB Amazon CloudSearch S3 bucket Amazon Glacier Content provider Lifecycle policies Lifecycle policies S3 IA
  21. 21. Storage and Ingest — Serverless AWS Import/ Export S3 Ingest For source, renditions, metadata sidecar files API Gateway CloudFront distribution S3 notification Amazon Elastic Transcoder proxy / thumbnail generation AWS Lambda Metadata extract & persist Amazon DynamoDB Amazon CloudSearch AWS Lambda S3 bucket Amazon Glacier Content provider Lifecycle policies Lifecycle policies S3 IA
  22. 22. Processing & Management Content Production Content Distribution Content Storage Content Consumption
  23. 23. Encoding, Packaging, Encrypting Challenges Multiple client devices Higher quality content Parallel, complex workflow Uneven load distribution
  24. 24. Content Processing Pipeline (Using Lambda) AWS Import/ Export S3 Ingest Ingest
  25. 25. Content Processing Pipeline (Using Lambda) AWS Import/ Export S3 Ingest S3 notification Ingest
  26. 26. Content Processing Pipeline (Using Lambda) AWS Import/ Export S3 Ingest S3 as backend storage for content files accessible to other processing tasks Amazon Elastic Transcoder S3 notification Trigger a Lambda function to start a transcoding job Ingest
  27. 27. Content Processing Pipeline (Using Lambda) AWS Import/ Export S3 Ingest S3 as backend storage for content files accessible to other processing tasks Amazon Elastic Transcoder S3 notification Trigger a Lambda function to start a transcoding job Ingest S3 notification
  28. 28. Content Processing Pipeline (Using Lambda) AWS Import/ Export S3 Ingest S3 as backend storage for content files accessible to other processing tasks Amazon Elastic Transcoder S3 notification Trigger a Lambda function to start a transcoding job Ingest S3 notification Lambda function to extract metadata Update CMS or metadata
  29. 29. Content Processing Pipeline (Using Lambda) AWS Import/ Export S3 Ingest S3 as backend storage for content files accessible to other processing tasks Amazon Elastic Transcoder S3 notification Trigger a Lambda function to restart a transcoding job Ingest S3 notification Lambda function to extract metadata Update CMS or metadata API Gateway
  30. 30. What About Packaging? All moving towards HTTP – RTMP is hard to scale Battle of the standards – HLS, HDS, SmoothStream, MPEG-DASH Alliance for Open Media
  31. 31. And Encryption / DRM? Still largely driven by studio requirements Just-in-time encryption (hard) Reusability across packaging methods (PlayReady across HLS & SmoothStream)
  32. 32. Pitfalls of Content Prep Betting big on closed standards Technologies in vogue Adoption is device-driven
  33. 33. Avoiding Pitfalls of Content Prep Keep your mezzanine / masters (S3 IA, Glacier) Mix and match your encoding Contradictory — JIT packaging and heavy caching Remember 80/20 (95/5) rule of content
  34. 34. Top Tip CPU-based encoding c4.8xlarge / m4.10xlarge P-State, C-State configuration Varies with particular encoding libraries – experiment and baseline (Thanks PocketMath!)
  35. 35. Processing & Management Content Production Content Distribution Content Storage Content Consumption
  36. 36. Live RTMP ingest to origin Repackaged and encrypted on the fly DRM less common WebRTC for 1-to-few, then RTMP/HTTP when traffic increases Content Delivery Architectures VOD / Catchup File-based – stored on S3/CDN Usually prepared (encrypted/packaged) before final storage DRM by default ISPs worried about unicast delivery
  37. 37. HTTP Live Streaming (HDS & HLS) HLS was pretty close to de facto Space becoming disrupted again Network considerations in the real world
  38. 38. HLS and HDS Uses “parent / child / chunk” model HLS: Playlist contains chunklists, which contain chunks HDS: Manifest contains bootstrap files, which contain fragments
  39. 39. HTTP Dynamic Streaming — HDS > GET /live/channel1.abst HTTP/1.1 < Seg1-Frag55 < Seg1-Frag56 […] > GET /live/channel1.abst HTTP/1.1 < Seg1-Frag1 < Seg1-Frag2
  40. 40. HTTP Dynamic Streaming — HDS Use fragment name alignment in the event of a republish Ensure that fragments are aligned across bitrates Players will error otherwise
  41. 41. Top Tip HDS bootstraps are binary To view, you must decode them first https://bitly.com/abstdecoder (Thanks SwiftServe!)
  42. 42. HTTP Live Streaming — HLS > GET /live/channel1.m3u8 HTTP/1.1 < HTTP/1.1 200 OK < Date: Tue, 15 Sep 2015 13:37:56 GMT < Server: Apache < Last-Modified: Mon, 03 Aug 2015 3:14:15 GMT < Accept-Ranges: bytes < Content-Length: 219
  43. 43. HTTP Live Streaming — HLS > GET /live/channel1.m3u8 HTTP/1.1 < HTTP/1.1 200 OK < Date: Tue, 15 Sep 2015 13:37:56 GMT < Server: Apache < Last-Modified: Mon, 03 Aug 2015 3:14:15 GMT < Accept-Ranges: bytes < Content-Length: 219 < Cache-Control: max-age=5
  44. 44. HTTP Live Streaming Add a no, or short age, cache header Use segment name randomisation Use Player metrics to detect problem ISPs / CDNs
  45. 45. Multi-CDN
  46. 46. CDN Selector Build vs. buy Excellent offerings on the market Metadata often tightly coupled with platform, so tricky to use a third party
  47. 47. CDN Selection Methods DNS-based Geo / latency / intelligent routing at DNS level Asset sharding 50% of assets on CDN A, 50% on CDN B CDN-aware asset info service
  48. 48. CDN-aware Asset Info Service AWS Elastic Beanstalk CDN Authentication Authorisation Asset info request
  49. 49. CDN-aware Asset Info Service > GET /asset/31337 HTTP/1.1 < assetUrl: "http://cdn-a.alexjs.im/vod/31337.m3u8" < adProvider: "alexjsAds" < countryCode: "[im, sg, id]"
  50. 50. CDN-aware Asset Info Service S3 Ingest Ingest Amazon DynamoDB Media processing Amazon Elastic Transcoder Processing
  51. 51. CDN-aware Asset Info Service S3 Ingest Ingest AWS Elastic Beanstalk Amazon DynamoDB Media processing Amazon Elastic Transcoder Processing Authentication Authorisation Asset info request
  52. 52. CDN-aware Asset Info Service S3 IngestAuthoritative Source – for multi-CDN Ingest AWS Elastic Beanstalk CDN Amazon DynamoDB Media processing Amazon Elastic Transcoder Processing Authentication Authorisation Asset info request
  53. 53. S3 IngestAuthoritative Source – for multi-CDN Ingest API Gateway CDN Amazon DynamoDB Media processing Amazon Elastic Transcoder Processing Authentication Authorisation CDN-aware Asset Info Service Perfect for microservices – and for serverless computing Lambda
  54. 54. Processing & Management Content Production Content Distribution Content Storage Content Consumption
  55. 55. Concurrency Management (and More)
  56. 56. Concurrency Problem Studio mandate on stream concurrency Cross-account sharing Approach Player heartbeat Subscription-based thresholds
  57. 57. Subscriber Concurrency Management Heartbeat sent from every player at regular intervals Supports both native and web-based players Amazon Kinesis Lambda
  58. 58. Subscriber Concurrency Management { assetId: "d6f9fe" // Programme ID userId: "33114220875dc" // Token / User ID timeStamp: "T00:00:05" // Progress deviceId: "93d2d4fef95cb" // Fingerprint deviceType: "Amazon Fire" // From API } (Pseudo JSON) Amazon Kinesis Lambda
  59. 59. Subscriber Concurrency Management Heartbeat received by Amazon Kinesis Data fed into DynamoDB DRM licence server reads DynamoDB table Decision made Amazon Kinesis DynamoDB Lambda EC2
  60. 60. Subscriber Concurrency Management Amazon Kinesis Lambda Track drop-off Highlight popular content Feed back into content developmentDynamoDB Amazon Redshift
  61. 61. Subscriber Concurrency Management Amazon Kinesis Lambda Syndicate data to ads server Recommendations-based demographic DynamoDB Amazon Redshift Amazon MLAd server
  62. 62. Subscriber Concurrency Management Amazon Kinesis Lambda Cross-device Play / pause / resume No servers needed Client-side “stop” can be enabled, too (non-DRM based) DynamoDB Amazon Redshift Lambda API Gateway
  63. 63. Go90 — Verizon
  64. 64. Go90 go90.com, #go90
  65. 65. Processing & Management Content Production Content Distribution Content Storage Content Consumption
  66. 66. Go90 — from 30,000 ft 100+ micro services/systems 10+ deployments per week Multiple clients, large target user base High reliability/availability, low latency, superior user experience
  67. 67. APIs — Edge Services
  68. 68. Typical Service Stack Storage Logging Service Client Monitoring Configuration Domain Logic NGINX https://github.com/oncue
  69. 69. Governor Accounts Log Upload Search Profile Image Discovery Event Q DRM Metadata Processing Metadata Ingest Content Ops Analytics, Ad, Curate 3rd Party Analytic s Favorites Cert Prov AWS HSM Ads Offers and Packages DAI backend STB Maestro Content Ingest CDN Reco Discovery Accounts Favorites Search DRM Profile Rights SU and…a 100 others Profiles Profiles Q
  70. 70. STB Maestro Content Ingest CDN Governor
  71. 71. Accounts Log Upload Search Profile Image Discovery Event Q DRM Analytic s Favorites Cert Prov AWS HSM Ads Offers and Packages DAI backend STB Maestro Content Ingest CDN Reco and…a 100 others Profiles Profiles Q Governor
  72. 72. Accounts Log Upload Search Profile Image Discovery Event Q DRM Favorites Cert Prov AWS HSM Ads Offers and Packages DAI backend STB Maestro Content Ingest CDN Reco Discovery Accounts Favorites Search DRM Profile Rights SU and…a 100 others Profiles Profiles Q Governor Analytic s
  73. 73. Governor Accounts Log Upload Search Profile Image Discovery Event Q DRM Metadata Processing Metadata Ingest Content Ops Analytics, Ad, Curate 3rd Party Analytics Favorites Cert Prov AWS HSM Ads Offers and Packages DAI backend STB Maestro Content Ingest CDN Reco Discovery Accounts Favorites Search DRM Profile Rights SU and…a 100 others Profiles Profiles Q
  74. 74. Metadata — Ingest and Storage
  75. 75. Courtesy: IMDB
  76. 76. Nile Linear (Airings) VODLive Nile MMS (content ops) Linear Ingest Provisioning Workflow Content Types Input Process Nile Workflow Package Info Raw Data Normalized Data Blended Data Search River DB Profile Service Search River (Publish Logic) Amazon EC2 Image Processing Amazon EC2Auto Scaling Amazon S3 CDN Catalog Index Analytics River Ads River 3rd Party
  77. 77. Linear (Airings) VODLive Content Types Package Info Amazon EC2
  78. 78. Linear (Airings) VODLive MMS (content ops) Linear Ingest Provisioning Workflow Content Types Input Process Package Info Amazon EC2
  79. 79. Linear (Airings) VODLive Nile MMS (content ops) Linear Ingest Provisioning Workflow Content Types Input Process Nile Workflow Package Info Raw Data Normalized Data Blended Data Search River DB Amazon EC2
  80. 80. Nile Linear (Airings) VODLive Nile MMS (content ops) Linear Ingest Provisioning Workflow Content Types Input Process Nile Workflow Package Info Raw Data Normalized Data Blended Data Search River DB Profile Service Search River (Publish Logic) Amazon EC2 Catalog Index Amazon EC2
  81. 81. Nile Linear (Airings) VODLive Nile MMS (content ops) Linear Ingest Provisioning Workflow Content Types Input Process Nile Workflow Package Info Raw Data Normalized Data Blended Data Search River DB Profile Service Search River (Publish Logic) Amazon EC2 Catalog Index Analytics River Ads River 3rd Party
  82. 82. Nile Linear (Airings) VODLive Nile MMS (content ops) Linear Ingest Provisioning Workflow Content Types Input Process Nile Workflow Package Info Raw Data Normalized Data Blended Data Search River DB Profile Service Search River (Publish Logic) Amazon EC2 Image Processing Amazon EC2Auto Scaling Amazon S3 CDN Catalo g Index Analytics River Ads River 3rd Party
  83. 83. Product Intelligence
  84. 84. Data Collection Camus (Consume events) Amazon S3 Real-time Processing Amazon S3 Amazon Redshift Data Warehouse Batch Processing Amazon SWF Amazon EMR Amazon S3
  85. 85. Best Practices Avoid a priori optimization Deploy often Keep simple and separate systems Don’t always give in to “right tool for the right job”
  86. 86. Summary
  87. 87. Summary • Avoid a priori optimization • Deploy often • Keep simple and separate systems • Don’t always give in to “right tool for the right job” Processing & Management Content Production Content Distribution Content Storage Content Consumption
  88. 88. Summary • Serverless by default • Avoid lock-in • Be cautious of the outside • Reuse data for good • Avoid a priori optimization • Deploy often • Keep simple and separate systems • Don’t always give in to “right tool for the right job” Processing & Management Content Production Content Distribution Content Storage Content Consumption
  89. 89. Summary • Serverless by default • Avoid lock-in • Be cautious of the outside • Reuse data for good • Avoid a priori optimization • Deploy often • Keep simple and separate systems • Don’t always give in to “right tool for the right job” Go Build.
  90. 90. Thank you!
  91. 91. Remember to complete your evaluations!

×