Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Serverless architecture

1,641 views

Published on

AWS Media and Entertainment Cloud Symposium - Managing digital assets in a serverless architecture

Published in: Business
  • Be the first to comment

Serverless architecture

  1. 1. Managing Digital Assets in a Serverless Architecture Chris Elleman – Manager, Solutions Architect (Media Practice) Erik Åhlin – Founder and CEO, Vidispine March 2016
  2. 2. We see lots of customers at different points on the spectrum File Based Management Asset Management Challenges of Managing Content
  3. 3. Integrated services for ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets (photos, video, audio, documents) CMS DAM MAM Documents Web Content Photos Creatives Models Video Audio Wordpress Drupal Censhare Escenic Adobe CQ OpenText CHP T3 Media/Wazee Vidispine Dalet Examples Creation Distribution Edit Repurpose Review Approve Asset Management Definition
  4. 4. • Gap: Reconciling Legacy vs. Born-In-The- Cloud – Large enterprises are challenged with keeping legacy systems and processes in- house – Need to share/collaborate is strong Legacy DAM S3 Glacier DAM DAM DAM Asset Management Gaps (1) • Solution: Federate DAM solutions Edge DAMs + Core DAMs Deep Storage on AWS
  5. 5. • Gap: Reconciling Content Gravity – For heavy-duty DAM/MAM requirements the volume and mass of the generated content makes file- based workflows very painful • Solution: co-locating the creative/edit/render/review process with the data – Amazon AppStream – GPU Enabled VDI (AWS Workspaces, Teradici) – Others: browser based tooling (edits, modeling, correction) S3(ex: Avere,SoftNAS) DAM Virtual Creative Apps Asset Management Gaps (2)
  6. 6. Online Storage Catalog Management Proxy & Transcode Processing Search | Collaboration Services Process Management | Workflow Services Nearline Storage Offline Storage Import / Export Services Tools Adapter Ingest Services Security | Rights Management Services CDN File Transfer WAFS Architecture Concepts – Digital Asset Management
  7. 7. Online Storage Catalog Management Proxy & Transcode Processing Search | Collaboration Services Process Management | Workflow Services Nearline Storage Offline Storage Import / Export Services Tools Adapter Ingest Services Security | Rights Management Services CDN File Transfer WAFS Standard S3-IA Glacier S3 Glacier Architecture Concepts – with AWS Services
  8. 8. Online Storage Catalog Management Proxy & Transcode Processing Search | Collaboration Services Process Management | Workflow Services Nearline Storage Offline Storage Import / Export Services Tools Adapter Ingest Services Security | Rights Management Services CDN File Transfer WAFS Standard S3-IA Glacier S3 Glacier Cognito Identity & Access Management Architecture Concepts – with AWS Services
  9. 9. Online Storage Catalog Management Proxy & Transcode Processing Search | Collaboration Services Process Management | Workflow Services Nearline Storage Offline Storage Import / Export Services Tools Adapter Ingest Services Security | Rights Management Services CDN File Transfer WAFS Standard S3-IA Glacier S3 Glacier Cognito Identity & Access Management DynamoDB Architecture Concepts – with AWS Services
  10. 10. Online Storage Catalog Management Proxy & Transcode Processing Search | Collaboration Services Process Management | Workflow Services Nearline Storage Offline Storage Import / Export Services Tools Adapter Ingest Services Security | Rights Management Services CDN File Transfer WAFS Standard S3-IA Glacier S3 Glacier Cognito Identity & Access Management DynamoDB Elastic Transcoder & Elemental Architecture Concepts – with AWS Services
  11. 11. Online Storage Catalog Management Proxy & Transcode Processing Search | Collaboration Services Process Management | Workflow Services Nearline Storage Offline Storage Import / Export Services Tools Adapter Ingest Services Security | Rights Management Services CDN File Transfer WAFS Standard S3-IA Glacier S3 Glacier Cognito Identity & Access Management DynamoDB Elastic Transcoder & Elemental API GatewayLambda Architecture Concepts – with AWS Services
  12. 12. Online Storage Catalog Management Proxy & Transcode Processing Search | Collaboration Services Process Management | Workflow Services Nearline Storage Offline Storage Import / Export Services Tools Adapter Ingest Services Security | Rights Management Services CDN File Transfer WAFS Standard S3-IA Glacier S3 Glacier Cognito Identity & Access Management DynamoDB Elastic Transcoder & Elemental API GatewayLambda ElasticSearch Architecture Concepts – with AWS Services
  13. 13. Online Storage Catalog Management Proxy & Transcode Processing Search | Collaboration Services Process Management | Workflow Services Nearline Storage Offline Storage Import / Export Services Tools Adapter Ingest Services Security | Rights Management Services CDN File Transfer WAFS Standard S3-IA Glacier S3 Glacier Cognito Identity & Access Management DynamoDB Elastic Transcoder & Elemental API GatewayLambda ElasticSearch AWS Import/ Export Snowball Architecture Concepts – with AWS Services
  14. 14. Ingest Bucket CloudFormation Template DynamoDB Cognito IAM role Lambda MetaData Extract Asset Bucket multimedia User Index HTML5 Static Site Elastic Transcoder IAM ElasticSearch API Gateway Lambda App Logic SimpleDAM – Serverless Architecture Glacier Archive JS SDK Meta Data Searches
  15. 15. Ingest Bucket Multimedia file 1. S3 Upload 2. S3 Event Lambda Function 4. Transcode Job Lambda Sub-Function (metaDataExtract) Elastic Transcoder 3. Extract MetaData 5. Build source XML Lambda Sub-Function (metaTransform) + DynamoDB 6. ETL Job format data for DynamoDB 7. Create DynamoDB record Asset Bucket 9. Copy Asset Into Asset Bucket ElasticSearch 8. Index Content SNS Topic 10. On Success or Failure, Remove Ingest Object Lambda Function SimpleDAM – Video Ingest Process
  16. 16. Key Decision for a DAM – Buy or Build? Buy • Lack of free development resource or skills • Exiting on premise commercial relationship • DAM capabilities aren’t a market differentiator • Broad set of requirements • Fast to set-up and provision Build • Development resource and skills available in-house • Lack of cap’ex budget for licensing • DAM capabilities will be a market differentiator • Narrow or specialist requirements • Start with an MVP and iterate with the needs of the business
  17. 17. ©Amazon.com, Inc. and its affiliates. All rights reserved. Thank you
  18. 18. Vidispine - Cloud Native Content Management for Developers and Media Professionals Erik Åhlin, CEO, co-founder
  19. 19. Vidispine in 15s – the problem we solve Broadcasters and media companies are looking for • Incorporate cloud and cloud services in a pace the budget allows. An evolution rather than a revolution. • Complete, managed and real elasticity for video content to control cost but still being agile • Repurposing content to a much higher degree • Fast turn-around story-telling with video and images on all platforms • Flexibility in business models and technology choices
  20. 20. Vidispine API-based Content Management PaaS Fully featured back-end for any media application Key areas • Metadata • Multi-format management • Performance & Scalability • Cloud native architecture (componentized and distributed) • Support for Multi-application
  21. 21. VidiXplore – The Gravity Point for your content ”Easy to Like” user experience Support for core media management tasks Extensible Component in a ”bigger picture” SaaS (private or public) Leverage Vidispine Content Management PaaS
  22. 22. Combine on-prem & cloud - start your migration 1. Place Vidispine Server Agent (VSA) next to existing on-prem storage 2. Create proxy, analyze metadata, connect to MAM/DAM locally 3. Keep link between ’cloud-side’ asset and ’on-prem’ asset 4. Start execute your cloud strategy by doing what cloud does best 5. Migrate infrastructure in your pace means you can focus on business
  23. 23. VidiXplore – The Gravity Point for your content VIDISPINE PaaS with APIs Transcoder & QC VDA Storage On-Prem Infrastructure Vidispine Server Agent Cloud Services VSAs and/or Biz Destinations
  24. 24. Real World Example #1 Global Premium Content Delivery Spread out workforce and clients Cost efficient and adaptive workflows
  25. 25. VidiXplore at Premium Content Delivery Leader VidiXplore with Custom Panels for managing audio tracks, mezzanine files, delivery points, etc Amazon EC2 Amazon CloudFront Amazon S3 w SSE hosting 8 TB proxies 65 GB thumbnails Amazon VPC Amazon Route 53 On-Prem VSAs connected to local storage Vidispine APIs for integration to workflow engine and more Data & Metadata to Vidispine PaaS Proxies direct to S3 for performance gain Amazon RDS
  26. 26. Migrate and Scale By knowing your content you are ready to scale Spend wisely - let cloud do what cloud do best • Workflows, common user interface for spread-out workforce • Distribute and share ready content • Store/archive and ”second copy” • Burst-out transcoding and compute Integrate and leverage popular cloud services • Dropbox, Slack, Box.com, WeVideo, CloudConvert, Metadata harvesting Migrate, Scale, Adapt also Business Processes, Functions,
  27. 27. Real World Example #2 Partner Company
  28. 28. Alfred – the Dev Ops Butler Partner Company: DSB Using Vidispine dockerized Scaling, Logging, Action Packs AWS Services used • EC2 Container Service • Docker • Node.Js • MongoDB
  29. 29. Real World Example #3 Leading US-based Media Giant Quickly prototyped a complete DAM workflow (Clip Library) Validated the technology stack on very low budget
  30. 30. Prototyping on AWS Marketplace Leading US-based Media Giant 10+ MAMs and DAMs already Some workflows are fit and ready for cloud Overview • Total POC went 6 weeks – total of 4 developers. • Total assets added (5 thousand). • Used the bare minimum box that Vidispine AMI Trial would run on (m3.medium)
  31. 31. Prototyping on AWS Marketplace Next steps • Run Vidispine as HA, where SOLR runs in another EC2 and Postgres runs in RDS • Performance testing with hundreds of thousands of assets Resources used • AWS EC2 (website PoC ran here) • AWS RDS (Postgres DB) • AWS S3 (hot folder in/out) • Vidispine Developer Edition AMI • Bitbucket (PoC code) • Jenkins (for deploying to EC2) • MEAN stack(code stack)
  32. 32. What’s next? Editing • Some very promising prototypes using AWS WorkSpaces Expert systems based on machine learning • Systems adapting to the task at hand rather than how it was originally designed Large-scale metadata harvesting True ubiquitous processing and storing • What is a file, what is a computer? • What to we really want computers to do for us and how? Amazon WorkSpaces Amazon Machine Learning AWS Lambda

×