WSO2's API Vision: Unifying Control, Empowering Developers
Delivering on the promise of the cloud for digital media, aspera on demand
1. Delivering on the Promise of
the Cloud for Digital Media
Aspera OnDemand
Bhavik Vyas | Director of Cloud Solutions & Partnerships | 14 th June 2012
2. ASPERA SOFTWARE INC
Creating next-generation transport technologies that move
the world’s digital assets at maximum speed, regardless of
file size, transfer distance and network conditions.
Distribute Collaborate Automate
14
Transport
3. Aspera Corporate Overview
Headquarters: Emeryville, CA
Founded: 2004
Ownership: Privately Held
Employees: ~100
Patents: Issued and pending in over 32 countries
Customers: over 1,200 customers, over 10,000 licenses deployed
Markets Served: Media and Entertainment, Federal Government, Life Sciences, Healthcare, Cloud
Computing, Software and Gaming, Financial Services, Legal, eDiscovery, Engineering, Technology,
Telecommunications, Service Providers, Architecture and Design, Enterprise IT
Creators of the fasp™ protocol
• Innovative, patented, highly efficient bulk data transport technology
• Unique and core to Aspera’s high-performance file transfer software suite
• Outperforms software and hardware WAN acceleration solutions
• Ranked first in every WAN transfer throughput benchmark
5. THE MEDIA CLOUD?... IT’S HERE.
Thursday, March 8, 2012, New York, NY
Aspera, Amazon Web Services and Zencoder teamed up for a special event at New York City’s Tribeca
Grand Hotel on Thursday, March 8th, 2012. During this afternoon of presentations by industry experts,
attendees experienced what is now possible in the cloud for digital media companies.
6. ASPERA PRODUCT & TECHNOLOGY PORTFOLIO
DISTRIBUTE COLLABORATE AUTOMATE
Complete portfolio of Global person-to-person Web-based application
servers and clients for and project-based and SDK for creating and
high-speed data delivery exchange and managing automated file-
and distribution. collaboration of files based workflows.
and directories.
TRANSPORT
Our unique, patented faspTM transport technologies provide unparalleled speed,
efficiency, and bandwidth control over any size, distance, and network.
9. HOW BIG? BIG.
STANDARD FORMATS
CD DVD Blu-ray
.75 GB 4.7 GB 25 GB
COMPRESSED VIDEO
FORMAT SD 720p HD 1080p HD
Stream rate (Mbps) 1.5 5 8
1 hour of content .7 GB 2.3 GB 3.6 GB
UNCOMPRESSED VIDEO
FORMAT SD 720p HD 1080p HD
Stream rate (Mbps) 125 553 1290
1 hour of content 56 GB 249 GB 581 GB
UNCOMPRESSED VIDEO
FORMAT Digital Cinema 2K Digital Cinema 4K 3D Cinema Stream
Stream rate (Mbps) 1900 3800 5730
1 hour of content 855 GB 1710 GB 2579 GB
A single digital cinema production can be 800K–1M 2K/4K frames
11. TCP PERFORMANCE: BOUND BY DESIGN
TCP powers:
• FTP
• SCP
• RSYNC
• HTTP
• CIFS
• NFS
(
Throughput µ 1 RTT * p )
RTT = round trip time
p = packet loss rate
12. BIG-DATA and WAN TRANSFER WITH TCP
TCP WAS DESIGNED IN THE EARLY 80’S
• When data was small & bandwidth was limited
• Fantastic for reliable data delivery
• Not fast enough for big-data
TCP IS THE ENGINE THAT DRIVES
• FTP, HTTP & HTTPS
• RSYNC, SCP
• CIFS & NFS
TCP DOES NOT LIKE NETWORK LATENCY/ RTT
• Geographic distance increases latency
• Network congestion increases latency
TCP DOES NOT LIKE PACKET LOSS
• Loss is caused by congestion
• Different network capacity
• Wireless and satellite communications
13. ASPERA FASP: HIGH-PERFORMANCE DATA TRANSPORT
Maximum transfer speed
• Line-rate transfer performance
• Transfer performance scales with bandwidth
Uncompromising security and reliability
• Secure, user/endpoint authentication
• AES-128 cryptography in transit and at-rest
Congestion Avoidance and Policy Control
• Automatic, full utilization of available bandwidth
• On-the-fly prioritization and bandwidth allocation
Scalable management, monitoring and control
• Real-time progress, performance and bandwidth utilization
• Detailed transfer history, logging, and manifest
Low Overhead
• Optimal end-to-end throughput efficiency
• High performance with large files or large sets of small files
Enterprise-class File Transfer
• Transfers up to thousands of times faster than FTP
• Precise and predictable transfer times
17. OVERCOMING BOTH BOTTLENECKS
#1 — TRANSFER DATA TO EC2 OVER WAN EFFECTIVE THROUGHPUT
• HTTP transfer over WAN (single stream)
<10 Mbps
• Typical internet conditions
• 50–250ms latency & 0.1–3% packet loss
<10 to 100 Mbps
• 15 parallel http streams
up to 1Gbps
• Aspera fasp transfer over WAN to EC2
(per EC2 Extra Large Instance)
#2 — TRANSFER DATA FROM EC2 TO S3 EFFECTIVE THROUGHPUT
• Single HTTP stream Up to 100 Mbps
• 15 parallel http streams
• Must use S3 Multipart API
• Must chuck data on r/w
Up to 1Gbps
• Aspera S3 Proxy Up to 1Gbps
• With parallel I/O http streams (per EC2 Extra Large Instance)
#1 + #2 — WAN TRANSFER: ON-PREM TO/FROM TO
EFFECTIVE THROUGHPUT
S3
• Aspera On Demand for AWS Up to 1Gbps
• With parallel I/O http streams (per EC2 Extra Large Instance)
20. ASPERA SOFTWARE PRODUCT & TECHNOLOGY PORTFOLIO
Distribute Collaborate Automate
Complete portfolio of servers and clients for high- Global person-to-person and project-based Web-based application and SDK for creating and
speed data delivery and distribution. exchange and collaboration of files and directories. managing automated file-based workflows.
Enterprise and Connect Server faspex™ Server Orchestrator
• Universal file transfer server and web-based • Secure digital delivery and collaborative file • Intuitive graphical workflow designer
interface and directory listing transfers with remote users and partners • File processing decision tree and flow
Client and Point-to-point • Web, email, mobile client options • Rich and flexible plug-in architecture for third-
• Uni- and bi-directional transfer clients • Comprehensive administration, user party process integration
Connect management & access control • Comprehensive library of plug-ins for
• Web browser plug-in faspex™ Multi-Server / HA transcoding, A/V, QC, archive, notifications
Mobile • Automated bi-directional relays between sites • High volume processing
• High-speed transfer for mobile devices
• 3-tier architecture with support for clustering, HA • Detailed dashboard, workflow, and step-level
Sync progress reporting.
• Highly scalable, multidirectional file replication Cargo
• Automated package downloads • Open development framework for designing
and synchronization
and integrating automation pipelines
Transport
Our unique, patented transport technologies provide unparalleled speed, efficiency, concurrency and bandwidth control over any size,
distance, and network
fasp™ fasp3™ Aspera On-Demand S3|Direct
Patented, file-based bulk data transport Next-gen fasp for any block-data High-speed transfer direct to cloud storage (S3)
fasp-AIR™ fasp-MC™ Console transport management
3G, LTE and Wi-Fi networks fasp for delivery over multicast web-based management, monitoring, and reporting
21. OPEN PLATFORM 3RD PARTY LIBRARY
Transcoding Video Quality Control Ad Insertion &
Advertising Planning
Watermarking
IT Management
File Transfer Antivirus
Encryption
22. ASPERA -AWS CASE STUDIES
• Internationally disperse venues and broadcasters
International • Contribution, processing and AWS S3 cloud archive
Sports Tournament
• Automated processing, routing, notifications and delivery
• Business critical, revenue generating system
High Volume • Thousands of content and metadata files per hour
Broadcast Operations • Lights-out, hands-off, 7x24x365
23. INTERNATIONAL SPORTS TOURNAMENT
Globally-Dispersed
Broadcasters
People’s
Homes
Media Centers at the Sports Venues
International
Sports Tournament
24. INTERNATIONAL SPORTS TOURNAMENT
– ASPERA FASP™ TRANSFERS
International Data/Broadcast Center
•HD
•Metadata
fasp
HD + Metadata fasp
Sports Venue
Media Center
Broadcasters
•Low-Res Proxies
•Audio Globally-Dispersed
fasp •Metadata Broadcasters
People’
s
Content Homes
Web Portal
International
Sports Tournament
25. INTERNATIONAL SPORTS TOURNAMENT – WORKFLOW
ORCHESTRATION
International Data/Broadcast Center
On-going: On-going:
• R/T monitoring • Cloud monitoring
• Notifications • HA management
1. Detect arrival
2. Notify Operations
Aspera Orchestrator: Controlled fasp
fasp
Processing and Distribution Workflow
fasp
HD + Metadata
3. Stage for processing
Sports Venue 4. Detect processed content
Media Center 5. Transfer HD to AWS
6. Transfer Low-Res to Portal
Globally-Dispersed
Broadcasters
People’
fasp
s
Homes
Aspera Faspex API
call for remote
Content package creation
Web and distribution
International Portal
Sports Tournament
27. Netflix
Post Netflix
Vendors Cloud CDN’s
Houses Staging
Encoding
28. Tens of TB arriving each month
Even bigger spikes before a new region launch
Files up to hundreds of GB in size
Multiple vendors and post houses delivering to us
from across the world
29. Proprietary hardware is painful
And doesn’t scale well
And doesn’t scale quickly!
Vendors claim delivery when they haven’t
delivered
30. more encoding as each axis grows
2012
Content Catalog
2007
Device Ecosystem
2007 2012
31. At Netflix
For better business agility
For unpredictable business growth
We want to use clouds, not build them
▪ Our raison d’etre is movies, not datacenters
In Encoding Operations
For response to spikes of content
▪ Big launch days
▪ Entire region launches
▪ New device launches (Wii, PS3, iPad etc)
Vast unified pool of resources
32. No cloud quite like AWS at the moment
Literally thousands of machines can be
transcoding at any one time
Amazon is also a great Netflix partner
Open to multiple clouds when they can meet
our scale requirements
33. Direct delivery into Amazon S3 Cloud
100% scalable
One less hop
34. We could build something suitable
But is our time best spent here?
And how much time would we need to try and
shift an industry in this area?
Buying makes more sense
Focus: We aren’t the only company with this need
Talent density: We spend money to license the
technology, but we gain even more talent and
bandwidth to solve Netflix specific problems
35. Industry recognized digital delivery
technology
Excellent core product
Optimized for huge file transfers under adverse
network conditions
Built in monitoring and controls
Extensive structured logging for diagnosing issues
Strong and responsive support team
36. Worrying about disk space
Months of lag time to scale out
Distracting IT with proprietary hardware
Building for spikes instead of averages
We get to spend more time growing and less
time worrying about the logistics of growth