Vimeo?The best place for video creators to put their videoThe best experience for everyone: • Video quality • Video player • Community • Tools for sharing and privacy Free Users Plus Users 500mb upload per week 5GB per week Advanced stats, customization, advanced Basic tools privacy options Wait for transcoding No wait for transcoding Free $60/year 2
Why AWS?S3, 2007 It was there High cost, small servers in managed hosting S3 had more features (query string auth – store and serve)EC2, 2008 Everyone uploads on Sundays Needed auto-scaling: two peak days per week, peak was 4x normal encoding load Free & fast transfer between EC2 and S3 3
The Workflow EC2 • Metadata User’s Raw Sourcefile Upload S3Browser sourcefile • Verified stored Machine • HD EC2 • SD Playable CDNTranscoder • Mobile video files • Thumbnails 4
On-demand, ReservedReserved capacity is cheaper Reserve capacity for paying customers • Guaranteed availability, even when Amazon runs out of on-demand Reserve capacity for 100% utilization Reserve capacity for 75% utilization Still need on-demand for elasticity 5
Upload/Transcoding InfrastructureUpload m1.large instances Nginx + nodejs + SQSTranscoding c1.xlarge instances Reserved, On demand and spots Spots for low priority jobs. Instances will shut down without warning 6
Spots@VimeoLeverage open source libraries (boto in python)Keep it SimpleOnetime spot requests, One instance per request, across all availability zonesBidding strategy 7
Conceptualization the ObviousThinking of spots for your Database and Web servers?Failover to On demand when spot market is saturatedNever kill spots. Let Amazon do it.Watch out for open spot requests. Add expiry to your requests.Keep tweaking your Algorithm. 8
ResultsSome months we save 50% over pure on-demand instancesGuaranteed availability for our plus users 9
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.