Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Transcoding video in the cloud for a Top 5 video site


Published on

Slides written by Derek Buitenhuis, check out the talk given at IBC in Amsterdam for the EBU Open Source Meetup:

In this talk we explain how Vimeo internals work and how much we rely on (and contribute back to) awesome open source projects.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Transcoding video in the cloud for a Top 5 video site

  1. 1. Open Source Meetup
  2. 2. 1 Intro Place Screen Here Vimeo—About Us Who are we? •Top 5 Video Site •Content Creators •Multimedia Hackers •Hipsters What do we do? •At-scale transcoding and video hosting at a high level of quality. •Create educational and entertaining content (VimeoVideo School) •Professional Video On Demand Who uses us? •Independent artists (e.g. Katy Perry before she got big) •Small to Large Professionals (Everyone from Joe Unknown to Joss Whedon) •Businesses (GitHub, Square Mile) •Regular people!
  3. 3. 2 Architecture Architecture and Infrastructure Transcoding in The Cloud (formerly known as data centers) Video Upload Metadata Collector (Caching) Job Scheduler GearmanDaemon 1 Audio Encode GlusterFS Chunk 1 Chunk 2 Combine and Multiplex … Chunk N GearmanDaemon 2 … GearmanDaemon N Server 1 GearmanDaemon 1 Video Worker 1 Video Worker 2 Video Worker N Audio Worker Mux/Combine Worker Server 2 Video Worker 1 Video Worker 2 Video Worker M Audio Worker … Server N GearmanDaemon M Video Worker 1 Video Worker 2 Audio Worker
  4. 4. 3 Open Source Policies Long Term Maintainability and Benefits •Unlike many of the companies at IBC, we do not use Open Source software in bad faith; we contribute bug fixes, features, and at-scale-testing back, and do not violate licenses. •4of 5 members of the transcode team are or were upstream FFmpeg/Libavdevelopers, and all are active open source contributors. •It is not viable in the long term to fork internally, maintenance-wise, or feature-wise. •Many things we have needed have been gladly implemented by non-employee Open Source developers who appreciate our contributions and resources. •In short: Collaborating with the people whose code you are benefiting from has benefits for everyone. Being a jerk benefits nobody. •We open source a lot of our metrics stuff, and very soon, many of our media middleware libraries and packages. Stay tuned to the Vimeoand Mozilla Hacks blogs! Open Source
  5. 5. 4 Software What We Use A Few Awesome Media Projects •FFmpeg–Some companies at IBC use this, but won’t admit it. •x264–Ditto. •L-SMASH–Our preferred library for MP4 creation and demuxing/info. Rigidly strict to the specs, nice upstream developers, practical and realistic, has fewer LoCthan the space shuttle. •FFMS2–Offers frame-accurate, and pre-indexed seeking using the libav* libraries, and has an easy-to- use API. •Gearman–Great distributed job server with a simple protocol usable from any language. •FDK AAC–FraunhoferAAC encoder. •Opus–I’ll be working with Tim Terriberry(Xiph) and Yusuke Nakamura (L-SMASH) to define a sane Opus-in- MP4 spec that can be ratified by a proper authority. •Many packages for Go. •Many more! Ask me if you are curious. Things for the Future Daala–Doing interesting, possibly great things. Worth keeping an eye on. DASH–All client and server libraries are currently awful. We’re rolling our own. x265–Shaping up well, still not practical yet.