Slides written by Derek Buitenhuis, check out the talk given at IBC in Amsterdam for the EBU Open Source Meetup: https://vimeo.com/106189441
In this talk we explain how Vimeo internals work and how much we rely on (and contribute back to) awesome open source projects.
3. 1
Intro
Place Screen Here
Vimeo—About Us
Who are we?
•Top 5 Video Site
•Content Creators
•Multimedia Hackers
•Hipsters
What do we do?
•At-scale transcoding and video hosting at a high level of quality.
•Create educational and entertaining content (VimeoVideo School)
•Professional Video On Demand
Who uses us?
•Independent artists (e.g. Katy Perry before she got big)
•Small to Large Professionals (Everyone from Joe Unknown to Joss Whedon)
•Businesses (GitHub, Square Mile)
•Regular people!
4. 2
Architecture
Architecture and Infrastructure
Transcoding in The Cloud (formerly known as data centers)
Video Upload
Metadata Collector (Caching)
Job Scheduler
GearmanDaemon 1
Audio Encode
GlusterFS
Chunk 1
Chunk 2
Combine and Multiplex
…
Chunk N
GearmanDaemon 2
…
GearmanDaemon N
Server 1
GearmanDaemon 1
Video Worker 1
Video Worker 2
Video Worker N
Audio Worker
Mux/Combine Worker
Server 2
Video Worker 1
Video Worker 2
Video Worker M
Audio Worker
…
Server N
GearmanDaemon M
Video Worker 1
Video Worker 2
Audio Worker
5. 3
Open Source Policies
Long Term Maintainability and Benefits
•Unlike many of the companies at IBC, we do not use Open Source software in bad faith; we contribute bug fixes, features, and at-scale-testing back, and do not violate licenses.
•4of 5 members of the transcode team are or were upstream FFmpeg/Libavdevelopers, and all are active open source contributors.
•It is not viable in the long term to fork internally, maintenance-wise, or feature-wise.
•Many things we have needed have been gladly implemented by non-employee Open Source developers who appreciate our contributions and resources.
•In short: Collaborating with the people whose code you are benefiting from has benefits for everyone. Being a jerk benefits nobody.
•We open source a lot of our metrics stuff, and very soon, many of our media middleware libraries and packages. Stay tuned to the Vimeoand Mozilla Hacks blogs!
Open Source
6. 4
Software
What We Use
A Few Awesome Media Projects
•FFmpeg–Some companies at IBC use this, but won’t admit it.
•x264–Ditto.
•L-SMASH–Our preferred library for MP4 creation and demuxing/info. Rigidly strict to the specs, nice upstream developers, practical and realistic, has fewer LoCthan the space shuttle.
•FFMS2–Offers frame-accurate, and pre-indexed seeking using the libav* libraries, and has an easy-to- use API.
•Gearman–Great distributed job server with a simple protocol usable from any language.
•FDK AAC–FraunhoferAAC encoder.
•Opus–I’ll be working with Tim Terriberry(Xiph) and Yusuke Nakamura (L-SMASH) to define a sane Opus-in- MP4 spec that can be ratified by a proper authority.
•Many packages for Go.
•Many more! Ask me if you are curious.
Things for the Future
Daala–Doing interesting, possibly great things. Worth keeping an eye on.
DASH–All client and server libraries are currently awful. We’re rolling our own.
x265–Shaping up well, still not practical yet.
