SlideShare a Scribd company logo
1 of 6
Download to read offline
Open Source Meetup
1 
Intro 
Place Screen Here 
Vimeo—About Us 
Who are we? 
•Top 5 Video Site 
•Content Creators 
•Multimedia Hackers 
•Hipsters 
What do we do? 
•At-scale transcoding and video hosting at a high level of quality. 
•Create educational and entertaining content (VimeoVideo School) 
•Professional Video On Demand 
Who uses us? 
•Independent artists (e.g. Katy Perry before she got big) 
•Small to Large Professionals (Everyone from Joe Unknown to Joss Whedon) 
•Businesses (GitHub, Square Mile) 
•Regular people!
2 
Architecture 
Architecture and Infrastructure 
Transcoding in The Cloud (formerly known as data centers) 
Video Upload 
Metadata Collector (Caching) 
Job Scheduler 
GearmanDaemon 1 
Audio Encode 
GlusterFS 
Chunk 1 
Chunk 2 
Combine and Multiplex 
… 
Chunk N 
GearmanDaemon 2 
… 
GearmanDaemon N 
Server 1 
GearmanDaemon 1 
Video Worker 1 
Video Worker 2 
Video Worker N 
Audio Worker 
Mux/Combine Worker 
Server 2 
Video Worker 1 
Video Worker 2 
Video Worker M 
Audio Worker 
… 
Server N 
GearmanDaemon M 
Video Worker 1 
Video Worker 2 
Audio Worker
3 
Open Source Policies 
Long Term Maintainability and Benefits 
•Unlike many of the companies at IBC, we do not use Open Source software in bad faith; we contribute bug fixes, features, and at-scale-testing back, and do not violate licenses. 
•4of 5 members of the transcode team are or were upstream FFmpeg/Libavdevelopers, and all are active open source contributors. 
•It is not viable in the long term to fork internally, maintenance-wise, or feature-wise. 
•Many things we have needed have been gladly implemented by non-employee Open Source developers who appreciate our contributions and resources. 
•In short: Collaborating with the people whose code you are benefiting from has benefits for everyone. Being a jerk benefits nobody. 
•We open source a lot of our metrics stuff, and very soon, many of our media middleware libraries and packages. Stay tuned to the Vimeoand Mozilla Hacks blogs! 
Open Source
4 
Software 
What We Use 
A Few Awesome Media Projects 
•FFmpeg–Some companies at IBC use this, but won’t admit it. 
•x264–Ditto. 
•L-SMASH–Our preferred library for MP4 creation and demuxing/info. Rigidly strict to the specs, nice upstream developers, practical and realistic, has fewer LoCthan the space shuttle. 
•FFMS2–Offers frame-accurate, and pre-indexed seeking using the libav* libraries, and has an easy-to- use API. 
•Gearman–Great distributed job server with a simple protocol usable from any language. 
•FDK AAC–FraunhoferAAC encoder. 
•Opus–I’ll be working with Tim Terriberry(Xiph) and Yusuke Nakamura (L-SMASH) to define a sane Opus-in- MP4 spec that can be ratified by a proper authority. 
•Many packages for Go. 
•Many more! Ask me if you are curious. 
Things for the Future 
Daala–Doing interesting, possibly great things. Worth keeping an eye on. 
DASH–All client and server libraries are currently awful. We’re rolling our own. 
x265–Shaping up well, still not practical yet.

More Related Content

More from Vittorio Giovara

Block Cipher Modes of Operation And Cmac For Authentication
Block Cipher Modes of Operation And Cmac For AuthenticationBlock Cipher Modes of Operation And Cmac For Authentication
Block Cipher Modes of Operation And Cmac For AuthenticationVittorio Giovara
 
Fuzzing Techniques for Software Vulnerability Discovery
Fuzzing Techniques for Software Vulnerability DiscoveryFuzzing Techniques for Software Vulnerability Discovery
Fuzzing Techniques for Software Vulnerability DiscoveryVittorio Giovara
 
Parallel and Distributed Computing on Low Latency Clusters
Parallel and Distributed Computing on Low Latency ClustersParallel and Distributed Computing on Low Latency Clusters
Parallel and Distributed Computing on Low Latency ClustersVittorio Giovara
 
Software Requirements for Safety-related Systems
Software Requirements for Safety-related SystemsSoftware Requirements for Safety-related Systems
Software Requirements for Safety-related SystemsVittorio Giovara
 
Microprocessor-based Systems 48/32bit Division Algorithm
Microprocessor-based Systems 48/32bit Division AlgorithmMicroprocessor-based Systems 48/32bit Division Algorithm
Microprocessor-based Systems 48/32bit Division AlgorithmVittorio Giovara
 
Misra C Software Development Standard
Misra C Software Development StandardMisra C Software Development Standard
Misra C Software Development StandardVittorio Giovara
 
OpenSSL User Manual and Data Format
OpenSSL User Manual and Data FormatOpenSSL User Manual and Data Format
OpenSSL User Manual and Data FormatVittorio Giovara
 
Authenticated Encryption Gcm Ccm
Authenticated Encryption Gcm CcmAuthenticated Encryption Gcm Ccm
Authenticated Encryption Gcm CcmVittorio Giovara
 

More from Vittorio Giovara (11)

Il Caso Ryanair
Il Caso RyanairIl Caso Ryanair
Il Caso Ryanair
 
I Mercati Geografici
I Mercati GeograficiI Mercati Geografici
I Mercati Geografici
 
Block Cipher Modes of Operation And Cmac For Authentication
Block Cipher Modes of Operation And Cmac For AuthenticationBlock Cipher Modes of Operation And Cmac For Authentication
Block Cipher Modes of Operation And Cmac For Authentication
 
Crittografia Quantistica
Crittografia QuantisticaCrittografia Quantistica
Crittografia Quantistica
 
Fuzzing Techniques for Software Vulnerability Discovery
Fuzzing Techniques for Software Vulnerability DiscoveryFuzzing Techniques for Software Vulnerability Discovery
Fuzzing Techniques for Software Vulnerability Discovery
 
Parallel and Distributed Computing on Low Latency Clusters
Parallel and Distributed Computing on Low Latency ClustersParallel and Distributed Computing on Low Latency Clusters
Parallel and Distributed Computing on Low Latency Clusters
 
Software Requirements for Safety-related Systems
Software Requirements for Safety-related SystemsSoftware Requirements for Safety-related Systems
Software Requirements for Safety-related Systems
 
Microprocessor-based Systems 48/32bit Division Algorithm
Microprocessor-based Systems 48/32bit Division AlgorithmMicroprocessor-based Systems 48/32bit Division Algorithm
Microprocessor-based Systems 48/32bit Division Algorithm
 
Misra C Software Development Standard
Misra C Software Development StandardMisra C Software Development Standard
Misra C Software Development Standard
 
OpenSSL User Manual and Data Format
OpenSSL User Manual and Data FormatOpenSSL User Manual and Data Format
OpenSSL User Manual and Data Format
 
Authenticated Encryption Gcm Ccm
Authenticated Encryption Gcm CcmAuthenticated Encryption Gcm Ccm
Authenticated Encryption Gcm Ccm
 

Recently uploaded

"United Nations Park" Site Visit Report.
"United Nations Park" Site  Visit Report."United Nations Park" Site  Visit Report.
"United Nations Park" Site Visit Report.MdManikurRahman
 
Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)NareenAsad
 
Fabrication Of Automatic Star Delta Starter Using Relay And GSM Module By Utk...
Fabrication Of Automatic Star Delta Starter Using Relay And GSM Module By Utk...Fabrication Of Automatic Star Delta Starter Using Relay And GSM Module By Utk...
Fabrication Of Automatic Star Delta Starter Using Relay And GSM Module By Utk...ShivamTiwari995432
 
BURGER ORDERING SYSYTEM PROJECT REPORT..pdf
BURGER ORDERING SYSYTEM PROJECT REPORT..pdfBURGER ORDERING SYSYTEM PROJECT REPORT..pdf
BURGER ORDERING SYSYTEM PROJECT REPORT..pdfKamal Acharya
 
Interfacing Analog to Digital Data Converters ee3404.pdf
Interfacing Analog to Digital Data Converters ee3404.pdfInterfacing Analog to Digital Data Converters ee3404.pdf
Interfacing Analog to Digital Data Converters ee3404.pdfragupathi90
 
Theory for How to calculation capacitor bank
Theory for How to calculation capacitor bankTheory for How to calculation capacitor bank
Theory for How to calculation capacitor banktawat puangthong
 
RM&IPR M5 notes.pdfResearch Methodolgy & Intellectual Property Rights Series 5
RM&IPR M5 notes.pdfResearch Methodolgy & Intellectual Property Rights Series 5RM&IPR M5 notes.pdfResearch Methodolgy & Intellectual Property Rights Series 5
RM&IPR M5 notes.pdfResearch Methodolgy & Intellectual Property Rights Series 5T.D. Shashikala
 
Piping and instrumentation diagram p.pdf
Piping and instrumentation diagram p.pdfPiping and instrumentation diagram p.pdf
Piping and instrumentation diagram p.pdfAshrafRagab14
 
Introduction to Arduino Programming: Features of Arduino
Introduction to Arduino Programming: Features of ArduinoIntroduction to Arduino Programming: Features of Arduino
Introduction to Arduino Programming: Features of ArduinoAbhimanyu Sangale
 
Online crime reporting system project.pdf
Online crime reporting system project.pdfOnline crime reporting system project.pdf
Online crime reporting system project.pdfKamal Acharya
 
Intelligent Agents, A discovery on How A Rational Agent Acts
Intelligent Agents, A discovery on How A Rational Agent ActsIntelligent Agents, A discovery on How A Rational Agent Acts
Intelligent Agents, A discovery on How A Rational Agent ActsSheetal Jain
 
EMPLOYEE MANAGEMENT SYSTEM FINAL presentation
EMPLOYEE MANAGEMENT SYSTEM FINAL presentationEMPLOYEE MANAGEMENT SYSTEM FINAL presentation
EMPLOYEE MANAGEMENT SYSTEM FINAL presentationAmayJaiswal4
 
15-Minute City: A Completely New Horizon
15-Minute City: A Completely New Horizon15-Minute City: A Completely New Horizon
15-Minute City: A Completely New HorizonMorshed Ahmed Rahath
 
5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...archanaece3
 
Instruct Nirmaana 24-Smart and Lean Construction Through Technology.pdf
Instruct Nirmaana 24-Smart and Lean Construction Through Technology.pdfInstruct Nirmaana 24-Smart and Lean Construction Through Technology.pdf
Instruct Nirmaana 24-Smart and Lean Construction Through Technology.pdfEr.Sonali Nasikkar
 
ChatGPT Prompt Engineering for project managers.pdf
ChatGPT Prompt Engineering for project managers.pdfChatGPT Prompt Engineering for project managers.pdf
ChatGPT Prompt Engineering for project managers.pdfqasastareekh
 
Introduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AIIntroduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AISheetal Jain
 
Circuit Breaker arc phenomenon.pdf engineering
Circuit Breaker arc phenomenon.pdf engineeringCircuit Breaker arc phenomenon.pdf engineering
Circuit Breaker arc phenomenon.pdf engineeringKanchhaTamang
 
Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2T.D. Shashikala
 
Diploma Engineering Drawing Qp-2024 Ece .pdf
Diploma Engineering Drawing Qp-2024 Ece .pdfDiploma Engineering Drawing Qp-2024 Ece .pdf
Diploma Engineering Drawing Qp-2024 Ece .pdfJNTUA
 

Recently uploaded (20)

"United Nations Park" Site Visit Report.
"United Nations Park" Site  Visit Report."United Nations Park" Site  Visit Report.
"United Nations Park" Site Visit Report.
 
Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)
 
Fabrication Of Automatic Star Delta Starter Using Relay And GSM Module By Utk...
Fabrication Of Automatic Star Delta Starter Using Relay And GSM Module By Utk...Fabrication Of Automatic Star Delta Starter Using Relay And GSM Module By Utk...
Fabrication Of Automatic Star Delta Starter Using Relay And GSM Module By Utk...
 
BURGER ORDERING SYSYTEM PROJECT REPORT..pdf
BURGER ORDERING SYSYTEM PROJECT REPORT..pdfBURGER ORDERING SYSYTEM PROJECT REPORT..pdf
BURGER ORDERING SYSYTEM PROJECT REPORT..pdf
 
Interfacing Analog to Digital Data Converters ee3404.pdf
Interfacing Analog to Digital Data Converters ee3404.pdfInterfacing Analog to Digital Data Converters ee3404.pdf
Interfacing Analog to Digital Data Converters ee3404.pdf
 
Theory for How to calculation capacitor bank
Theory for How to calculation capacitor bankTheory for How to calculation capacitor bank
Theory for How to calculation capacitor bank
 
RM&IPR M5 notes.pdfResearch Methodolgy & Intellectual Property Rights Series 5
RM&IPR M5 notes.pdfResearch Methodolgy & Intellectual Property Rights Series 5RM&IPR M5 notes.pdfResearch Methodolgy & Intellectual Property Rights Series 5
RM&IPR M5 notes.pdfResearch Methodolgy & Intellectual Property Rights Series 5
 
Piping and instrumentation diagram p.pdf
Piping and instrumentation diagram p.pdfPiping and instrumentation diagram p.pdf
Piping and instrumentation diagram p.pdf
 
Introduction to Arduino Programming: Features of Arduino
Introduction to Arduino Programming: Features of ArduinoIntroduction to Arduino Programming: Features of Arduino
Introduction to Arduino Programming: Features of Arduino
 
Online crime reporting system project.pdf
Online crime reporting system project.pdfOnline crime reporting system project.pdf
Online crime reporting system project.pdf
 
Intelligent Agents, A discovery on How A Rational Agent Acts
Intelligent Agents, A discovery on How A Rational Agent ActsIntelligent Agents, A discovery on How A Rational Agent Acts
Intelligent Agents, A discovery on How A Rational Agent Acts
 
EMPLOYEE MANAGEMENT SYSTEM FINAL presentation
EMPLOYEE MANAGEMENT SYSTEM FINAL presentationEMPLOYEE MANAGEMENT SYSTEM FINAL presentation
EMPLOYEE MANAGEMENT SYSTEM FINAL presentation
 
15-Minute City: A Completely New Horizon
15-Minute City: A Completely New Horizon15-Minute City: A Completely New Horizon
15-Minute City: A Completely New Horizon
 
5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...
 
Instruct Nirmaana 24-Smart and Lean Construction Through Technology.pdf
Instruct Nirmaana 24-Smart and Lean Construction Through Technology.pdfInstruct Nirmaana 24-Smart and Lean Construction Through Technology.pdf
Instruct Nirmaana 24-Smart and Lean Construction Through Technology.pdf
 
ChatGPT Prompt Engineering for project managers.pdf
ChatGPT Prompt Engineering for project managers.pdfChatGPT Prompt Engineering for project managers.pdf
ChatGPT Prompt Engineering for project managers.pdf
 
Introduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AIIntroduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AI
 
Circuit Breaker arc phenomenon.pdf engineering
Circuit Breaker arc phenomenon.pdf engineeringCircuit Breaker arc phenomenon.pdf engineering
Circuit Breaker arc phenomenon.pdf engineering
 
Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2
 
Diploma Engineering Drawing Qp-2024 Ece .pdf
Diploma Engineering Drawing Qp-2024 Ece .pdfDiploma Engineering Drawing Qp-2024 Ece .pdf
Diploma Engineering Drawing Qp-2024 Ece .pdf
 

Transcoding video in the cloud for a Top 5 video site

  • 1.
  • 3. 1 Intro Place Screen Here Vimeo—About Us Who are we? •Top 5 Video Site •Content Creators •Multimedia Hackers •Hipsters What do we do? •At-scale transcoding and video hosting at a high level of quality. •Create educational and entertaining content (VimeoVideo School) •Professional Video On Demand Who uses us? •Independent artists (e.g. Katy Perry before she got big) •Small to Large Professionals (Everyone from Joe Unknown to Joss Whedon) •Businesses (GitHub, Square Mile) •Regular people!
  • 4. 2 Architecture Architecture and Infrastructure Transcoding in The Cloud (formerly known as data centers) Video Upload Metadata Collector (Caching) Job Scheduler GearmanDaemon 1 Audio Encode GlusterFS Chunk 1 Chunk 2 Combine and Multiplex … Chunk N GearmanDaemon 2 … GearmanDaemon N Server 1 GearmanDaemon 1 Video Worker 1 Video Worker 2 Video Worker N Audio Worker Mux/Combine Worker Server 2 Video Worker 1 Video Worker 2 Video Worker M Audio Worker … Server N GearmanDaemon M Video Worker 1 Video Worker 2 Audio Worker
  • 5. 3 Open Source Policies Long Term Maintainability and Benefits •Unlike many of the companies at IBC, we do not use Open Source software in bad faith; we contribute bug fixes, features, and at-scale-testing back, and do not violate licenses. •4of 5 members of the transcode team are or were upstream FFmpeg/Libavdevelopers, and all are active open source contributors. •It is not viable in the long term to fork internally, maintenance-wise, or feature-wise. •Many things we have needed have been gladly implemented by non-employee Open Source developers who appreciate our contributions and resources. •In short: Collaborating with the people whose code you are benefiting from has benefits for everyone. Being a jerk benefits nobody. •We open source a lot of our metrics stuff, and very soon, many of our media middleware libraries and packages. Stay tuned to the Vimeoand Mozilla Hacks blogs! Open Source
  • 6. 4 Software What We Use A Few Awesome Media Projects •FFmpeg–Some companies at IBC use this, but won’t admit it. •x264–Ditto. •L-SMASH–Our preferred library for MP4 creation and demuxing/info. Rigidly strict to the specs, nice upstream developers, practical and realistic, has fewer LoCthan the space shuttle. •FFMS2–Offers frame-accurate, and pre-indexed seeking using the libav* libraries, and has an easy-to- use API. •Gearman–Great distributed job server with a simple protocol usable from any language. •FDK AAC–FraunhoferAAC encoder. •Opus–I’ll be working with Tim Terriberry(Xiph) and Yusuke Nakamura (L-SMASH) to define a sane Opus-in- MP4 spec that can be ratified by a proper authority. •Many packages for Go. •Many more! Ask me if you are curious. Things for the Future Daala–Doing interesting, possibly great things. Worth keeping an eye on. DASH–All client and server libraries are currently awful. We’re rolling our own. x265–Shaping up well, still not practical yet.