SlideShare a Scribd company logo
1 of 25
Download to read offline
Erlang
                         for video delivery platforms




Friday, October 19, 12
About me

                         Worked for Delta, Bellsouth, Sun
                         Microsystems, Weather Channel

                         Love realtime, high volume systems

                         Sr. Director Engineering Ericsson
                           N2 Broadband -> Tandberg TV -> Ericsson


                         hwatkins @ github



Friday, October 19, 12
Media Delivery
                         Management System
                         Erlang (of course!)

                           Realtime processing

                           Protocol specific pieces

                         Ruby on Rails

                           Front end and some API’s



Friday, October 19, 12
Media Delivery
                         Management System
                         MongoDB

                           Persistence layer

                           Geo redundant

                         RabbitMQ (AMQP)

                           Messaging Bus



Friday, October 19, 12
VOD ecosystem




Friday, October 19, 12
Async processing




Friday, October 19, 12
Sync to Async




Friday, October 19, 12
Dynamic Routing
                         Ability to add
                         components on the
                         fly

                         Apply components
                         to different
                         profiles

                         In harmony with
                         multiple VM’s



Friday, October 19, 12
Competing Consumer




Friday, October 19, 12
Clients




                 HTTP               DSM-CC
                                   ISO/IEC 13818-6



Friday, October 19, 12
Java binary protocol

                         Class per section

                         Lots of code

                         Lots of objects

                         Lots of GC




Friday, October 19, 12
Binary Data




Friday, October 19, 12
Pattern Matching




Friday, October 19, 12
Memory Management

                         Java

                           Shared memory with reference count

                         Erlang

                           per thread heap

                           single assignment



Friday, October 19, 12
Memory Footprint

                         Identical functionality developed in
                         Java and Erlang

                           AMQP workers and connected to
                           MongoDB

                           Java averaged 1 GB memory

                           Erlang averaged 200 MB memory



Friday, October 19, 12
Threading Model
                         Threading is ideal for things like
                         http requests

                         Java

                           Threads heavyweight - thread pools

                         Erlang

                           Lightweight - spawn on demand

                           Think linear

Friday, October 19, 12
Clustering

                         Java

                           Memory - Terracotta

                         Erlang

                           Built in netadm:ping

                           Mnesia - disk or memory



Friday, October 19, 12
Runtime Debugging

                         REPL (Read-Eval-Print-Loop)

                         Cookie

                         Remote Shell

                         Issue commands




Friday, October 19, 12
OTP
                         Built for the real world - Use it!

                         application, gen_server and
                         supervisor

                           Use alot!

                         gen_fsm and gen_event

                           Not as much but still useful


Friday, October 19, 12
OTP Custom Behavior




Friday, October 19, 12
Packaging
                         reltool

                           rebar generate

                         start, stop,
                         console, attach

                         runtime vm and
                         libraries

                         small ~29M release


Friday, October 19, 12
Useful Libraries




Friday, October 19, 12
Rebar

                         De facto build
                         system

                         Wrap in simple
                         makefile for ease

                         Make dependancies
                         easy to manage




Friday, October 19, 12
folsom

                         Github boundary/
                         folsom

                         Metrics and
                         counters during
                         runtime

                         Expose through
                         mochiweb



Friday, October 19, 12
Others



                         Mochiweb - Web Server

                         Webmachine - Rest




Friday, October 19, 12

More Related Content

Similar to Erlang for video delivery

Caching, sharding, distributing - Scaling best practices
Caching, sharding, distributing - Scaling best practicesCaching, sharding, distributing - Scaling best practices
Caching, sharding, distributing - Scaling best practicesLars Jankowfsky
 
Apache Hadoop Talk at QCon
Apache Hadoop Talk at QConApache Hadoop Talk at QCon
Apache Hadoop Talk at QConCloudera, Inc.
 
PyJS intro
PyJS introPyJS intro
PyJS introDemix
 
Big app design for Node.js
Big app design for Node.jsBig app design for Node.js
Big app design for Node.jsSergi Mansilla
 
Triage: real-world error logging for web applications
Triage: real-world error logging for web applicationsTriage: real-world error logging for web applications
Triage: real-world error logging for web applicationsLuke Cawood
 
WordPress: Performance Optimization and Scaling - WordCamp Las Vegas 2011
WordPress: Performance Optimization and Scaling - WordCamp Las Vegas 2011WordPress: Performance Optimization and Scaling - WordCamp Las Vegas 2011
WordPress: Performance Optimization and Scaling - WordCamp Las Vegas 2011Matt Martz
 
OpenStack-Design-Summit-HA-Pairs-Are-Not-The-Only-Answer copy.pdf
OpenStack-Design-Summit-HA-Pairs-Are-Not-The-Only-Answer copy.pdfOpenStack-Design-Summit-HA-Pairs-Are-Not-The-Only-Answer copy.pdf
OpenStack-Design-Summit-HA-Pairs-Are-Not-The-Only-Answer copy.pdfOpenStack Foundation
 
OpenStack Summit :: Redundancy Doesn't Always Mean "HA" or "Cluster"
OpenStack Summit :: Redundancy Doesn't Always Mean "HA" or "Cluster"OpenStack Summit :: Redundancy Doesn't Always Mean "HA" or "Cluster"
OpenStack Summit :: Redundancy Doesn't Always Mean "HA" or "Cluster"Randy Bias
 
Complex Made Simple: Sleep Better With TorqueBox
Complex Made Simple: Sleep Better With TorqueBoxComplex Made Simple: Sleep Better With TorqueBox
Complex Made Simple: Sleep Better With TorqueBoxLance Ball
 
CloudFoundry and MongoDb, a marriage made in heaven
CloudFoundry and MongoDb, a marriage made in heavenCloudFoundry and MongoDb, a marriage made in heaven
CloudFoundry and MongoDb, a marriage made in heavenPatrick Chanezon
 
Mobile crossplatformchallenges siggraph
Mobile crossplatformchallenges siggraphMobile crossplatformchallenges siggraph
Mobile crossplatformchallenges siggraphchangehee lee
 
Mobile crossplatformchallenges siggraph
Mobile crossplatformchallenges siggraphMobile crossplatformchallenges siggraph
Mobile crossplatformchallenges siggraphJP Lee
 
Using Puppet and Cobbler to Automate Your Infrastructure
Using Puppet and Cobbler to Automate Your InfrastructureUsing Puppet and Cobbler to Automate Your Infrastructure
Using Puppet and Cobbler to Automate Your InfrastructurePhil Windley
 
Backend as a Service
Backend as a ServiceBackend as a Service
Backend as a ServiceLutz Kohl
 
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012kennethaliu
 
Rcos presentation
Rcos presentationRcos presentation
Rcos presentationmskmoorthy
 

Similar to Erlang for video delivery (20)

Caching, sharding, distributing - Scaling best practices
Caching, sharding, distributing - Scaling best practicesCaching, sharding, distributing - Scaling best practices
Caching, sharding, distributing - Scaling best practices
 
Apache Hadoop Talk at QCon
Apache Hadoop Talk at QConApache Hadoop Talk at QCon
Apache Hadoop Talk at QCon
 
PyJS intro
PyJS introPyJS intro
PyJS intro
 
Big app design for Node.js
Big app design for Node.jsBig app design for Node.js
Big app design for Node.js
 
Triage: real-world error logging for web applications
Triage: real-world error logging for web applicationsTriage: real-world error logging for web applications
Triage: real-world error logging for web applications
 
WordPress: Performance Optimization and Scaling - WordCamp Las Vegas 2011
WordPress: Performance Optimization and Scaling - WordCamp Las Vegas 2011WordPress: Performance Optimization and Scaling - WordCamp Las Vegas 2011
WordPress: Performance Optimization and Scaling - WordCamp Las Vegas 2011
 
OpenStack-Design-Summit-HA-Pairs-Are-Not-The-Only-Answer copy.pdf
OpenStack-Design-Summit-HA-Pairs-Are-Not-The-Only-Answer copy.pdfOpenStack-Design-Summit-HA-Pairs-Are-Not-The-Only-Answer copy.pdf
OpenStack-Design-Summit-HA-Pairs-Are-Not-The-Only-Answer copy.pdf
 
OpenStack Summit :: Redundancy Doesn't Always Mean "HA" or "Cluster"
OpenStack Summit :: Redundancy Doesn't Always Mean "HA" or "Cluster"OpenStack Summit :: Redundancy Doesn't Always Mean "HA" or "Cluster"
OpenStack Summit :: Redundancy Doesn't Always Mean "HA" or "Cluster"
 
Complex Made Simple: Sleep Better With TorqueBox
Complex Made Simple: Sleep Better With TorqueBoxComplex Made Simple: Sleep Better With TorqueBox
Complex Made Simple: Sleep Better With TorqueBox
 
Rubypalooza 2009
Rubypalooza 2009Rubypalooza 2009
Rubypalooza 2009
 
CloudFoundry and MongoDb, a marriage made in heaven
CloudFoundry and MongoDb, a marriage made in heavenCloudFoundry and MongoDb, a marriage made in heaven
CloudFoundry and MongoDb, a marriage made in heaven
 
Mobile crossplatformchallenges siggraph
Mobile crossplatformchallenges siggraphMobile crossplatformchallenges siggraph
Mobile crossplatformchallenges siggraph
 
Mobile crossplatformchallenges siggraph
Mobile crossplatformchallenges siggraphMobile crossplatformchallenges siggraph
Mobile crossplatformchallenges siggraph
 
Using Puppet and Cobbler to Automate Your Infrastructure
Using Puppet and Cobbler to Automate Your InfrastructureUsing Puppet and Cobbler to Automate Your Infrastructure
Using Puppet and Cobbler to Automate Your Infrastructure
 
Pagetypes
PagetypesPagetypes
Pagetypes
 
Backend as a Service
Backend as a ServiceBackend as a Service
Backend as a Service
 
Barcamp PT
Barcamp PTBarcamp PT
Barcamp PT
 
Xensummit Asia 2009 Talk
Xensummit Asia 2009 TalkXensummit Asia 2009 Talk
Xensummit Asia 2009 Talk
 
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
 
Rcos presentation
Rcos presentationRcos presentation
Rcos presentation
 

Recently uploaded

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfhans926745
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdflior mazor
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdfChristopherTHyatt
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 

Recently uploaded (20)

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdf
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 

Erlang for video delivery

  • 1. Erlang for video delivery platforms Friday, October 19, 12
  • 2. About me Worked for Delta, Bellsouth, Sun Microsystems, Weather Channel Love realtime, high volume systems Sr. Director Engineering Ericsson N2 Broadband -> Tandberg TV -> Ericsson hwatkins @ github Friday, October 19, 12
  • 3. Media Delivery Management System Erlang (of course!) Realtime processing Protocol specific pieces Ruby on Rails Front end and some API’s Friday, October 19, 12
  • 4. Media Delivery Management System MongoDB Persistence layer Geo redundant RabbitMQ (AMQP) Messaging Bus Friday, October 19, 12
  • 7. Sync to Async Friday, October 19, 12
  • 8. Dynamic Routing Ability to add components on the fly Apply components to different profiles In harmony with multiple VM’s Friday, October 19, 12
  • 10. Clients HTTP DSM-CC ISO/IEC 13818-6 Friday, October 19, 12
  • 11. Java binary protocol Class per section Lots of code Lots of objects Lots of GC Friday, October 19, 12
  • 14. Memory Management Java Shared memory with reference count Erlang per thread heap single assignment Friday, October 19, 12
  • 15. Memory Footprint Identical functionality developed in Java and Erlang AMQP workers and connected to MongoDB Java averaged 1 GB memory Erlang averaged 200 MB memory Friday, October 19, 12
  • 16. Threading Model Threading is ideal for things like http requests Java Threads heavyweight - thread pools Erlang Lightweight - spawn on demand Think linear Friday, October 19, 12
  • 17. Clustering Java Memory - Terracotta Erlang Built in netadm:ping Mnesia - disk or memory Friday, October 19, 12
  • 18. Runtime Debugging REPL (Read-Eval-Print-Loop) Cookie Remote Shell Issue commands Friday, October 19, 12
  • 19. OTP Built for the real world - Use it! application, gen_server and supervisor Use alot! gen_fsm and gen_event Not as much but still useful Friday, October 19, 12
  • 20. OTP Custom Behavior Friday, October 19, 12
  • 21. Packaging reltool rebar generate start, stop, console, attach runtime vm and libraries small ~29M release Friday, October 19, 12
  • 23. Rebar De facto build system Wrap in simple makefile for ease Make dependancies easy to manage Friday, October 19, 12
  • 24. folsom Github boundary/ folsom Metrics and counters during runtime Expose through mochiweb Friday, October 19, 12
  • 25. Others Mochiweb - Web Server Webmachine - Rest Friday, October 19, 12