SlideShare a Scribd company logo
1 of 21
Facebook
    Scaling Walkthrough

Moritz Haarmann - Ultrasuperlargescale Systems
Numbers
•   800.000.000 active users

•   > 50% log on every given day

•   250.000.000 Photos every single day ( Flickr:
    6Bn total )

•   30.000.000.000 new pieces of content monthly
30.000.000.000
30.000.000.000
As if everyone now living on earth posts 4.28 updates every month.
Building Blocks
Facebook is built using



•   Web Servers ( Running Hip-Hop PHP )

•   Services ( Search, Ads )

•   Memcached & MySQL

•   Immense amounts of glue
Write Strategy

•   Writes take place centrally in California

•   3.5 Million Changed Rows per Second ( Peak )

•   2010: 1800 DB Servers

•   horizontal scaling approach not disclosed

•   consistency is important ( avoiding „unhappy
    users“ )
Glue

•   Massively distributed architecture

•   Glue keeping it together

•   Many systems built in-house to meet
    giiaaanoourmus requirements
Haystack


•   Photos

•   Handles everything from HTTP to storage

•   Aimed at minimizing IO-Operations

•   Append-Only!
Memcached


•   Placed between MySQL and Web Tier

•   Stores only „plain data“, no joins or other
    complicated stuff

•   Faster if Web Server works on data
BigPipe


•   Assembles the output pages

•   everything that is needed retrieved in parallel

•   Fault tolerant, will work even if parts of a page
    are not available
What else?
Live Profiling


•   Facebook monitors their life systems
    continously at a PHP-Method level ( using
    XHProf ).
Graceful Degradation


•   High awareness ( Monitoring ) of perfomance
    problems

•   Features can be disabled ( very ne-grained )
    to keep the core features running smoothly
Keeping it running


•   New features are launched ,dark‘, without
    visible elements, to stress test the backend
    with real load

•   Incremental roll-outs decrease the impact of
    bug or malfunction
Open Source



•   Most parts are open source

•   Either used or created and then os‘d
Big Bang

             •     On September 23, 2010, Facebook was down
                   for most users for about 3 hours

             •     A wrongly identi ed ,invalid‘ cache value lead
                   to requests hammering the DB tier

             •     A system designed to prevent failures created
                   one!

             •     Only way to recover was to completely shut
                   down access to the DB - downtime

https://www.facebook.com/note.php?note_id=431441338919&id=9445547199&ref=mf ( great comments, too )
Thanks.
Sources

•   http://royal.pingdom.com/2010/06/18/the-software-behind-
    facebook/

•   https://www.facebook.com/note.php?note_id=76191543919

•   https://www.facebook.com/notes/facebook-engineering/bigpipe-
    pipelining-web-pages-for-high-performance/389414033919

•   http://blog.kissmetrics.com/facebook-statistics/

More Related Content

Viewers also liked

Mau 2015 scaling facebook at care.com
Mau 2015   scaling facebook at care.com Mau 2015   scaling facebook at care.com
Mau 2015 scaling facebook at care.com Grow.co
 
Marketing and Technology Go Hand-in Hand, But Where are They Going?
Marketing and Technology Go Hand-in Hand, But Where are They Going? Marketing and Technology Go Hand-in Hand, But Where are They Going?
Marketing and Technology Go Hand-in Hand, But Where are They Going? Mogul Marketing
 
Use open source and rapid prototyping to put magic in magical products in IoT
Use open source and rapid prototyping to put magic in magical products in IoTUse open source and rapid prototyping to put magic in magical products in IoT
Use open source and rapid prototyping to put magic in magical products in IoTMoe Tanabian
 
Memory Management in Android
Memory Management in AndroidMemory Management in Android
Memory Management in AndroidOpersys inc.
 
The Performance Marketer's Guide to Scaling Growth with Mobile App Install Ads
The Performance Marketer's Guide to Scaling Growth with Mobile App Install AdsThe Performance Marketer's Guide to Scaling Growth with Mobile App Install Ads
The Performance Marketer's Guide to Scaling Growth with Mobile App Install AdsNanigans
 
Simple Web Design Case Study (Website Design Process Walkthrough)
Simple Web Design Case Study (Website Design Process Walkthrough)Simple Web Design Case Study (Website Design Process Walkthrough)
Simple Web Design Case Study (Website Design Process Walkthrough)Followbright
 
Scheduling in Android
Scheduling in AndroidScheduling in Android
Scheduling in AndroidOpersys inc.
 
Android Things Internals
Android Things InternalsAndroid Things Internals
Android Things InternalsOpersys inc.
 
SRE - drupal day aveiro 2016
SRE - drupal day aveiro 2016SRE - drupal day aveiro 2016
SRE - drupal day aveiro 2016Ricardo Amaro
 
Site Reliability Engineering Helps Google Conquer The World
Site Reliability Engineering Helps Google Conquer The WorldSite Reliability Engineering Helps Google Conquer The World
Site Reliability Engineering Helps Google Conquer The WorldVistara
 

Viewers also liked (12)

Google Go Overview
Google Go OverviewGoogle Go Overview
Google Go Overview
 
Mau 2015 scaling facebook at care.com
Mau 2015   scaling facebook at care.com Mau 2015   scaling facebook at care.com
Mau 2015 scaling facebook at care.com
 
Marketing and Technology Go Hand-in Hand, But Where are They Going?
Marketing and Technology Go Hand-in Hand, But Where are They Going? Marketing and Technology Go Hand-in Hand, But Where are They Going?
Marketing and Technology Go Hand-in Hand, But Where are They Going?
 
Use open source and rapid prototyping to put magic in magical products in IoT
Use open source and rapid prototyping to put magic in magical products in IoTUse open source and rapid prototyping to put magic in magical products in IoT
Use open source and rapid prototyping to put magic in magical products in IoT
 
Memory Management in Android
Memory Management in AndroidMemory Management in Android
Memory Management in Android
 
The Performance Marketer's Guide to Scaling Growth with Mobile App Install Ads
The Performance Marketer's Guide to Scaling Growth with Mobile App Install AdsThe Performance Marketer's Guide to Scaling Growth with Mobile App Install Ads
The Performance Marketer's Guide to Scaling Growth with Mobile App Install Ads
 
Simple Web Design Case Study (Website Design Process Walkthrough)
Simple Web Design Case Study (Website Design Process Walkthrough)Simple Web Design Case Study (Website Design Process Walkthrough)
Simple Web Design Case Study (Website Design Process Walkthrough)
 
Scheduling in Android
Scheduling in AndroidScheduling in Android
Scheduling in Android
 
Android Things Internals
Android Things InternalsAndroid Things Internals
Android Things Internals
 
SRE - drupal day aveiro 2016
SRE - drupal day aveiro 2016SRE - drupal day aveiro 2016
SRE - drupal day aveiro 2016
 
Site Reliability Engineering Helps Google Conquer The World
Site Reliability Engineering Helps Google Conquer The WorldSite Reliability Engineering Helps Google Conquer The World
Site Reliability Engineering Helps Google Conquer The World
 
Die Android Plattform
Die Android PlattformDie Android Plattform
Die Android Plattform
 

Similar to Facebook Scaling Overview

Architecture Patterns - Open Discussion
Architecture Patterns - Open DiscussionArchitecture Patterns - Open Discussion
Architecture Patterns - Open DiscussionNguyen Tung
 
Single Page Applications: Your Browser is the OS!
Single Page Applications: Your Browser is the OS!Single Page Applications: Your Browser is the OS!
Single Page Applications: Your Browser is the OS!Jeremy Likness
 
Facebook[The Nuts and Bolts Technology]
Facebook[The Nuts and Bolts Technology]Facebook[The Nuts and Bolts Technology]
Facebook[The Nuts and Bolts Technology]Koushik Reddy
 
10 Things You Can Do to Speed Up Your Web App Today
10 Things You Can Do to Speed Up Your Web App Today10 Things You Can Do to Speed Up Your Web App Today
10 Things You Can Do to Speed Up Your Web App TodayChris Love
 
Building high performance and scalable share point applications
Building high performance and scalable share point applicationsBuilding high performance and scalable share point applications
Building high performance and scalable share point applicationsTalbott Crowell
 
UK Community day 20180427 Microsoft Flow hackathon
UK Community day 20180427 Microsoft Flow hackathonUK Community day 20180427 Microsoft Flow hackathon
UK Community day 20180427 Microsoft Flow hackathonPenny Coventry
 
How Facebook actually works????
How Facebook actually works????How Facebook actually works????
How Facebook actually works????Dhruv Patel
 
EscConf - Deep Dive Frontend Optimization
EscConf - Deep Dive Frontend OptimizationEscConf - Deep Dive Frontend Optimization
EscConf - Deep Dive Frontend OptimizationJonathan Klein
 
OVERVIEW OF FACEBOOK SCALABLE ARCHITECTURE.
OVERVIEW  OF FACEBOOK SCALABLE ARCHITECTURE.OVERVIEW  OF FACEBOOK SCALABLE ARCHITECTURE.
OVERVIEW OF FACEBOOK SCALABLE ARCHITECTURE.Rishikese MR
 
Modernize Solutions with SharePoint & the Power Platform
Modernize Solutions with SharePoint & the Power PlatformModernize Solutions with SharePoint & the Power Platform
Modernize Solutions with SharePoint & the Power PlatformJonathan Schultz
 
Building data intensive applications
Building data intensive applicationsBuilding data intensive applications
Building data intensive applicationsAmit Kejriwal
 
Cvcc performance tuning
Cvcc performance tuningCvcc performance tuning
Cvcc performance tuningJohn McCaffrey
 
Static Site Generators - Developing Websites in Low-resource Condition
Static Site Generators - Developing Websites in Low-resource ConditionStatic Site Generators - Developing Websites in Low-resource Condition
Static Site Generators - Developing Websites in Low-resource ConditionIWMW
 
What ya gonna do?
What ya gonna do?What ya gonna do?
What ya gonna do?CQD
 
facebook architecture for 600M users
facebook architecture for 600M usersfacebook architecture for 600M users
facebook architecture for 600M usersJongyoon Choi
 
Four Practices to Fix Your Top .NET Performance Problems
Four Practices to Fix Your Top .NET Performance ProblemsFour Practices to Fix Your Top .NET Performance Problems
Four Practices to Fix Your Top .NET Performance ProblemsAndreas Grabner
 
BaaS Comparison - iOS.mn
BaaS Comparison - iOS.mnBaaS Comparison - iOS.mn
BaaS Comparison - iOS.mnpyro2927
 

Similar to Facebook Scaling Overview (20)

Architecture Patterns - Open Discussion
Architecture Patterns - Open DiscussionArchitecture Patterns - Open Discussion
Architecture Patterns - Open Discussion
 
Single Page Applications: Your Browser is the OS!
Single Page Applications: Your Browser is the OS!Single Page Applications: Your Browser is the OS!
Single Page Applications: Your Browser is the OS!
 
Facebook[The Nuts and Bolts Technology]
Facebook[The Nuts and Bolts Technology]Facebook[The Nuts and Bolts Technology]
Facebook[The Nuts and Bolts Technology]
 
10 Things You Can Do to Speed Up Your Web App Today
10 Things You Can Do to Speed Up Your Web App Today10 Things You Can Do to Speed Up Your Web App Today
10 Things You Can Do to Speed Up Your Web App Today
 
Fb mechanism
Fb mechanismFb mechanism
Fb mechanism
 
Building high performance and scalable share point applications
Building high performance and scalable share point applicationsBuilding high performance and scalable share point applications
Building high performance and scalable share point applications
 
Build social apps for Facebook
Build social apps for FacebookBuild social apps for Facebook
Build social apps for Facebook
 
UK Community day 20180427 Microsoft Flow hackathon
UK Community day 20180427 Microsoft Flow hackathonUK Community day 20180427 Microsoft Flow hackathon
UK Community day 20180427 Microsoft Flow hackathon
 
How Facebook actually works????
How Facebook actually works????How Facebook actually works????
How Facebook actually works????
 
EscConf - Deep Dive Frontend Optimization
EscConf - Deep Dive Frontend OptimizationEscConf - Deep Dive Frontend Optimization
EscConf - Deep Dive Frontend Optimization
 
OVERVIEW OF FACEBOOK SCALABLE ARCHITECTURE.
OVERVIEW  OF FACEBOOK SCALABLE ARCHITECTURE.OVERVIEW  OF FACEBOOK SCALABLE ARCHITECTURE.
OVERVIEW OF FACEBOOK SCALABLE ARCHITECTURE.
 
Modernize Solutions with SharePoint & the Power Platform
Modernize Solutions with SharePoint & the Power PlatformModernize Solutions with SharePoint & the Power Platform
Modernize Solutions with SharePoint & the Power Platform
 
Building data intensive applications
Building data intensive applicationsBuilding data intensive applications
Building data intensive applications
 
Cvcc performance tuning
Cvcc performance tuningCvcc performance tuning
Cvcc performance tuning
 
Static Site Generators - Developing Websites in Low-resource Condition
Static Site Generators - Developing Websites in Low-resource ConditionStatic Site Generators - Developing Websites in Low-resource Condition
Static Site Generators - Developing Websites in Low-resource Condition
 
What ya gonna do?
What ya gonna do?What ya gonna do?
What ya gonna do?
 
facebook architecture for 600M users
facebook architecture for 600M usersfacebook architecture for 600M users
facebook architecture for 600M users
 
Four Practices to Fix Your Top .NET Performance Problems
Four Practices to Fix Your Top .NET Performance ProblemsFour Practices to Fix Your Top .NET Performance Problems
Four Practices to Fix Your Top .NET Performance Problems
 
BaaS Comparison - iOS.mn
BaaS Comparison - iOS.mnBaaS Comparison - iOS.mn
BaaS Comparison - iOS.mn
 
Frontender in-2016
Frontender in-2016Frontender in-2016
Frontender in-2016
 

Recently uploaded

Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Fact vs. Fiction: Autodetecting Hallucinations in LLMs
Fact vs. Fiction: Autodetecting Hallucinations in LLMsFact vs. Fiction: Autodetecting Hallucinations in LLMs
Fact vs. Fiction: Autodetecting Hallucinations in LLMsZilliz
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dashnarutouzumaki53779
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Scott Andery
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 

Recently uploaded (20)

Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Fact vs. Fiction: Autodetecting Hallucinations in LLMs
Fact vs. Fiction: Autodetecting Hallucinations in LLMsFact vs. Fiction: Autodetecting Hallucinations in LLMs
Fact vs. Fiction: Autodetecting Hallucinations in LLMs
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dash
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 

Facebook Scaling Overview

  • 1. Facebook Scaling Walkthrough Moritz Haarmann - Ultrasuperlargescale Systems
  • 3. 800.000.000 active users • > 50% log on every given day • 250.000.000 Photos every single day ( Flickr: 6Bn total ) • 30.000.000.000 new pieces of content monthly
  • 5. 30.000.000.000 As if everyone now living on earth posts 4.28 updates every month.
  • 7. Facebook is built using • Web Servers ( Running Hip-Hop PHP ) • Services ( Search, Ads ) • Memcached & MySQL • Immense amounts of glue
  • 8. Write Strategy • Writes take place centrally in California • 3.5 Million Changed Rows per Second ( Peak ) • 2010: 1800 DB Servers • horizontal scaling approach not disclosed • consistency is important ( avoiding „unhappy users“ )
  • 9. Glue • Massively distributed architecture • Glue keeping it together • Many systems built in-house to meet giiaaanoourmus requirements
  • 10. Haystack • Photos • Handles everything from HTTP to storage • Aimed at minimizing IO-Operations • Append-Only!
  • 11. Memcached • Placed between MySQL and Web Tier • Stores only „plain data“, no joins or other complicated stuff • Faster if Web Server works on data
  • 12. BigPipe • Assembles the output pages • everything that is needed retrieved in parallel • Fault tolerant, will work even if parts of a page are not available
  • 13.
  • 15. Live Profiling • Facebook monitors their life systems continously at a PHP-Method level ( using XHProf ).
  • 16. Graceful Degradation • High awareness ( Monitoring ) of perfomance problems • Features can be disabled ( very ne-grained ) to keep the core features running smoothly
  • 17. Keeping it running • New features are launched ,dark‘, without visible elements, to stress test the backend with real load • Incremental roll-outs decrease the impact of bug or malfunction
  • 18. Open Source • Most parts are open source • Either used or created and then os‘d
  • 19. Big Bang • On September 23, 2010, Facebook was down for most users for about 3 hours • A wrongly identi ed ,invalid‘ cache value lead to requests hammering the DB tier • A system designed to prevent failures created one! • Only way to recover was to completely shut down access to the DB - downtime https://www.facebook.com/note.php?note_id=431441338919&id=9445547199&ref=mf ( great comments, too )
  • 21. Sources • http://royal.pingdom.com/2010/06/18/the-software-behind- facebook/ • https://www.facebook.com/note.php?note_id=76191543919 • https://www.facebook.com/notes/facebook-engineering/bigpipe- pipelining-web-pages-for-high-performance/389414033919 • http://blog.kissmetrics.com/facebook-statistics/