SlideShare a Scribd company logo
1 of 38
Performance metrics
for a social network
About Me

•   Thierry Schellenbach
•   Founder/ CTO Fashiolista
•   Author of Django Facebook
•   Github/tschellenbach

• Blog: mellowmorning.com
• @tschellenbach
Global Fashion Discovery
5.000.000+
8.000.000+
Growth

2nd largest fashion community

• 1mln members
• 17mln loves/month
• 94mln non-bot pageviews
Powered By

•   Django/Python
•   PostgreSQL
•   Solr
•   Redis
•   Celery
•   AWS/ Ubuntu
•   Nginx/ Gunicorn/ Supervisor
Sexy Metrics driven optimization
Hard Because

• All content is personalized

• Activity is clustered around a few
  users (>100k followers)

• Individual users are insanely
  active (7 hours in a day is
  normal)

• Social network, can’t easily shard
  data
Speed is a Feature
Metrics across the board
• Development
  – Spot things early on, wrong usage of ORM etc
• System Health
  – Is my DB healthy, my Redis cluster etc
• Page level
  – Why is my page slow
  – What is the average speed of the components
    (DB, Redis, Solr etc)
Tools we use
  Development               System Health           Page Level

                        •    Cloudwatch          • New Relic
• Debug toolbar
                        •    Munin
   – Cache calls                                 • Graphite
                        •    Nagios
   – Graphite
     Timings            •    DB slow log
   – Queries and        •    Redis slow log
     their explains     •    Integration Tests
   – Duplicate query    •    PgFouine
     detection
Development


                                   StatsD

              Duplicates




                           Cache Calls
Today’s Presentation
    New Relic            Graphite           PgFouine

• Dashboard, High   • Stash all data,    • Understand
  level insights      query it any way     what keeps
                      you want             your DB busy
                    • Tool, not a
                      dashboard
New Relic

• Frontend -> App ->
  Components (DB, Solr, etc.)
• Breaks page performance
  down into it’s components
• Tracks deploys and compares
  before and after
Are you Supported?

•   Ruby              •   Pip install newrelic
•   Java              •   Edit the .ini
•   .NET              •   Add the WSGI middleware
•   PHP               •   Wait for Magic
•   Python
End user load times
• Drill down all the way to Database calls
• The purple line is our app, the rest frontend
                                  Frontend
                                   (97%)




                                                  App
Global page loads
Page Level
• Average frontend performance per page
• Click to view App level breakdown

                 Page. Not URL.           To App Level
Drill down/ App overview

 History




                          Memcached
               DB Query
Database
        • See which tables are
          under most load
        • See which pages cause the
          load




• Development over time
Deploys
Deploys part Two
Response Time Pre & Post




                           Memory Utilization
Background Task

Number of Task
calls (sample)
Graphite Insights
• NewRelic has the overview,
  Graphite the detail
• Open Source!

• Throw data at it via UDP

• Popularized by Etsy
  (see mellowmorning.com for
  link)
It’s Complicated
Tracks Everything
Setup
• Track using StatsD
   – Support for (PHP, Python, Ruby, Node, Java)

• Hierarchy (python example)
• get.<app>.<view>.<component>
  with request.timings('get.user.profile_page.sql'):
     print ‘database query here’
Data tool/ Not a dashboard

• Wildcards
  – get.<app>.<view>.*.upper_90

  – get.<app>.*.redis.zadd.upper_90

  – limit(sortByMaxima(get.<app>.<view>.*.up
    per_90),4)
/style/<user>/ performance



                   Memcached
                   Slowdown




            ZADD


                       Set Many
Including Functional parts of Pages

• More like this part is
  tracked

• Solr & Redis Cache
What we Track

•   Loadtime per bit of functionality
•   Database calls per DB
•   90th percentile load times
•   Task broker roundtrip times
•   Facebook API calls
PgFouine
• Run on samples of all queries (say 5m)
• Not just slow queries
• Repeating a simple query many times is also
  wrong, PgFouine finds it

• See Instagram’s fabric snippet
• https://gist.github.com/2307647
PgFouine Continued
Queries that took up
the most time (N)

• Spots issues with
  many small queries                Normalized

Compare multiple
reports
PgFouine Tips

• My colleague wrote a fast C++ version
• github.com/WoLpH/pg_query_analyser

  Also look at:
• Pg Stat Statement
• Pg Badger
Concluding
    New Relic            Graphite           PgFouine

• Dashboard, High   • Stash all data,    • Understand
  level insights      query it any way     what keeps
                      you want             your DB busy
                    • Tool, not a
                      dashboard
Q&A

We’re Searching for Django Developers & Linux
system administrators!

              Fashiolista.com/jobs

Open source projects:

            Github.com/tschellenbach
               Try Django Facebook!

More Related Content

What's hot

How to prepare your site for content migration
How to prepare your site for content migrationHow to prepare your site for content migration
How to prepare your site for content migrationBlend Interactive
 
SQL Server In-Memory OLTP Migration Overview
SQL Server In-Memory OLTP Migration OverviewSQL Server In-Memory OLTP Migration Overview
SQL Server In-Memory OLTP Migration OverviewJose Rivera Miranda
 
Delivering changes for applications and databases
Delivering changes for applications and databasesDelivering changes for applications and databases
Delivering changes for applications and databasesEduardo Piairo
 
Managing Descriptive Metadata with Open XML...For Now
Managing Descriptive Metadata with Open XML...For NowManaging Descriptive Metadata with Open XML...For Now
Managing Descriptive Metadata with Open XML...For NowGregory Wiedeman
 
Drupal Site Audit - SFDUG
Drupal Site Audit - SFDUGDrupal Site Audit - SFDUG
Drupal Site Audit - SFDUGJon Peck
 
Untangling - fall2017 - week 9
Untangling - fall2017 - week 9Untangling - fall2017 - week 9
Untangling - fall2017 - week 9Derek Jacoby
 
Architecture Patterns - Open Discussion
Architecture Patterns - Open DiscussionArchitecture Patterns - Open Discussion
Architecture Patterns - Open DiscussionNguyen Tung
 
SenchaCon 2016: Oracle Forms Modernisation - Owen Pagan
SenchaCon 2016: Oracle Forms Modernisation - Owen PaganSenchaCon 2016: Oracle Forms Modernisation - Owen Pagan
SenchaCon 2016: Oracle Forms Modernisation - Owen PaganSencha
 
7 Stages of Scaling Web Applications
7 Stages of Scaling Web Applications7 Stages of Scaling Web Applications
7 Stages of Scaling Web ApplicationsDavid Mitzenmacher
 
Plone pas.plugins.ldap user/group search
Plone pas.plugins.ldap user/group searchPlone pas.plugins.ldap user/group search
Plone pas.plugins.ldap user/group searchfredvd
 
Make Drupal Run Fast - increase page load speed
Make Drupal Run Fast - increase page load speedMake Drupal Run Fast - increase page load speed
Make Drupal Run Fast - increase page load speedAndy Kucharski
 
Road to database automation: database source control
Road to database automation: database source controlRoad to database automation: database source control
Road to database automation: database source controlEduardo Piairo
 
Lessons from Highly Scalable Architectures at Social Networking Sites
Lessons from Highly Scalable Architectures at Social Networking SitesLessons from Highly Scalable Architectures at Social Networking Sites
Lessons from Highly Scalable Architectures at Social Networking SitesPatrick Senti
 
Road to database automation - Database source control
Road to database automation - Database source controlRoad to database automation - Database source control
Road to database automation - Database source controlEduardo Piairo
 
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...Jon Peck
 
Engage 2019: Modernising Your Domino and XPages Applications
Engage 2019: Modernising Your Domino and XPages Applications Engage 2019: Modernising Your Domino and XPages Applications
Engage 2019: Modernising Your Domino and XPages Applications Paul Withers
 

What's hot (20)

How to prepare your site for content migration
How to prepare your site for content migrationHow to prepare your site for content migration
How to prepare your site for content migration
 
SQL Server In-Memory OLTP Migration Overview
SQL Server In-Memory OLTP Migration OverviewSQL Server In-Memory OLTP Migration Overview
SQL Server In-Memory OLTP Migration Overview
 
Azkaban
AzkabanAzkaban
Azkaban
 
Delivering changes for applications and databases
Delivering changes for applications and databasesDelivering changes for applications and databases
Delivering changes for applications and databases
 
Resolving problems & high availability
Resolving problems & high availabilityResolving problems & high availability
Resolving problems & high availability
 
Managing Descriptive Metadata with Open XML...For Now
Managing Descriptive Metadata with Open XML...For NowManaging Descriptive Metadata with Open XML...For Now
Managing Descriptive Metadata with Open XML...For Now
 
Drupal Site Audit - SFDUG
Drupal Site Audit - SFDUGDrupal Site Audit - SFDUG
Drupal Site Audit - SFDUG
 
Untangling - fall2017 - week 9
Untangling - fall2017 - week 9Untangling - fall2017 - week 9
Untangling - fall2017 - week 9
 
Architecture Patterns - Open Discussion
Architecture Patterns - Open DiscussionArchitecture Patterns - Open Discussion
Architecture Patterns - Open Discussion
 
WebDev Crash Course
WebDev Crash CourseWebDev Crash Course
WebDev Crash Course
 
SenchaCon 2016: Oracle Forms Modernisation - Owen Pagan
SenchaCon 2016: Oracle Forms Modernisation - Owen PaganSenchaCon 2016: Oracle Forms Modernisation - Owen Pagan
SenchaCon 2016: Oracle Forms Modernisation - Owen Pagan
 
7 Stages of Scaling Web Applications
7 Stages of Scaling Web Applications7 Stages of Scaling Web Applications
7 Stages of Scaling Web Applications
 
Plone pas.plugins.ldap user/group search
Plone pas.plugins.ldap user/group searchPlone pas.plugins.ldap user/group search
Plone pas.plugins.ldap user/group search
 
Make Drupal Run Fast - increase page load speed
Make Drupal Run Fast - increase page load speedMake Drupal Run Fast - increase page load speed
Make Drupal Run Fast - increase page load speed
 
Road to database automation: database source control
Road to database automation: database source controlRoad to database automation: database source control
Road to database automation: database source control
 
Lessons from Highly Scalable Architectures at Social Networking Sites
Lessons from Highly Scalable Architectures at Social Networking SitesLessons from Highly Scalable Architectures at Social Networking Sites
Lessons from Highly Scalable Architectures at Social Networking Sites
 
Road to database automation - Database source control
Road to database automation - Database source controlRoad to database automation - Database source control
Road to database automation - Database source control
 
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...
 
Engage 2019: Modernising Your Domino and XPages Applications
Engage 2019: Modernising Your Domino and XPages Applications Engage 2019: Modernising Your Domino and XPages Applications
Engage 2019: Modernising Your Domino and XPages Applications
 
Qcon
QconQcon
Qcon
 

Similar to Performance metrics for a social network

My Site is slow - Drupal Camp London 2013
My Site is slow - Drupal Camp London 2013My Site is slow - Drupal Camp London 2013
My Site is slow - Drupal Camp London 2013hernanibf
 
Profiling and Tuning a Web Application - The Dirty Details
Profiling and Tuning a Web Application - The Dirty DetailsProfiling and Tuning a Web Application - The Dirty Details
Profiling and Tuning a Web Application - The Dirty DetailsAchievers Tech
 
My site is slow
My site is slowMy site is slow
My site is slowhernanibf
 
Ship It ! with Ruby/ Rails Ecosystem
Ship It ! with Ruby/ Rails EcosystemShip It ! with Ruby/ Rails Ecosystem
Ship It ! with Ruby/ Rails EcosystemYi-Ting Cheng
 
Tuenti Release Workflow
Tuenti Release WorkflowTuenti Release Workflow
Tuenti Release WorkflowTuenti
 
Does Your Stuff Scale?
Does Your Stuff Scale?Does Your Stuff Scale?
Does Your Stuff Scale?stevenh0lmes
 
Picnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable applicationPicnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable applicationNick Josevski
 
USG Summit - September 2014 - Web Management using Drupal
USG Summit - September 2014 - Web Management using DrupalUSG Summit - September 2014 - Web Management using Drupal
USG Summit - September 2014 - Web Management using DrupalEric Sembrat
 
How to Do a Performance Audit of Your .NET Website
How to Do a Performance Audit of Your .NET WebsiteHow to Do a Performance Audit of Your .NET Website
How to Do a Performance Audit of Your .NET WebsiteDNN
 
Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)
Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)
Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)Sascha Wenninger
 
From Pilot to Product - Morning@Lohika
From Pilot to Product - Morning@LohikaFrom Pilot to Product - Morning@Lohika
From Pilot to Product - Morning@LohikaIvan Verhun
 
Drupalmonitor.com - Drupal User Group Meetup Basel
Drupalmonitor.com - Drupal User Group Meetup BaselDrupalmonitor.com - Drupal User Group Meetup Basel
Drupalmonitor.com - Drupal User Group Meetup BaselNETNODE AG
 
Front End page speed performance improvements for Drupal
Front End page speed performance improvements for DrupalFront End page speed performance improvements for Drupal
Front End page speed performance improvements for DrupalAndy Kucharski
 
5 Common Mistakes You are Making on your Website
 5 Common Mistakes You are Making on your Website 5 Common Mistakes You are Making on your Website
5 Common Mistakes You are Making on your WebsiteAcquia
 
Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...
Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...
Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...Jon Peck
 
Tuenti Release Workflow v1.1
Tuenti Release Workflow v1.1Tuenti Release Workflow v1.1
Tuenti Release Workflow v1.1Tuenti
 
Make It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version ControlMake It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version Controlindiver
 

Similar to Performance metrics for a social network (20)

My Site is slow - Drupal Camp London 2013
My Site is slow - Drupal Camp London 2013My Site is slow - Drupal Camp London 2013
My Site is slow - Drupal Camp London 2013
 
Profiling and Tuning a Web Application - The Dirty Details
Profiling and Tuning a Web Application - The Dirty DetailsProfiling and Tuning a Web Application - The Dirty Details
Profiling and Tuning a Web Application - The Dirty Details
 
My site is slow
My site is slowMy site is slow
My site is slow
 
Ship It ! with Ruby/ Rails Ecosystem
Ship It ! with Ruby/ Rails EcosystemShip It ! with Ruby/ Rails Ecosystem
Ship It ! with Ruby/ Rails Ecosystem
 
Tuenti Release Workflow
Tuenti Release WorkflowTuenti Release Workflow
Tuenti Release Workflow
 
Does Your Stuff Scale?
Does Your Stuff Scale?Does Your Stuff Scale?
Does Your Stuff Scale?
 
Picnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable applicationPicnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable application
 
USG Summit - September 2014 - Web Management using Drupal
USG Summit - September 2014 - Web Management using DrupalUSG Summit - September 2014 - Web Management using Drupal
USG Summit - September 2014 - Web Management using Drupal
 
How to Do a Performance Audit of Your .NET Website
How to Do a Performance Audit of Your .NET WebsiteHow to Do a Performance Audit of Your .NET Website
How to Do a Performance Audit of Your .NET Website
 
Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)
Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)
Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)
 
From Pilot to Product - Morning@Lohika
From Pilot to Product - Morning@LohikaFrom Pilot to Product - Morning@Lohika
From Pilot to Product - Morning@Lohika
 
Redundant devops
Redundant devopsRedundant devops
Redundant devops
 
Drupalmonitor.com - Drupal User Group Meetup Basel
Drupalmonitor.com - Drupal User Group Meetup BaselDrupalmonitor.com - Drupal User Group Meetup Basel
Drupalmonitor.com - Drupal User Group Meetup Basel
 
Front End page speed performance improvements for Drupal
Front End page speed performance improvements for DrupalFront End page speed performance improvements for Drupal
Front End page speed performance improvements for Drupal
 
5 Common Mistakes You are Making on your Website
 5 Common Mistakes You are Making on your Website 5 Common Mistakes You are Making on your Website
5 Common Mistakes You are Making on your Website
 
Key alias dev standard final
Key alias   dev standard finalKey alias   dev standard final
Key alias dev standard final
 
Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...
Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...
Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...
 
Tuenti Release Workflow v1.1
Tuenti Release Workflow v1.1Tuenti Release Workflow v1.1
Tuenti Release Workflow v1.1
 
Be faster then rabbits
Be faster then rabbitsBe faster then rabbits
Be faster then rabbits
 
Make It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version ControlMake It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version Control
 

More from Thierry Schellenbach

13 tips for a highly engaging feed
13 tips for a highly engaging feed13 tips for a highly engaging feed
13 tips for a highly engaging feedThierry Schellenbach
 
AWS Summit Amsterdam - Thierry Schellenbach Founder/ Fashiolista
AWS Summit Amsterdam - Thierry Schellenbach Founder/ FashiolistaAWS Summit Amsterdam - Thierry Schellenbach Founder/ Fashiolista
AWS Summit Amsterdam - Thierry Schellenbach Founder/ FashiolistaThierry Schellenbach
 

More from Thierry Schellenbach (6)

Scalable News Feed with Mongo DB
Scalable News Feed with Mongo DBScalable News Feed with Mongo DB
Scalable News Feed with Mongo DB
 
13 tips for a highly engaging feed
13 tips for a highly engaging feed13 tips for a highly engaging feed
13 tips for a highly engaging feed
 
AWS Summit Amsterdam - Thierry Schellenbach Founder/ Fashiolista
AWS Summit Amsterdam - Thierry Schellenbach Founder/ FashiolistaAWS Summit Amsterdam - Thierry Schellenbach Founder/ Fashiolista
AWS Summit Amsterdam - Thierry Schellenbach Founder/ Fashiolista
 
Feedly & Cassandra at Fashiolista
Feedly & Cassandra at FashiolistaFeedly & Cassandra at Fashiolista
Feedly & Cassandra at Fashiolista
 
Fashiolista Notifications
Fashiolista NotificationsFashiolista Notifications
Fashiolista Notifications
 
Just a millisecond_
Just a millisecond_Just a millisecond_
Just a millisecond_
 

Recently uploaded

Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationKnoldus Inc.
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 

Recently uploaded (20)

Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 

Performance metrics for a social network

  • 1. Performance metrics for a social network
  • 2. About Me • Thierry Schellenbach • Founder/ CTO Fashiolista • Author of Django Facebook • Github/tschellenbach • Blog: mellowmorning.com • @tschellenbach
  • 4.
  • 5.
  • 6.
  • 8. Growth 2nd largest fashion community • 1mln members • 17mln loves/month • 94mln non-bot pageviews
  • 9. Powered By • Django/Python • PostgreSQL • Solr • Redis • Celery • AWS/ Ubuntu • Nginx/ Gunicorn/ Supervisor
  • 10. Sexy Metrics driven optimization Hard Because • All content is personalized • Activity is clustered around a few users (>100k followers) • Individual users are insanely active (7 hours in a day is normal) • Social network, can’t easily shard data
  • 11. Speed is a Feature
  • 12. Metrics across the board • Development – Spot things early on, wrong usage of ORM etc • System Health – Is my DB healthy, my Redis cluster etc • Page level – Why is my page slow – What is the average speed of the components (DB, Redis, Solr etc)
  • 13. Tools we use Development System Health Page Level • Cloudwatch • New Relic • Debug toolbar • Munin – Cache calls • Graphite • Nagios – Graphite Timings • DB slow log – Queries and • Redis slow log their explains • Integration Tests – Duplicate query • PgFouine detection
  • 14. Development StatsD Duplicates Cache Calls
  • 15. Today’s Presentation New Relic Graphite PgFouine • Dashboard, High • Stash all data, • Understand level insights query it any way what keeps you want your DB busy • Tool, not a dashboard
  • 16. New Relic • Frontend -> App -> Components (DB, Solr, etc.) • Breaks page performance down into it’s components • Tracks deploys and compares before and after
  • 17. Are you Supported? • Ruby • Pip install newrelic • Java • Edit the .ini • .NET • Add the WSGI middleware • PHP • Wait for Magic • Python
  • 18. End user load times • Drill down all the way to Database calls • The purple line is our app, the rest frontend Frontend (97%) App
  • 20. Page Level • Average frontend performance per page • Click to view App level breakdown Page. Not URL. To App Level
  • 21. Drill down/ App overview History Memcached DB Query
  • 22. Database • See which tables are under most load • See which pages cause the load • Development over time
  • 24. Deploys part Two Response Time Pre & Post Memory Utilization
  • 25. Background Task Number of Task calls (sample)
  • 26. Graphite Insights • NewRelic has the overview, Graphite the detail • Open Source! • Throw data at it via UDP • Popularized by Etsy (see mellowmorning.com for link)
  • 29. Setup • Track using StatsD – Support for (PHP, Python, Ruby, Node, Java) • Hierarchy (python example) • get.<app>.<view>.<component> with request.timings('get.user.profile_page.sql'): print ‘database query here’
  • 30. Data tool/ Not a dashboard • Wildcards – get.<app>.<view>.*.upper_90 – get.<app>.*.redis.zadd.upper_90 – limit(sortByMaxima(get.<app>.<view>.*.up per_90),4)
  • 31. /style/<user>/ performance Memcached Slowdown ZADD Set Many
  • 32. Including Functional parts of Pages • More like this part is tracked • Solr & Redis Cache
  • 33. What we Track • Loadtime per bit of functionality • Database calls per DB • 90th percentile load times • Task broker roundtrip times • Facebook API calls
  • 34. PgFouine • Run on samples of all queries (say 5m) • Not just slow queries • Repeating a simple query many times is also wrong, PgFouine finds it • See Instagram’s fabric snippet • https://gist.github.com/2307647
  • 35. PgFouine Continued Queries that took up the most time (N) • Spots issues with many small queries Normalized Compare multiple reports
  • 36. PgFouine Tips • My colleague wrote a fast C++ version • github.com/WoLpH/pg_query_analyser Also look at: • Pg Stat Statement • Pg Badger
  • 37. Concluding New Relic Graphite PgFouine • Dashboard, High • Stash all data, • Understand level insights query it any way what keeps you want your DB busy • Tool, not a dashboard
  • 38. Q&A We’re Searching for Django Developers & Linux system administrators! Fashiolista.com/jobs Open source projects: Github.com/tschellenbach Try Django Facebook!

Editor's Notes

  1. Fashiolista
  2. Users of Fashiolista install the so called “love button”. While browsing around the web they can use this button to add their favourite fashion finds to Fashiolista.
  3. Once they click the button, we figure out the relevant image on the page and allow you to add it to your profile.
  4. The find is added to your profile and other people can follow the items you love.
  5. Over the past 2 years thing have moved along rapidly.Currently we’re the second largest fashion community worldwide.With close to 1mln members, and massive monthly engagement.So, a quick check.Who In this room is a member of Fashiolista
  6. So, let’s focus on the tech side of things.Powered by
  7. On to the topic of this talkTracking the right things to optimize your web application.Now, optimizing a social network is hard.I won’t go into the techniques we use at Fashiolista for speeding up the application, today we’ll focus on the metrics enabling us to focus on the bottlenecks.
  8. Why does it matter?We’ve consistently seen massive growth in pageviews after speeding up the site.You can often add 20% to the number of pageviews, just by making the site faster.Once you have an initial audience, making your product work well is really powerfull.
  9. We can divide most of our measurement tools into 3 categoriesdevelopment, system health, page level- Answering the specific questions
  10. Some of the tools we use.We care mainly about the page level.CPU on the database is interesting, but tells you more about when you’ll run into system limits.While working on optimizing you application you want to focus on the page level.How fast are the applications used to generate this page.Did we wait on the database, or was it solr.
  11. Small interlude, because I’m really happy with the Django debug toolbar.I used to work with Symonfy and Django borrowed this, but it’s awesome.Duplicate queriesStatsd Functional reportingCache calls
  12. A few tools are really slick and definitely deserve a little demo.NewRelicGraphitePgFouineExplain their use cases
  13. New Relic gives you the full drilldown. Starting from frontend to the app to the components.The apdex score tells you the percentage of users which had a good experience (page generated under 500ms).
  14. Useful to see how your CDN is doing it’s job.We’ve recently switched from Akamai to Cloudfront.Which seems to work quite well in most countries.
  15. At the page level you get a lot of cool information.You see the load times per component.- Such as the the time spent querying the db. (Entity_love table in this case)Or the time spent querying memcached (Which is quite substantial about 20ms)In addition you see the development over time.Which is great for spotting problems which are recently introduced.
  16. New Relic also offers database level components.Tables under most loadThe awesome bit, it relates this to pages cause the loadShow again the development over time. For fun try dropping an index and you’ll see it popping up immediately here.
  17. That nicely brings me to the deploy overview of newrelic.Every deploy gives you a nice change report.Showing what happened to the speed before and after your deploy.This allows you to quickly spot mistakes landing in production.
  18. Shows the average response time before and after your deploy.Also shows things like memory or CPU utilization which will pick major mistakes in those areas.
  19. Newrelic does pretty much the same thing for background tasks as for views.You can zoom into the specific components.If yourautoscaling suddenly boots up twice the number of task workers, NewRelic tells you why.
  20. So It’s clear I’m exited about New Relic. It’s an awesome tool and helped us a lot with scaling the site.However sometimes you have questions about your data which NewRelic can’t answer.We stick all the metrics we can think of in Graphite.Now NewRelic is really slick. Graphite is designed by engineers and, well looks like this.
  21. But it tracks everything you throw at it.And has a very powerful querying interface.Graphite is a data analysis tool. It’s not a dashboard.For instance in this case calls per database server.
  22. It’s however a data tool though. And not a dashboard.It has a really techy interface.You use * for wildcardsAnd call functions to run on your data.Quite ugly.
  23. You can also retrieve data similar to new relic load time breakdowns.With the added advantage that Graphite is a free tool.
  24. It also tracks functional parts of pages.So we see which part of the page is slowing down load times.
  25. We track things like loadtime per functionality.We track all database calls.We track 90th percentile loadtimes.Adding new measurements is super easy.
  26. Lastly we’re using PgFouineIt’s an awesome tool to get a complete understanding on what your database is actually doing.