SlideShare a Scribd company logo
Performance metrics
for a social network
About Me

•   Thierry Schellenbach
•   Founder/ CTO Fashiolista
•   Author of Django Facebook
•   Github/tschellenbach

• Blog: mellowmorning.com
• @tschellenbach
Global Fashion Discovery
Performance metrics for a social network
Performance metrics for a social network
Performance metrics for a social network
5.000.000+
8.000.000+
Growth

2nd largest fashion community

• 1mln members
• 17mln loves/month
• 94mln non-bot pageviews
Powered By

•   Django/Python
•   PostgreSQL
•   Solr
•   Redis
•   Celery
•   AWS/ Ubuntu
•   Nginx/ Gunicorn/ Supervisor
Sexy Metrics driven optimization
Hard Because

• All content is personalized

• Activity is clustered around a few
  users (>100k followers)

• Individual users are insanely
  active (7 hours in a day is
  normal)

• Social network, can’t easily shard
  data
Speed is a Feature
Metrics across the board
• Development
  – Spot things early on, wrong usage of ORM etc
• System Health
  – Is my DB healthy, my Redis cluster etc
• Page level
  – Why is my page slow
  – What is the average speed of the components
    (DB, Redis, Solr etc)
Tools we use
  Development               System Health           Page Level

                        •    Cloudwatch          • New Relic
• Debug toolbar
                        •    Munin
   – Cache calls                                 • Graphite
                        •    Nagios
   – Graphite
     Timings            •    DB slow log
   – Queries and        •    Redis slow log
     their explains     •    Integration Tests
   – Duplicate query    •    PgFouine
     detection
Development


                                   StatsD

              Duplicates




                           Cache Calls
Today’s Presentation
    New Relic            Graphite           PgFouine

• Dashboard, High   • Stash all data,    • Understand
  level insights      query it any way     what keeps
                      you want             your DB busy
                    • Tool, not a
                      dashboard
New Relic

• Frontend -> App ->
  Components (DB, Solr, etc.)
• Breaks page performance
  down into it’s components
• Tracks deploys and compares
  before and after
Are you Supported?

•   Ruby              •   Pip install newrelic
•   Java              •   Edit the .ini
•   .NET              •   Add the WSGI middleware
•   PHP               •   Wait for Magic
•   Python
End user load times
• Drill down all the way to Database calls
• The purple line is our app, the rest frontend
                                  Frontend
                                   (97%)




                                                  App
Global page loads
Page Level
• Average frontend performance per page
• Click to view App level breakdown

                 Page. Not URL.           To App Level
Drill down/ App overview

 History




                          Memcached
               DB Query
Database
        • See which tables are
          under most load
        • See which pages cause the
          load




• Development over time
Deploys
Deploys part Two
Response Time Pre & Post




                           Memory Utilization
Background Task

Number of Task
calls (sample)
Graphite Insights
• NewRelic has the overview,
  Graphite the detail
• Open Source!

• Throw data at it via UDP

• Popularized by Etsy
  (see mellowmorning.com for
  link)
It’s Complicated
Tracks Everything
Setup
• Track using StatsD
   – Support for (PHP, Python, Ruby, Node, Java)

• Hierarchy (python example)
• get.<app>.<view>.<component>
  with request.timings('get.user.profile_page.sql'):
     print ‘database query here’
Data tool/ Not a dashboard

• Wildcards
  – get.<app>.<view>.*.upper_90

  – get.<app>.*.redis.zadd.upper_90

  – limit(sortByMaxima(get.<app>.<view>.*.up
    per_90),4)
/style/<user>/ performance



                   Memcached
                   Slowdown




            ZADD


                       Set Many
Including Functional parts of Pages

• More like this part is
  tracked

• Solr & Redis Cache
What we Track

•   Loadtime per bit of functionality
•   Database calls per DB
•   90th percentile load times
•   Task broker roundtrip times
•   Facebook API calls
PgFouine
• Run on samples of all queries (say 5m)
• Not just slow queries
• Repeating a simple query many times is also
  wrong, PgFouine finds it

• See Instagram’s fabric snippet
• https://gist.github.com/2307647
PgFouine Continued
Queries that took up
the most time (N)

• Spots issues with
  many small queries                Normalized

Compare multiple
reports
PgFouine Tips

• My colleague wrote a fast C++ version
• github.com/WoLpH/pg_query_analyser

  Also look at:
• Pg Stat Statement
• Pg Badger
Concluding
    New Relic            Graphite           PgFouine

• Dashboard, High   • Stash all data,    • Understand
  level insights      query it any way     what keeps
                      you want             your DB busy
                    • Tool, not a
                      dashboard
Q&A

We’re Searching for Django Developers & Linux
system administrators!

              Fashiolista.com/jobs

Open source projects:

            Github.com/tschellenbach
               Try Django Facebook!

More Related Content

What's hot

How to prepare your site for content migration
How to prepare your site for content migrationHow to prepare your site for content migration
How to prepare your site for content migration
Blend Interactive
 
SQL Server In-Memory OLTP Migration Overview
SQL Server In-Memory OLTP Migration OverviewSQL Server In-Memory OLTP Migration Overview
SQL Server In-Memory OLTP Migration Overview
Jose Rivera Miranda
 
Azkaban
AzkabanAzkaban
Delivering changes for applications and databases
Delivering changes for applications and databasesDelivering changes for applications and databases
Delivering changes for applications and databases
Eduardo Piairo
 
Resolving problems & high availability
Resolving problems & high availabilityResolving problems & high availability
Resolving problems & high availability
Zend by Rogue Wave Software
 
Managing Descriptive Metadata with Open XML...For Now
Managing Descriptive Metadata with Open XML...For NowManaging Descriptive Metadata with Open XML...For Now
Managing Descriptive Metadata with Open XML...For Now
Gregory Wiedeman
 
Drupal Site Audit - SFDUG
Drupal Site Audit - SFDUGDrupal Site Audit - SFDUG
Drupal Site Audit - SFDUG
Jon Peck
 
Untangling - fall2017 - week 9
Untangling - fall2017 - week 9Untangling - fall2017 - week 9
Untangling - fall2017 - week 9
Derek Jacoby
 
Architecture Patterns - Open Discussion
Architecture Patterns - Open DiscussionArchitecture Patterns - Open Discussion
Architecture Patterns - Open Discussion
Nguyen Tung
 
WebDev Crash Course
WebDev Crash CourseWebDev Crash Course
WebDev Crash Course
Cesar Martinez
 
SenchaCon 2016: Oracle Forms Modernisation - Owen Pagan
SenchaCon 2016: Oracle Forms Modernisation - Owen PaganSenchaCon 2016: Oracle Forms Modernisation - Owen Pagan
SenchaCon 2016: Oracle Forms Modernisation - Owen Pagan
Sencha
 
7 Stages of Scaling Web Applications
7 Stages of Scaling Web Applications7 Stages of Scaling Web Applications
7 Stages of Scaling Web Applications
David Mitzenmacher
 
Plone pas.plugins.ldap user/group search
Plone pas.plugins.ldap user/group searchPlone pas.plugins.ldap user/group search
Plone pas.plugins.ldap user/group search
fredvd
 
Make Drupal Run Fast - increase page load speed
Make Drupal Run Fast - increase page load speedMake Drupal Run Fast - increase page load speed
Make Drupal Run Fast - increase page load speed
Andy Kucharski
 
Road to database automation: database source control
Road to database automation: database source controlRoad to database automation: database source control
Road to database automation: database source control
Eduardo Piairo
 
Lessons from Highly Scalable Architectures at Social Networking Sites
Lessons from Highly Scalable Architectures at Social Networking SitesLessons from Highly Scalable Architectures at Social Networking Sites
Lessons from Highly Scalable Architectures at Social Networking Sites
Patrick Senti
 
Road to database automation - Database source control
Road to database automation - Database source controlRoad to database automation - Database source control
Road to database automation - Database source control
Eduardo Piairo
 
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...
Jon Peck
 
Engage 2019: Modernising Your Domino and XPages Applications
Engage 2019: Modernising Your Domino and XPages Applications Engage 2019: Modernising Your Domino and XPages Applications
Engage 2019: Modernising Your Domino and XPages Applications
Paul Withers
 
Qcon
QconQcon

What's hot (20)

How to prepare your site for content migration
How to prepare your site for content migrationHow to prepare your site for content migration
How to prepare your site for content migration
 
SQL Server In-Memory OLTP Migration Overview
SQL Server In-Memory OLTP Migration OverviewSQL Server In-Memory OLTP Migration Overview
SQL Server In-Memory OLTP Migration Overview
 
Azkaban
AzkabanAzkaban
Azkaban
 
Delivering changes for applications and databases
Delivering changes for applications and databasesDelivering changes for applications and databases
Delivering changes for applications and databases
 
Resolving problems & high availability
Resolving problems & high availabilityResolving problems & high availability
Resolving problems & high availability
 
Managing Descriptive Metadata with Open XML...For Now
Managing Descriptive Metadata with Open XML...For NowManaging Descriptive Metadata with Open XML...For Now
Managing Descriptive Metadata with Open XML...For Now
 
Drupal Site Audit - SFDUG
Drupal Site Audit - SFDUGDrupal Site Audit - SFDUG
Drupal Site Audit - SFDUG
 
Untangling - fall2017 - week 9
Untangling - fall2017 - week 9Untangling - fall2017 - week 9
Untangling - fall2017 - week 9
 
Architecture Patterns - Open Discussion
Architecture Patterns - Open DiscussionArchitecture Patterns - Open Discussion
Architecture Patterns - Open Discussion
 
WebDev Crash Course
WebDev Crash CourseWebDev Crash Course
WebDev Crash Course
 
SenchaCon 2016: Oracle Forms Modernisation - Owen Pagan
SenchaCon 2016: Oracle Forms Modernisation - Owen PaganSenchaCon 2016: Oracle Forms Modernisation - Owen Pagan
SenchaCon 2016: Oracle Forms Modernisation - Owen Pagan
 
7 Stages of Scaling Web Applications
7 Stages of Scaling Web Applications7 Stages of Scaling Web Applications
7 Stages of Scaling Web Applications
 
Plone pas.plugins.ldap user/group search
Plone pas.plugins.ldap user/group searchPlone pas.plugins.ldap user/group search
Plone pas.plugins.ldap user/group search
 
Make Drupal Run Fast - increase page load speed
Make Drupal Run Fast - increase page load speedMake Drupal Run Fast - increase page load speed
Make Drupal Run Fast - increase page load speed
 
Road to database automation: database source control
Road to database automation: database source controlRoad to database automation: database source control
Road to database automation: database source control
 
Lessons from Highly Scalable Architectures at Social Networking Sites
Lessons from Highly Scalable Architectures at Social Networking SitesLessons from Highly Scalable Architectures at Social Networking Sites
Lessons from Highly Scalable Architectures at Social Networking Sites
 
Road to database automation - Database source control
Road to database automation - Database source controlRoad to database automation - Database source control
Road to database automation - Database source control
 
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcam...
 
Engage 2019: Modernising Your Domino and XPages Applications
Engage 2019: Modernising Your Domino and XPages Applications Engage 2019: Modernising Your Domino and XPages Applications
Engage 2019: Modernising Your Domino and XPages Applications
 
Qcon
QconQcon
Qcon
 

Similar to Performance metrics for a social network

My Site is slow - Drupal Camp London 2013
My Site is slow - Drupal Camp London 2013My Site is slow - Drupal Camp London 2013
My Site is slow - Drupal Camp London 2013
hernanibf
 
Profiling and Tuning a Web Application - The Dirty Details
Profiling and Tuning a Web Application - The Dirty DetailsProfiling and Tuning a Web Application - The Dirty Details
Profiling and Tuning a Web Application - The Dirty Details
Achievers Tech
 
My site is slow
My site is slowMy site is slow
My site is slow
hernanibf
 
Ship It ! with Ruby/ Rails Ecosystem
Ship It ! with Ruby/ Rails EcosystemShip It ! with Ruby/ Rails Ecosystem
Ship It ! with Ruby/ Rails Ecosystem
Yi-Ting Cheng
 
Tuenti Release Workflow
Tuenti Release WorkflowTuenti Release Workflow
Tuenti Release Workflow
Tuenti
 
Does Your Stuff Scale?
Does Your Stuff Scale?Does Your Stuff Scale?
Does Your Stuff Scale?
stevenh0lmes
 
Picnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable applicationPicnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable application
Nick Josevski
 
USG Summit - September 2014 - Web Management using Drupal
USG Summit - September 2014 - Web Management using DrupalUSG Summit - September 2014 - Web Management using Drupal
USG Summit - September 2014 - Web Management using Drupal
Eric Sembrat
 
How to Do a Performance Audit of Your .NET Website
How to Do a Performance Audit of Your .NET WebsiteHow to Do a Performance Audit of Your .NET Website
How to Do a Performance Audit of Your .NET Website
DNN
 
Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)
Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)
Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)
Sascha Wenninger
 
From Pilot to Product - Morning@Lohika
From Pilot to Product - Morning@LohikaFrom Pilot to Product - Morning@Lohika
From Pilot to Product - Morning@Lohika
Ivan Verhun
 
Redundant devops
Redundant devopsRedundant devops
Redundant devops
Szabolcs Szabolcsi-Tóth
 
Drupalmonitor.com - Drupal User Group Meetup Basel
Drupalmonitor.com - Drupal User Group Meetup BaselDrupalmonitor.com - Drupal User Group Meetup Basel
Drupalmonitor.com - Drupal User Group Meetup Basel
NETNODE AG
 
Front End page speed performance improvements for Drupal
Front End page speed performance improvements for DrupalFront End page speed performance improvements for Drupal
Front End page speed performance improvements for Drupal
Andy Kucharski
 
5 Common Mistakes You are Making on your Website
 5 Common Mistakes You are Making on your Website 5 Common Mistakes You are Making on your Website
5 Common Mistakes You are Making on your Website
Acquia
 
Key alias dev standard final
Key alias   dev standard finalKey alias   dev standard final
Key alias dev standard final
Raditya Alwafi Surachman
 
Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...
Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...
Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...
Jon Peck
 
Tuenti Release Workflow v1.1
Tuenti Release Workflow v1.1Tuenti Release Workflow v1.1
Tuenti Release Workflow v1.1
Tuenti
 
Be faster then rabbits
Be faster then rabbitsBe faster then rabbits
Be faster then rabbits
Vladislav Bauer
 
Make It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version ControlMake It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version Control
indiver
 

Similar to Performance metrics for a social network (20)

My Site is slow - Drupal Camp London 2013
My Site is slow - Drupal Camp London 2013My Site is slow - Drupal Camp London 2013
My Site is slow - Drupal Camp London 2013
 
Profiling and Tuning a Web Application - The Dirty Details
Profiling and Tuning a Web Application - The Dirty DetailsProfiling and Tuning a Web Application - The Dirty Details
Profiling and Tuning a Web Application - The Dirty Details
 
My site is slow
My site is slowMy site is slow
My site is slow
 
Ship It ! with Ruby/ Rails Ecosystem
Ship It ! with Ruby/ Rails EcosystemShip It ! with Ruby/ Rails Ecosystem
Ship It ! with Ruby/ Rails Ecosystem
 
Tuenti Release Workflow
Tuenti Release WorkflowTuenti Release Workflow
Tuenti Release Workflow
 
Does Your Stuff Scale?
Does Your Stuff Scale?Does Your Stuff Scale?
Does Your Stuff Scale?
 
Picnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable applicationPicnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable application
 
USG Summit - September 2014 - Web Management using Drupal
USG Summit - September 2014 - Web Management using DrupalUSG Summit - September 2014 - Web Management using Drupal
USG Summit - September 2014 - Web Management using Drupal
 
How to Do a Performance Audit of Your .NET Website
How to Do a Performance Audit of Your .NET WebsiteHow to Do a Performance Audit of Your .NET Website
How to Do a Performance Audit of Your .NET Website
 
Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)
Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)
Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)
 
From Pilot to Product - Morning@Lohika
From Pilot to Product - Morning@LohikaFrom Pilot to Product - Morning@Lohika
From Pilot to Product - Morning@Lohika
 
Redundant devops
Redundant devopsRedundant devops
Redundant devops
 
Drupalmonitor.com - Drupal User Group Meetup Basel
Drupalmonitor.com - Drupal User Group Meetup BaselDrupalmonitor.com - Drupal User Group Meetup Basel
Drupalmonitor.com - Drupal User Group Meetup Basel
 
Front End page speed performance improvements for Drupal
Front End page speed performance improvements for DrupalFront End page speed performance improvements for Drupal
Front End page speed performance improvements for Drupal
 
5 Common Mistakes You are Making on your Website
 5 Common Mistakes You are Making on your Website 5 Common Mistakes You are Making on your Website
5 Common Mistakes You are Making on your Website
 
Key alias dev standard final
Key alias   dev standard finalKey alias   dev standard final
Key alias dev standard final
 
Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...
Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...
Auditing Drupal Sites for Performance, Content and Optimal Configuration - SA...
 
Tuenti Release Workflow v1.1
Tuenti Release Workflow v1.1Tuenti Release Workflow v1.1
Tuenti Release Workflow v1.1
 
Be faster then rabbits
Be faster then rabbitsBe faster then rabbits
Be faster then rabbits
 
Make It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version ControlMake It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version Control
 

More from Thierry Schellenbach

Scalable News Feed with Mongo DB
Scalable News Feed with Mongo DBScalable News Feed with Mongo DB
Scalable News Feed with Mongo DB
Thierry Schellenbach
 
13 tips for a highly engaging feed
13 tips for a highly engaging feed13 tips for a highly engaging feed
13 tips for a highly engaging feed
Thierry Schellenbach
 
AWS Summit Amsterdam - Thierry Schellenbach Founder/ Fashiolista
AWS Summit Amsterdam - Thierry Schellenbach Founder/ FashiolistaAWS Summit Amsterdam - Thierry Schellenbach Founder/ Fashiolista
AWS Summit Amsterdam - Thierry Schellenbach Founder/ Fashiolista
Thierry Schellenbach
 
Feedly & Cassandra at Fashiolista
Feedly & Cassandra at FashiolistaFeedly & Cassandra at Fashiolista
Feedly & Cassandra at Fashiolista
Thierry Schellenbach
 
Fashiolista Notifications
Fashiolista NotificationsFashiolista Notifications
Fashiolista Notifications
Thierry Schellenbach
 
Just a millisecond_
Just a millisecond_Just a millisecond_
Just a millisecond_
Thierry Schellenbach
 

More from Thierry Schellenbach (6)

Scalable News Feed with Mongo DB
Scalable News Feed with Mongo DBScalable News Feed with Mongo DB
Scalable News Feed with Mongo DB
 
13 tips for a highly engaging feed
13 tips for a highly engaging feed13 tips for a highly engaging feed
13 tips for a highly engaging feed
 
AWS Summit Amsterdam - Thierry Schellenbach Founder/ Fashiolista
AWS Summit Amsterdam - Thierry Schellenbach Founder/ FashiolistaAWS Summit Amsterdam - Thierry Schellenbach Founder/ Fashiolista
AWS Summit Amsterdam - Thierry Schellenbach Founder/ Fashiolista
 
Feedly & Cassandra at Fashiolista
Feedly & Cassandra at FashiolistaFeedly & Cassandra at Fashiolista
Feedly & Cassandra at Fashiolista
 
Fashiolista Notifications
Fashiolista NotificationsFashiolista Notifications
Fashiolista Notifications
 
Just a millisecond_
Just a millisecond_Just a millisecond_
Just a millisecond_
 

Recently uploaded

Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
DianaGray10
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
DianaGray10
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
Finetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and DefendingFinetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and Defending
Priyanka Aash
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
Steven Carlson
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
Step-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From ScratchStep-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From Scratch
softsuave
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
maigasapphire
 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
janagijoythi
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
Bhajan Mehta
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
ssuser1915fe1
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
Zilliz
 
Types of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technologyTypes of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technology
ldtexsolbl
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
 
It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...
Zilliz
 
Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)
Debmalya Biswas
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
shanihomely
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
Matthias Neugebauer
 
Improving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning ContentImproving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning Content
Enterprise Knowledge
 
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdfLeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
SelfMade bd
 

Recently uploaded (20)

Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
Finetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and DefendingFinetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and Defending
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 
Step-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From ScratchStep-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From Scratch
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
 
Types of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technologyTypes of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technology
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
 
It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...
 
Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
 
Improving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning ContentImproving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning Content
 
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdfLeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
 

Performance metrics for a social network

  • 1. Performance metrics for a social network
  • 2. About Me • Thierry Schellenbach • Founder/ CTO Fashiolista • Author of Django Facebook • Github/tschellenbach • Blog: mellowmorning.com • @tschellenbach
  • 8. Growth 2nd largest fashion community • 1mln members • 17mln loves/month • 94mln non-bot pageviews
  • 9. Powered By • Django/Python • PostgreSQL • Solr • Redis • Celery • AWS/ Ubuntu • Nginx/ Gunicorn/ Supervisor
  • 10. Sexy Metrics driven optimization Hard Because • All content is personalized • Activity is clustered around a few users (>100k followers) • Individual users are insanely active (7 hours in a day is normal) • Social network, can’t easily shard data
  • 11. Speed is a Feature
  • 12. Metrics across the board • Development – Spot things early on, wrong usage of ORM etc • System Health – Is my DB healthy, my Redis cluster etc • Page level – Why is my page slow – What is the average speed of the components (DB, Redis, Solr etc)
  • 13. Tools we use Development System Health Page Level • Cloudwatch • New Relic • Debug toolbar • Munin – Cache calls • Graphite • Nagios – Graphite Timings • DB slow log – Queries and • Redis slow log their explains • Integration Tests – Duplicate query • PgFouine detection
  • 14. Development StatsD Duplicates Cache Calls
  • 15. Today’s Presentation New Relic Graphite PgFouine • Dashboard, High • Stash all data, • Understand level insights query it any way what keeps you want your DB busy • Tool, not a dashboard
  • 16. New Relic • Frontend -> App -> Components (DB, Solr, etc.) • Breaks page performance down into it’s components • Tracks deploys and compares before and after
  • 17. Are you Supported? • Ruby • Pip install newrelic • Java • Edit the .ini • .NET • Add the WSGI middleware • PHP • Wait for Magic • Python
  • 18. End user load times • Drill down all the way to Database calls • The purple line is our app, the rest frontend Frontend (97%) App
  • 20. Page Level • Average frontend performance per page • Click to view App level breakdown Page. Not URL. To App Level
  • 21. Drill down/ App overview History Memcached DB Query
  • 22. Database • See which tables are under most load • See which pages cause the load • Development over time
  • 24. Deploys part Two Response Time Pre & Post Memory Utilization
  • 25. Background Task Number of Task calls (sample)
  • 26. Graphite Insights • NewRelic has the overview, Graphite the detail • Open Source! • Throw data at it via UDP • Popularized by Etsy (see mellowmorning.com for link)
  • 29. Setup • Track using StatsD – Support for (PHP, Python, Ruby, Node, Java) • Hierarchy (python example) • get.<app>.<view>.<component> with request.timings('get.user.profile_page.sql'): print ‘database query here’
  • 30. Data tool/ Not a dashboard • Wildcards – get.<app>.<view>.*.upper_90 – get.<app>.*.redis.zadd.upper_90 – limit(sortByMaxima(get.<app>.<view>.*.up per_90),4)
  • 31. /style/<user>/ performance Memcached Slowdown ZADD Set Many
  • 32. Including Functional parts of Pages • More like this part is tracked • Solr & Redis Cache
  • 33. What we Track • Loadtime per bit of functionality • Database calls per DB • 90th percentile load times • Task broker roundtrip times • Facebook API calls
  • 34. PgFouine • Run on samples of all queries (say 5m) • Not just slow queries • Repeating a simple query many times is also wrong, PgFouine finds it • See Instagram’s fabric snippet • https://gist.github.com/2307647
  • 35. PgFouine Continued Queries that took up the most time (N) • Spots issues with many small queries Normalized Compare multiple reports
  • 36. PgFouine Tips • My colleague wrote a fast C++ version • github.com/WoLpH/pg_query_analyser Also look at: • Pg Stat Statement • Pg Badger
  • 37. Concluding New Relic Graphite PgFouine • Dashboard, High • Stash all data, • Understand level insights query it any way what keeps you want your DB busy • Tool, not a dashboard
  • 38. Q&A We’re Searching for Django Developers & Linux system administrators! Fashiolista.com/jobs Open source projects: Github.com/tschellenbach Try Django Facebook!

Editor's Notes

  1. Fashiolista
  2. Users of Fashiolista install the so called “love button”. While browsing around the web they can use this button to add their favourite fashion finds to Fashiolista.
  3. Once they click the button, we figure out the relevant image on the page and allow you to add it to your profile.
  4. The find is added to your profile and other people can follow the items you love.
  5. Over the past 2 years thing have moved along rapidly.Currently we’re the second largest fashion community worldwide.With close to 1mln members, and massive monthly engagement.So, a quick check.Who In this room is a member of Fashiolista
  6. So, let’s focus on the tech side of things.Powered by
  7. On to the topic of this talkTracking the right things to optimize your web application.Now, optimizing a social network is hard.I won’t go into the techniques we use at Fashiolista for speeding up the application, today we’ll focus on the metrics enabling us to focus on the bottlenecks.
  8. Why does it matter?We’ve consistently seen massive growth in pageviews after speeding up the site.You can often add 20% to the number of pageviews, just by making the site faster.Once you have an initial audience, making your product work well is really powerfull.
  9. We can divide most of our measurement tools into 3 categoriesdevelopment, system health, page level- Answering the specific questions
  10. Some of the tools we use.We care mainly about the page level.CPU on the database is interesting, but tells you more about when you’ll run into system limits.While working on optimizing you application you want to focus on the page level.How fast are the applications used to generate this page.Did we wait on the database, or was it solr.
  11. Small interlude, because I’m really happy with the Django debug toolbar.I used to work with Symonfy and Django borrowed this, but it’s awesome.Duplicate queriesStatsd Functional reportingCache calls
  12. A few tools are really slick and definitely deserve a little demo.NewRelicGraphitePgFouineExplain their use cases
  13. New Relic gives you the full drilldown. Starting from frontend to the app to the components.The apdex score tells you the percentage of users which had a good experience (page generated under 500ms).
  14. Useful to see how your CDN is doing it’s job.We’ve recently switched from Akamai to Cloudfront.Which seems to work quite well in most countries.
  15. At the page level you get a lot of cool information.You see the load times per component.- Such as the the time spent querying the db. (Entity_love table in this case)Or the time spent querying memcached (Which is quite substantial about 20ms)In addition you see the development over time.Which is great for spotting problems which are recently introduced.
  16. New Relic also offers database level components.Tables under most loadThe awesome bit, it relates this to pages cause the loadShow again the development over time. For fun try dropping an index and you’ll see it popping up immediately here.
  17. That nicely brings me to the deploy overview of newrelic.Every deploy gives you a nice change report.Showing what happened to the speed before and after your deploy.This allows you to quickly spot mistakes landing in production.
  18. Shows the average response time before and after your deploy.Also shows things like memory or CPU utilization which will pick major mistakes in those areas.
  19. Newrelic does pretty much the same thing for background tasks as for views.You can zoom into the specific components.If yourautoscaling suddenly boots up twice the number of task workers, NewRelic tells you why.
  20. So It’s clear I’m exited about New Relic. It’s an awesome tool and helped us a lot with scaling the site.However sometimes you have questions about your data which NewRelic can’t answer.We stick all the metrics we can think of in Graphite.Now NewRelic is really slick. Graphite is designed by engineers and, well looks like this.
  21. But it tracks everything you throw at it.And has a very powerful querying interface.Graphite is a data analysis tool. It’s not a dashboard.For instance in this case calls per database server.
  22. It’s however a data tool though. And not a dashboard.It has a really techy interface.You use * for wildcardsAnd call functions to run on your data.Quite ugly.
  23. You can also retrieve data similar to new relic load time breakdowns.With the added advantage that Graphite is a free tool.
  24. It also tracks functional parts of pages.So we see which part of the page is slowing down load times.
  25. We track things like loadtime per functionality.We track all database calls.We track 90th percentile loadtimes.Adding new measurements is super easy.
  26. Lastly we’re using PgFouineIt’s an awesome tool to get a complete understanding on what your database is actually doing.