SlideShare a Scribd company logo
1 of 23
How to build an app with
Twitter-like throughput
on just 9 servers...
Lew Cirne, Founder & CEO - New Relic
I’m Lew Cirne
@sweetlew
What our app does


APM as a Service

In-app agent instrumentation (BCI, etc)

150,000+ app processes monitored, globally (10K customers)

Each process reports a few hundred metrics per minute

5 Languages (Ruby, Java, PHP, .NET, Python)
Each day we collect 20 billion measurements,
    from 150,000 application processes,
         for over 10,000 customers.
Each day we collect 20 billion measurements,
    from 150,000 application processes,
         for over 10,000 customers.

              All on 9 servers.
We capture “Timeslices”
                              Each o ne is about
Response Time                    250 bytes
4 hours from 11:04 to 15:04
Count: 1242                           A single tweet
Avg: 337 ms
                                       is about the
Min: 0.63 ms
Max: 95669 ms                            same size
Std Dev: 782
timeslice insertion rate: 100K/second

 >7 billion rows per day
                           Twitter peak insertion rate:
                             8K rows per second

  9 Servers handle all
  data collection
Collecting is one thing...
• We provide realtime monitoring
• One minute granularity
• Data is almost always stale
• Each user/account has different data
• Page caching and other easy solutions don’t work for us.
Our most popular page...

                                    age
                            e Full P
                      Averag Time:
                         Load
                          2.4 Sec
Our most popular page...

                                    age
                            e Full P
                      Averag Time:
                         Load
                          2.4 Sec
Main App Software stack
User Interface       Data Collectors        Data Store
  & REST API                                     MySQL
                       Servlets on Jetty   Sharded by accounts
   Rails 2.3
Simplified architecture...
                                     9 Collector / Aggregator / DB’s
                                                                            Sustained 100K
                                                                            insertion rate per
                                                                            second



                             S
Customer’s environment   HTTP



                                                    24 Core Intel Nehalem
                                                    48 GB RAM
                                                    SAS attached RAID 5
                                                    No Virtualization

      (either cloud
     or datacenter)
                                                                    2 Web App Servers



                                         12 Core Intel Nehalem
                                         48 GB RAM
Even more data!

On May 17, we launched Real User Monitoring
• Using Episodes to measure browser load time of every page view

• Browser reports data to our ‘Beacon’ servers

• Monitoring >1 Billion page views per week

• Doubled our total inbound HTTP requests in a MONTH
Beacon Architecture
                                                           Response Time 0.15ms


                                                          RUM Beacons
          Real User                                                               Asynchronously
          Browsers             Billions of metrics from
                                                          Servlets Capture and
                                   across the globe       enqueue (in-memory)     aggregate and
                                                                                  forward
                                                                                  Timeslices to our
                                                                                  Collectors
  Over 1 Billion user sessions
measured for performance in first                          Currently at EC2
             month.
Challenges
• Data Purging
• Determining what to pre-aggregate
• Large Accounts
• MySQL Optimization and Tuning
• I/O performance - (virtualized to
  dedicated) ...
5 Lessons Learned
1. Keep it simple
2. Less is more
3. Trendy != Reliable
4. Plan for scale
s
                                             s ode
                                         Epi      New

                              Ja                 Relic
                                va
                                                               y
                                                             ub
5. Use the right technology          Ngin
                                         x    Je/y
                                                           R

                                                         Rails
      for a given task
See New Relic
Monitor New Relic
   at our booth

More Related Content

What's hot

Canary releases & Blue green deployment
Canary releases & Blue green deploymentCanary releases & Blue green deployment
Canary releases & Blue green deploymentSQUADEX
 
The Workshop: Alcanzando una observabilidad unificada con Elastic APM
The Workshop: Alcanzando una observabilidad unificada con Elastic APMThe Workshop: Alcanzando una observabilidad unificada con Elastic APM
The Workshop: Alcanzando una observabilidad unificada con Elastic APMElasticsearch
 
New Relic: Optimizing The Database SQL and NoSQL Alike
New Relic: Optimizing The Database SQL and NoSQL AlikeNew Relic: Optimizing The Database SQL and NoSQL Alike
New Relic: Optimizing The Database SQL and NoSQL AlikeBrian Doll
 
Vulnerability Discovery in the Cloud
Vulnerability Discovery in the CloudVulnerability Discovery in the Cloud
Vulnerability Discovery in the CloudDevOps.com
 
Support Office Hour Webinar - LivePerson API
Support Office Hour Webinar - LivePerson API Support Office Hour Webinar - LivePerson API
Support Office Hour Webinar - LivePerson API LivePerson
 
(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix
(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix
(ISM309) Efficient Innovation:High-Velocity Cost Management at NetflixAmazon Web Services
 
Let's decipher the DevOps macedonia
Let's decipher the DevOps macedoniaLet's decipher the DevOps macedonia
Let's decipher the DevOps macedoniaWamika Singh
 
Driving TAS Enterprise Fitness
Driving TAS Enterprise FitnessDriving TAS Enterprise Fitness
Driving TAS Enterprise FitnessVMware Tanzu
 
DevOps Days Toronto: From 6 Months Waterfall to 1 hour Code Deploys
DevOps Days Toronto: From 6 Months Waterfall to 1 hour Code DeploysDevOps Days Toronto: From 6 Months Waterfall to 1 hour Code Deploys
DevOps Days Toronto: From 6 Months Waterfall to 1 hour Code DeploysAndreas Grabner
 
The Netflix API for a global service
The Netflix API for a global serviceThe Netflix API for a global service
The Netflix API for a global serviceKatharina Probst
 
AWS Summit - Trends in Advanced Monitoring for AWS environments
AWS Summit - Trends in Advanced Monitoring for AWS environmentsAWS Summit - Trends in Advanced Monitoring for AWS environments
AWS Summit - Trends in Advanced Monitoring for AWS environmentsAndreas Grabner
 
Flink Forward San Francisco 2018: Andrew Gao & Jeff Sharpe - "Finding Bad Ac...
Flink Forward San Francisco 2018: Andrew Gao &  Jeff Sharpe - "Finding Bad Ac...Flink Forward San Francisco 2018: Andrew Gao &  Jeff Sharpe - "Finding Bad Ac...
Flink Forward San Francisco 2018: Andrew Gao & Jeff Sharpe - "Finding Bad Ac...Flink Forward
 
AppDynamics VS New Relic – The Complete Guide
AppDynamics VS New Relic – The Complete GuideAppDynamics VS New Relic – The Complete Guide
AppDynamics VS New Relic – The Complete GuideTakipi
 
Thorben Lindhauer: Live Coding: Zeebe - Camunda Day San Francisco
Thorben Lindhauer: Live Coding: Zeebe - Camunda Day San FranciscoThorben Lindhauer: Live Coding: Zeebe - Camunda Day San Francisco
Thorben Lindhauer: Live Coding: Zeebe - Camunda Day San Franciscocamunda services GmbH
 
Wamika Singh, Suman Kumari - Let's decipher the DevOps macedonia - Codemotion...
Wamika Singh, Suman Kumari - Let's decipher the DevOps macedonia - Codemotion...Wamika Singh, Suman Kumari - Let's decipher the DevOps macedonia - Codemotion...
Wamika Singh, Suman Kumari - Let's decipher the DevOps macedonia - Codemotion...Codemotion
 
Building A System That Never Stops [FutureStack16 NYC]
Building  A System That Never Stops [FutureStack16 NYC]Building  A System That Never Stops [FutureStack16 NYC]
Building A System That Never Stops [FutureStack16 NYC]New Relic
 
Telling the LivePerson Technology Story at Couchbase [SF] 2013
Telling the LivePerson Technology Story at Couchbase [SF] 2013Telling the LivePerson Technology Story at Couchbase [SF] 2013
Telling the LivePerson Technology Story at Couchbase [SF] 2013LivePerson
 
Migrating Target to Fastly - Eddie Roger at Fastly Altitude 2015
Migrating Target to Fastly - Eddie Roger at Fastly Altitude 2015Migrating Target to Fastly - Eddie Roger at Fastly Altitude 2015
Migrating Target to Fastly - Eddie Roger at Fastly Altitude 2015Fastly
 

What's hot (20)

Canary releases & Blue green deployment
Canary releases & Blue green deploymentCanary releases & Blue green deployment
Canary releases & Blue green deployment
 
The Workshop: Alcanzando una observabilidad unificada con Elastic APM
The Workshop: Alcanzando una observabilidad unificada con Elastic APMThe Workshop: Alcanzando una observabilidad unificada con Elastic APM
The Workshop: Alcanzando una observabilidad unificada con Elastic APM
 
Measure() or die()
Measure() or die()Measure() or die()
Measure() or die()
 
New Relic: Optimizing The Database SQL and NoSQL Alike
New Relic: Optimizing The Database SQL and NoSQL AlikeNew Relic: Optimizing The Database SQL and NoSQL Alike
New Relic: Optimizing The Database SQL and NoSQL Alike
 
Vulnerability Discovery in the Cloud
Vulnerability Discovery in the CloudVulnerability Discovery in the Cloud
Vulnerability Discovery in the Cloud
 
Support Office Hour Webinar - LivePerson API
Support Office Hour Webinar - LivePerson API Support Office Hour Webinar - LivePerson API
Support Office Hour Webinar - LivePerson API
 
New Relic
New RelicNew Relic
New Relic
 
(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix
(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix
(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix
 
Let's decipher the DevOps macedonia
Let's decipher the DevOps macedoniaLet's decipher the DevOps macedonia
Let's decipher the DevOps macedonia
 
Driving TAS Enterprise Fitness
Driving TAS Enterprise FitnessDriving TAS Enterprise Fitness
Driving TAS Enterprise Fitness
 
DevOps Days Toronto: From 6 Months Waterfall to 1 hour Code Deploys
DevOps Days Toronto: From 6 Months Waterfall to 1 hour Code DeploysDevOps Days Toronto: From 6 Months Waterfall to 1 hour Code Deploys
DevOps Days Toronto: From 6 Months Waterfall to 1 hour Code Deploys
 
The Netflix API for a global service
The Netflix API for a global serviceThe Netflix API for a global service
The Netflix API for a global service
 
AWS Summit - Trends in Advanced Monitoring for AWS environments
AWS Summit - Trends in Advanced Monitoring for AWS environmentsAWS Summit - Trends in Advanced Monitoring for AWS environments
AWS Summit - Trends in Advanced Monitoring for AWS environments
 
Flink Forward San Francisco 2018: Andrew Gao & Jeff Sharpe - "Finding Bad Ac...
Flink Forward San Francisco 2018: Andrew Gao &  Jeff Sharpe - "Finding Bad Ac...Flink Forward San Francisco 2018: Andrew Gao &  Jeff Sharpe - "Finding Bad Ac...
Flink Forward San Francisco 2018: Andrew Gao & Jeff Sharpe - "Finding Bad Ac...
 
AppDynamics VS New Relic – The Complete Guide
AppDynamics VS New Relic – The Complete GuideAppDynamics VS New Relic – The Complete Guide
AppDynamics VS New Relic – The Complete Guide
 
Thorben Lindhauer: Live Coding: Zeebe - Camunda Day San Francisco
Thorben Lindhauer: Live Coding: Zeebe - Camunda Day San FranciscoThorben Lindhauer: Live Coding: Zeebe - Camunda Day San Francisco
Thorben Lindhauer: Live Coding: Zeebe - Camunda Day San Francisco
 
Wamika Singh, Suman Kumari - Let's decipher the DevOps macedonia - Codemotion...
Wamika Singh, Suman Kumari - Let's decipher the DevOps macedonia - Codemotion...Wamika Singh, Suman Kumari - Let's decipher the DevOps macedonia - Codemotion...
Wamika Singh, Suman Kumari - Let's decipher the DevOps macedonia - Codemotion...
 
Building A System That Never Stops [FutureStack16 NYC]
Building  A System That Never Stops [FutureStack16 NYC]Building  A System That Never Stops [FutureStack16 NYC]
Building A System That Never Stops [FutureStack16 NYC]
 
Telling the LivePerson Technology Story at Couchbase [SF] 2013
Telling the LivePerson Technology Story at Couchbase [SF] 2013Telling the LivePerson Technology Story at Couchbase [SF] 2013
Telling the LivePerson Technology Story at Couchbase [SF] 2013
 
Migrating Target to Fastly - Eddie Roger at Fastly Altitude 2015
Migrating Target to Fastly - Eddie Roger at Fastly Altitude 2015Migrating Target to Fastly - Eddie Roger at Fastly Altitude 2015
Migrating Target to Fastly - Eddie Roger at Fastly Altitude 2015
 

Viewers also liked

SaaS Introduction-May2014
SaaS Introduction-May2014SaaS Introduction-May2014
SaaS Introduction-May2014Nguyen Tung
 
The Sweet Science Of Virality
The Sweet Science Of ViralityThe Sweet Science Of Virality
The Sweet Science Of ViralityUpworthy
 
QA Automation course 2014 - DIO-soft, Kyiv
QA Automation course 2014 - DIO-soft, KyivQA Automation course 2014 - DIO-soft, Kyiv
QA Automation course 2014 - DIO-soft, KyivSergey Kochergan
 
SaaS Business Architecture - Definition Update
SaaS Business Architecture - Definition UpdateSaaS Business Architecture - Definition Update
SaaS Business Architecture - Definition UpdateLincoln Murphy
 
Building Highly Scalable and Flexible SaaS Solutions
Building Highly Scalable and Flexible SaaS SolutionsBuilding Highly Scalable and Flexible SaaS Solutions
Building Highly Scalable and Flexible SaaS SolutionsImpetus Technologies
 
SaaS Business Model: A Unique Business Architecture
SaaS Business Model: A Unique Business ArchitectureSaaS Business Model: A Unique Business Architecture
SaaS Business Model: A Unique Business ArchitectureLincoln Murphy
 
SaaS Business Architecture
SaaS Business ArchitectureSaaS Business Architecture
SaaS Business ArchitectureLincoln Murphy
 
LinkedIn Executive Playbook: 12 Steps to Become a Social Leader
LinkedIn Executive Playbook: 12 Steps to Become a Social LeaderLinkedIn Executive Playbook: 12 Steps to Become a Social Leader
LinkedIn Executive Playbook: 12 Steps to Become a Social LeaderLinkedIn Hong Kong
 
E-commerce Berlin Expo - Brand24 - Mike Sadowski
E-commerce Berlin Expo - Brand24 - Mike SadowskiE-commerce Berlin Expo - Brand24 - Mike Sadowski
E-commerce Berlin Expo - Brand24 - Mike SadowskiE-Commerce Berlin EXPO
 
Doing customer development (and stop wasting your time)
Doing customer development (and stop wasting your time)Doing customer development (and stop wasting your time)
Doing customer development (and stop wasting your time)Hans van Gent
 
Scaling Pinterest
Scaling PinterestScaling Pinterest
Scaling PinterestC4Media
 
Architecting SaaS: Doing It Right the First Time
Architecting SaaS: Doing It Right the First TimeArchitecting SaaS: Doing It Right the First Time
Architecting SaaS: Doing It Right the First TimeSerhiy (Serge) Haziyev
 
Key Takeaways from The Sales Development Playbook, part 1 and part 2
Key Takeaways from The Sales Development Playbook, part 1 and part 2Key Takeaways from The Sales Development Playbook, part 1 and part 2
Key Takeaways from The Sales Development Playbook, part 1 and part 2WhereDat
 
Chapter 10 Anomaly Detection
Chapter 10 Anomaly DetectionChapter 10 Anomaly Detection
Chapter 10 Anomaly DetectionKhalid Elshafie
 
Adobe Summit - Data Storytelling
Adobe Summit - Data StorytellingAdobe Summit - Data Storytelling
Adobe Summit - Data StorytellingChris Haleua
 
Bombora - Intent data - The secret to smarter sales prospecting? - April 2016
Bombora - Intent data - The secret to smarter sales prospecting? - April 2016 Bombora - Intent data - The secret to smarter sales prospecting? - April 2016
Bombora - Intent data - The secret to smarter sales prospecting? - April 2016 Bombora
 
Software As A Service Presentation
Software As A Service PresentationSoftware As A Service Presentation
Software As A Service Presentational95iii
 
Adobe Digital Index Q1 2015 Digital Video Report
Adobe Digital Index Q1 2015 Digital Video ReportAdobe Digital Index Q1 2015 Digital Video Report
Adobe Digital Index Q1 2015 Digital Video ReportAdobe
 

Viewers also liked (20)

SaaS Introduction-May2014
SaaS Introduction-May2014SaaS Introduction-May2014
SaaS Introduction-May2014
 
The Sweet Science Of Virality
The Sweet Science Of ViralityThe Sweet Science Of Virality
The Sweet Science Of Virality
 
QA Automation course 2014 - DIO-soft, Kyiv
QA Automation course 2014 - DIO-soft, KyivQA Automation course 2014 - DIO-soft, Kyiv
QA Automation course 2014 - DIO-soft, Kyiv
 
SaaS Business Architecture - Definition Update
SaaS Business Architecture - Definition UpdateSaaS Business Architecture - Definition Update
SaaS Business Architecture - Definition Update
 
Building Highly Scalable and Flexible SaaS Solutions
Building Highly Scalable and Flexible SaaS SolutionsBuilding Highly Scalable and Flexible SaaS Solutions
Building Highly Scalable and Flexible SaaS Solutions
 
SaaS Business Model: A Unique Business Architecture
SaaS Business Model: A Unique Business ArchitectureSaaS Business Model: A Unique Business Architecture
SaaS Business Model: A Unique Business Architecture
 
SaaS Business Architecture
SaaS Business ArchitectureSaaS Business Architecture
SaaS Business Architecture
 
LinkedIn Executive Playbook: 12 Steps to Become a Social Leader
LinkedIn Executive Playbook: 12 Steps to Become a Social LeaderLinkedIn Executive Playbook: 12 Steps to Become a Social Leader
LinkedIn Executive Playbook: 12 Steps to Become a Social Leader
 
E-commerce Berlin Expo - Brand24 - Mike Sadowski
E-commerce Berlin Expo - Brand24 - Mike SadowskiE-commerce Berlin Expo - Brand24 - Mike Sadowski
E-commerce Berlin Expo - Brand24 - Mike Sadowski
 
Doing customer development (and stop wasting your time)
Doing customer development (and stop wasting your time)Doing customer development (and stop wasting your time)
Doing customer development (and stop wasting your time)
 
An introduction and overview to Software as a Service
An introduction and overview to Software as a Service An introduction and overview to Software as a Service
An introduction and overview to Software as a Service
 
Scaling Pinterest
Scaling PinterestScaling Pinterest
Scaling Pinterest
 
Architecting SaaS: Doing It Right the First Time
Architecting SaaS: Doing It Right the First TimeArchitecting SaaS: Doing It Right the First Time
Architecting SaaS: Doing It Right the First Time
 
Key Takeaways from The Sales Development Playbook, part 1 and part 2
Key Takeaways from The Sales Development Playbook, part 1 and part 2Key Takeaways from The Sales Development Playbook, part 1 and part 2
Key Takeaways from The Sales Development Playbook, part 1 and part 2
 
Chapter 10 Anomaly Detection
Chapter 10 Anomaly DetectionChapter 10 Anomaly Detection
Chapter 10 Anomaly Detection
 
Adobe Summit - Data Storytelling
Adobe Summit - Data StorytellingAdobe Summit - Data Storytelling
Adobe Summit - Data Storytelling
 
Bombora - Intent data - The secret to smarter sales prospecting? - April 2016
Bombora - Intent data - The secret to smarter sales prospecting? - April 2016 Bombora - Intent data - The secret to smarter sales prospecting? - April 2016
Bombora - Intent data - The secret to smarter sales prospecting? - April 2016
 
Software As A Service Presentation
Software As A Service PresentationSoftware As A Service Presentation
Software As A Service Presentation
 
Adobe Digital Index Q1 2015 Digital Video Report
Adobe Digital Index Q1 2015 Digital Video ReportAdobe Digital Index Q1 2015 Digital Video Report
Adobe Digital Index Q1 2015 Digital Video Report
 
Top 10 Tech Jobs for 2016
Top 10 Tech Jobs for 2016Top 10 Tech Jobs for 2016
Top 10 Tech Jobs for 2016
 

Similar to How to Build a SaaS App With Twitter-like Throughput on Just 9 Servers

Overcoming the Top Four Challenges to Real‐Time Performance in Large‐Scale, D...
Overcoming the Top Four Challenges to Real‐Time Performance in Large‐Scale, D...Overcoming the Top Four Challenges to Real‐Time Performance in Large‐Scale, D...
Overcoming the Top Four Challenges to Real‐Time Performance in Large‐Scale, D...SL Corporation
 
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...SL Corporation
 
Coates bosc2010 clouds-fluff-and-no-substance
Coates bosc2010 clouds-fluff-and-no-substanceCoates bosc2010 clouds-fluff-and-no-substance
Coates bosc2010 clouds-fluff-and-no-substanceBOSC 2010
 
Optimizing Your Cloud Applications in RightScale
Optimizing Your Cloud Applications in RightScaleOptimizing Your Cloud Applications in RightScale
Optimizing Your Cloud Applications in RightScaleRightScale
 
Introduction to Databus
Introduction to DatabusIntroduction to Databus
Introduction to DatabusAmy W. Tang
 
Oracle in the Cloud
Oracle in the CloudOracle in the Cloud
Oracle in the Cloudzain1425
 
Complex Event Processing: What?, Why?, How?
Complex Event Processing: What?, Why?, How?Complex Event Processing: What?, Why?, How?
Complex Event Processing: What?, Why?, How?Fabien Coppens
 
Netflix Open Source Meetup Season 4 Episode 2
Netflix Open Source Meetup Season 4 Episode 2Netflix Open Source Meetup Season 4 Episode 2
Netflix Open Source Meetup Season 4 Episode 2aspyker
 
Reactive programming
Reactive programmingReactive programming
Reactive programmingSUDIP GHOSH
 
Scaling habits of ASP.NET
Scaling habits of ASP.NETScaling habits of ASP.NET
Scaling habits of ASP.NETDavid Giard
 
Introduction openstack-meetup-nov-28
Introduction openstack-meetup-nov-28Introduction openstack-meetup-nov-28
Introduction openstack-meetup-nov-28Sadique Puthen
 
OSDC 2017 | Something Openshift Kubernetes Containers by Kristian Köhntopp
OSDC 2017 | Something Openshift Kubernetes Containers by Kristian KöhntoppOSDC 2017 | Something Openshift Kubernetes Containers by Kristian Köhntopp
OSDC 2017 | Something Openshift Kubernetes Containers by Kristian KöhntoppNETWAYS
 
The Good, the Bad and the Ugly of Migrating Hundreds of Legacy Applications ...
 The Good, the Bad and the Ugly of Migrating Hundreds of Legacy Applications ... The Good, the Bad and the Ugly of Migrating Hundreds of Legacy Applications ...
The Good, the Bad and the Ugly of Migrating Hundreds of Legacy Applications ...Josef Adersberger
 
Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...
Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...
Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...QAware GmbH
 
Openstack on Fedora, Fedora on Openstack: An Introduction to cloud IaaS
Openstack on Fedora, Fedora on Openstack: An Introduction to cloud IaaSOpenstack on Fedora, Fedora on Openstack: An Introduction to cloud IaaS
Openstack on Fedora, Fedora on Openstack: An Introduction to cloud IaaSSadique Puthen
 
Azug - successfully breeding rabits
Azug - successfully breeding rabitsAzug - successfully breeding rabits
Azug - successfully breeding rabitsYves Goeleven
 
Scalable Resilient Web Services In .Net
Scalable Resilient Web Services In .NetScalable Resilient Web Services In .Net
Scalable Resilient Web Services In .NetBala Subra
 
Log everything!
Log everything!Log everything!
Log everything!ICANS GmbH
 

Similar to How to Build a SaaS App With Twitter-like Throughput on Just 9 Servers (20)

Overcoming the Top Four Challenges to Real‐Time Performance in Large‐Scale, D...
Overcoming the Top Four Challenges to Real‐Time Performance in Large‐Scale, D...Overcoming the Top Four Challenges to Real‐Time Performance in Large‐Scale, D...
Overcoming the Top Four Challenges to Real‐Time Performance in Large‐Scale, D...
 
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...
 
Coates bosc2010 clouds-fluff-and-no-substance
Coates bosc2010 clouds-fluff-and-no-substanceCoates bosc2010 clouds-fluff-and-no-substance
Coates bosc2010 clouds-fluff-and-no-substance
 
Optimizing Your Cloud Applications in RightScale
Optimizing Your Cloud Applications in RightScaleOptimizing Your Cloud Applications in RightScale
Optimizing Your Cloud Applications in RightScale
 
Introduction to Databus
Introduction to DatabusIntroduction to Databus
Introduction to Databus
 
Oracle in the Cloud
Oracle in the CloudOracle in the Cloud
Oracle in the Cloud
 
Complex Event Processing: What?, Why?, How?
Complex Event Processing: What?, Why?, How?Complex Event Processing: What?, Why?, How?
Complex Event Processing: What?, Why?, How?
 
Netflix Open Source Meetup Season 4 Episode 2
Netflix Open Source Meetup Season 4 Episode 2Netflix Open Source Meetup Season 4 Episode 2
Netflix Open Source Meetup Season 4 Episode 2
 
Reactive programming
Reactive programmingReactive programming
Reactive programming
 
Scaling habits of ASP.NET
Scaling habits of ASP.NETScaling habits of ASP.NET
Scaling habits of ASP.NET
 
Introduction openstack-meetup-nov-28
Introduction openstack-meetup-nov-28Introduction openstack-meetup-nov-28
Introduction openstack-meetup-nov-28
 
OSDC 2017 | Something Openshift Kubernetes Containers by Kristian Köhntopp
OSDC 2017 | Something Openshift Kubernetes Containers by Kristian KöhntoppOSDC 2017 | Something Openshift Kubernetes Containers by Kristian Köhntopp
OSDC 2017 | Something Openshift Kubernetes Containers by Kristian Köhntopp
 
The Good, the Bad and the Ugly of Migrating Hundreds of Legacy Applications ...
 The Good, the Bad and the Ugly of Migrating Hundreds of Legacy Applications ... The Good, the Bad and the Ugly of Migrating Hundreds of Legacy Applications ...
The Good, the Bad and the Ugly of Migrating Hundreds of Legacy Applications ...
 
Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...
Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...
Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...
 
Openstack on Fedora, Fedora on Openstack: An Introduction to cloud IaaS
Openstack on Fedora, Fedora on Openstack: An Introduction to cloud IaaSOpenstack on Fedora, Fedora on Openstack: An Introduction to cloud IaaS
Openstack on Fedora, Fedora on Openstack: An Introduction to cloud IaaS
 
Amazon Kinesis
Amazon KinesisAmazon Kinesis
Amazon Kinesis
 
Azug - successfully breeding rabits
Azug - successfully breeding rabitsAzug - successfully breeding rabits
Azug - successfully breeding rabits
 
Managed NoSQL databases
Managed NoSQL databasesManaged NoSQL databases
Managed NoSQL databases
 
Scalable Resilient Web Services In .Net
Scalable Resilient Web Services In .NetScalable Resilient Web Services In .Net
Scalable Resilient Web Services In .Net
 
Log everything!
Log everything!Log everything!
Log everything!
 

More from New Relic

7 Tips & Tricks to Having Happy Customers at Scale
7 Tips & Tricks to Having Happy Customers at Scale7 Tips & Tricks to Having Happy Customers at Scale
7 Tips & Tricks to Having Happy Customers at ScaleNew Relic
 
7 Tips & Tricks to Having Happy Customers at Scale
7 Tips & Tricks to Having Happy Customers at Scale7 Tips & Tricks to Having Happy Customers at Scale
7 Tips & Tricks to Having Happy Customers at ScaleNew Relic
 
New Relic University at Future Stack Tokyo 2019
New Relic University at Future Stack Tokyo 2019New Relic University at Future Stack Tokyo 2019
New Relic University at Future Stack Tokyo 2019New Relic
 
FutureStack Tokyo 19 -[事例講演]株式会社リクルートライフスタイル:年間9300万件以上のサロン予約を支えるホットペッパービューティ...
FutureStack Tokyo 19 -[事例講演]株式会社リクルートライフスタイル:年間9300万件以上のサロン予約を支えるホットペッパービューティ...FutureStack Tokyo 19 -[事例講演]株式会社リクルートライフスタイル:年間9300万件以上のサロン予約を支えるホットペッパービューティ...
FutureStack Tokyo 19 -[事例講演]株式会社リクルートライフスタイル:年間9300万件以上のサロン予約を支えるホットペッパービューティ...New Relic
 
FutureStack Tokyo 19 -[New Relic テクニカル講演]モニタリングと可視化がデジタルトランスフォーメーションを救う! - サ...
FutureStack  Tokyo 19 -[New Relic テクニカル講演]モニタリングと可視化がデジタルトランスフォーメーションを救う! - サ...FutureStack  Tokyo 19 -[New Relic テクニカル講演]モニタリングと可視化がデジタルトランスフォーメーションを救う! - サ...
FutureStack Tokyo 19 -[New Relic テクニカル講演]モニタリングと可視化がデジタルトランスフォーメーションを救う! - サ...New Relic
 
FutureStack Tokyo 19 -[特別講演]システム開発によろこびと驚きの連鎖を
FutureStack Tokyo 19 -[特別講演]システム開発によろこびと驚きの連鎖をFutureStack Tokyo 19 -[特別講演]システム開発によろこびと驚きの連鎖を
FutureStack Tokyo 19 -[特別講演]システム開発によろこびと驚きの連鎖をNew Relic
 
FutureStack Tokyo 19 -[パートナー講演]アマゾン ウェブ サービス ジャパン株式会社: New Relicを活用したAWSへのアプリ...
FutureStack Tokyo 19 -[パートナー講演]アマゾン ウェブ サービス ジャパン株式会社: New Relicを活用したAWSへのアプリ...FutureStack Tokyo 19 -[パートナー講演]アマゾン ウェブ サービス ジャパン株式会社: New Relicを活用したAWSへのアプリ...
FutureStack Tokyo 19 -[パートナー講演]アマゾン ウェブ サービス ジャパン株式会社: New Relicを活用したAWSへのアプリ...New Relic
 
FutureStack Tokyo 19_インサイトとデータを組織の力にする_株式会社ドワンゴ 池田 明啓 氏
FutureStack Tokyo 19_インサイトとデータを組織の力にする_株式会社ドワンゴ 池田 明啓 氏FutureStack Tokyo 19_インサイトとデータを組織の力にする_株式会社ドワンゴ 池田 明啓 氏
FutureStack Tokyo 19_インサイトとデータを組織の力にする_株式会社ドワンゴ 池田 明啓 氏New Relic
 
Three Monitoring Mistakes and How to Avoid Them
Three Monitoring Mistakes and How to Avoid ThemThree Monitoring Mistakes and How to Avoid Them
Three Monitoring Mistakes and How to Avoid ThemNew Relic
 
Intro to Multidimensional Kubernetes Monitoring
Intro to Multidimensional Kubernetes MonitoringIntro to Multidimensional Kubernetes Monitoring
Intro to Multidimensional Kubernetes MonitoringNew Relic
 
FS18 Chicago Keynote
FS18 Chicago Keynote FS18 Chicago Keynote
FS18 Chicago Keynote New Relic
 
10 Things You Can Do With New Relic - Number 9 Will Shock You
10 Things You Can Do With New Relic - Number 9 Will Shock You10 Things You Can Do With New Relic - Number 9 Will Shock You
10 Things You Can Do With New Relic - Number 9 Will Shock YouNew Relic
 
Ground Rules for Code Reviews
Ground Rules for Code ReviewsGround Rules for Code Reviews
Ground Rules for Code ReviewsNew Relic
 
Understanding Microservice Latency for DevOps Teams: An Introduction to New R...
Understanding Microservice Latency for DevOps Teams: An Introduction to New R...Understanding Microservice Latency for DevOps Teams: An Introduction to New R...
Understanding Microservice Latency for DevOps Teams: An Introduction to New R...New Relic
 
Monitor all your Kubernetes and EKS stack with New Relic
Monitor all your Kubernetes and EKS stack with New Relic	Monitor all your Kubernetes and EKS stack with New Relic
Monitor all your Kubernetes and EKS stack with New Relic New Relic
 
Host for the Most: Cloud Cost Optimization
Host for the Most: Cloud Cost OptimizationHost for the Most: Cloud Cost Optimization
Host for the Most: Cloud Cost OptimizationNew Relic
 
New Relic Infrastructure in the Real World: AWS
New Relic Infrastructure in the Real World: AWSNew Relic Infrastructure in the Real World: AWS
New Relic Infrastructure in the Real World: AWSNew Relic
 
Best Practices for Measuring your Code Pipeline
Best Practices for Measuring your Code PipelineBest Practices for Measuring your Code Pipeline
Best Practices for Measuring your Code PipelineNew Relic
 
Top Three Mistakes People Make with Monitoring
Top Three Mistakes People Make with MonitoringTop Three Mistakes People Make with Monitoring
Top Three Mistakes People Make with MonitoringNew Relic
 

More from New Relic (20)

7 Tips & Tricks to Having Happy Customers at Scale
7 Tips & Tricks to Having Happy Customers at Scale7 Tips & Tricks to Having Happy Customers at Scale
7 Tips & Tricks to Having Happy Customers at Scale
 
7 Tips & Tricks to Having Happy Customers at Scale
7 Tips & Tricks to Having Happy Customers at Scale7 Tips & Tricks to Having Happy Customers at Scale
7 Tips & Tricks to Having Happy Customers at Scale
 
New Relic University at Future Stack Tokyo 2019
New Relic University at Future Stack Tokyo 2019New Relic University at Future Stack Tokyo 2019
New Relic University at Future Stack Tokyo 2019
 
FutureStack Tokyo 19 -[事例講演]株式会社リクルートライフスタイル:年間9300万件以上のサロン予約を支えるホットペッパービューティ...
FutureStack Tokyo 19 -[事例講演]株式会社リクルートライフスタイル:年間9300万件以上のサロン予約を支えるホットペッパービューティ...FutureStack Tokyo 19 -[事例講演]株式会社リクルートライフスタイル:年間9300万件以上のサロン予約を支えるホットペッパービューティ...
FutureStack Tokyo 19 -[事例講演]株式会社リクルートライフスタイル:年間9300万件以上のサロン予約を支えるホットペッパービューティ...
 
FutureStack Tokyo 19 -[New Relic テクニカル講演]モニタリングと可視化がデジタルトランスフォーメーションを救う! - サ...
FutureStack  Tokyo 19 -[New Relic テクニカル講演]モニタリングと可視化がデジタルトランスフォーメーションを救う! - サ...FutureStack  Tokyo 19 -[New Relic テクニカル講演]モニタリングと可視化がデジタルトランスフォーメーションを救う! - サ...
FutureStack Tokyo 19 -[New Relic テクニカル講演]モニタリングと可視化がデジタルトランスフォーメーションを救う! - サ...
 
FutureStack Tokyo 19 -[特別講演]システム開発によろこびと驚きの連鎖を
FutureStack Tokyo 19 -[特別講演]システム開発によろこびと驚きの連鎖をFutureStack Tokyo 19 -[特別講演]システム開発によろこびと驚きの連鎖を
FutureStack Tokyo 19 -[特別講演]システム開発によろこびと驚きの連鎖を
 
FutureStack Tokyo 19 -[パートナー講演]アマゾン ウェブ サービス ジャパン株式会社: New Relicを活用したAWSへのアプリ...
FutureStack Tokyo 19 -[パートナー講演]アマゾン ウェブ サービス ジャパン株式会社: New Relicを活用したAWSへのアプリ...FutureStack Tokyo 19 -[パートナー講演]アマゾン ウェブ サービス ジャパン株式会社: New Relicを活用したAWSへのアプリ...
FutureStack Tokyo 19 -[パートナー講演]アマゾン ウェブ サービス ジャパン株式会社: New Relicを活用したAWSへのアプリ...
 
FutureStack Tokyo 19_インサイトとデータを組織の力にする_株式会社ドワンゴ 池田 明啓 氏
FutureStack Tokyo 19_インサイトとデータを組織の力にする_株式会社ドワンゴ 池田 明啓 氏FutureStack Tokyo 19_インサイトとデータを組織の力にする_株式会社ドワンゴ 池田 明啓 氏
FutureStack Tokyo 19_インサイトとデータを組織の力にする_株式会社ドワンゴ 池田 明啓 氏
 
Three Monitoring Mistakes and How to Avoid Them
Three Monitoring Mistakes and How to Avoid ThemThree Monitoring Mistakes and How to Avoid Them
Three Monitoring Mistakes and How to Avoid Them
 
Intro to Multidimensional Kubernetes Monitoring
Intro to Multidimensional Kubernetes MonitoringIntro to Multidimensional Kubernetes Monitoring
Intro to Multidimensional Kubernetes Monitoring
 
FS18 Chicago Keynote
FS18 Chicago Keynote FS18 Chicago Keynote
FS18 Chicago Keynote
 
SRE-iously
SRE-iouslySRE-iously
SRE-iously
 
10 Things You Can Do With New Relic - Number 9 Will Shock You
10 Things You Can Do With New Relic - Number 9 Will Shock You10 Things You Can Do With New Relic - Number 9 Will Shock You
10 Things You Can Do With New Relic - Number 9 Will Shock You
 
Ground Rules for Code Reviews
Ground Rules for Code ReviewsGround Rules for Code Reviews
Ground Rules for Code Reviews
 
Understanding Microservice Latency for DevOps Teams: An Introduction to New R...
Understanding Microservice Latency for DevOps Teams: An Introduction to New R...Understanding Microservice Latency for DevOps Teams: An Introduction to New R...
Understanding Microservice Latency for DevOps Teams: An Introduction to New R...
 
Monitor all your Kubernetes and EKS stack with New Relic
Monitor all your Kubernetes and EKS stack with New Relic	Monitor all your Kubernetes and EKS stack with New Relic
Monitor all your Kubernetes and EKS stack with New Relic
 
Host for the Most: Cloud Cost Optimization
Host for the Most: Cloud Cost OptimizationHost for the Most: Cloud Cost Optimization
Host for the Most: Cloud Cost Optimization
 
New Relic Infrastructure in the Real World: AWS
New Relic Infrastructure in the Real World: AWSNew Relic Infrastructure in the Real World: AWS
New Relic Infrastructure in the Real World: AWS
 
Best Practices for Measuring your Code Pipeline
Best Practices for Measuring your Code PipelineBest Practices for Measuring your Code Pipeline
Best Practices for Measuring your Code Pipeline
 
Top Three Mistakes People Make with Monitoring
Top Three Mistakes People Make with MonitoringTop Three Mistakes People Make with Monitoring
Top Three Mistakes People Make with Monitoring
 

Recently uploaded

ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityVictorSzoltysek
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...FIDO Alliance
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024Lorenzo Miniero
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...panagenda
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMKumar Satyam
 
Google I/O Extended 2024 Warsaw
Google I/O Extended 2024 WarsawGoogle I/O Extended 2024 Warsaw
Google I/O Extended 2024 WarsawGDSC PJATK
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)Wonjun Hwang
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsLeah Henrickson
 
Generative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdfGenerative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdfalexjohnson7307
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Skynet Technologies
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfdanishmna97
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهMohamed Sweelam
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...ScyllaDB
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxjbellis
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfAnubhavMangla3
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfSrushith Repakula
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftshyamraj55
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxFIDO Alliance
 

Recently uploaded (20)

ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
Google I/O Extended 2024 Warsaw
Google I/O Extended 2024 WarsawGoogle I/O Extended 2024 Warsaw
Google I/O Extended 2024 Warsaw
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
Generative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdfGenerative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdf
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cf
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهله
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptx
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptx
 

How to Build a SaaS App With Twitter-like Throughput on Just 9 Servers

  • 1. How to build an app with Twitter-like throughput on just 9 servers... Lew Cirne, Founder & CEO - New Relic
  • 3. What our app does APM as a Service In-app agent instrumentation (BCI, etc) 150,000+ app processes monitored, globally (10K customers) Each process reports a few hundred metrics per minute 5 Languages (Ruby, Java, PHP, .NET, Python)
  • 4. Each day we collect 20 billion measurements, from 150,000 application processes, for over 10,000 customers.
  • 5. Each day we collect 20 billion measurements, from 150,000 application processes, for over 10,000 customers. All on 9 servers.
  • 6. We capture “Timeslices” Each o ne is about Response Time 250 bytes 4 hours from 11:04 to 15:04 Count: 1242 A single tweet Avg: 337 ms is about the Min: 0.63 ms Max: 95669 ms same size Std Dev: 782
  • 7. timeslice insertion rate: 100K/second >7 billion rows per day Twitter peak insertion rate: 8K rows per second 9 Servers handle all data collection
  • 8.
  • 9. Collecting is one thing... • We provide realtime monitoring • One minute granularity • Data is almost always stale • Each user/account has different data • Page caching and other easy solutions don’t work for us.
  • 10. Our most popular page... age e Full P Averag Time: Load 2.4 Sec
  • 11. Our most popular page... age e Full P Averag Time: Load 2.4 Sec
  • 12. Main App Software stack User Interface Data Collectors Data Store & REST API MySQL Servlets on Jetty Sharded by accounts Rails 2.3
  • 13. Simplified architecture... 9 Collector / Aggregator / DB’s Sustained 100K insertion rate per second S Customer’s environment HTTP 24 Core Intel Nehalem 48 GB RAM SAS attached RAID 5 No Virtualization (either cloud or datacenter) 2 Web App Servers 12 Core Intel Nehalem 48 GB RAM
  • 14. Even more data! On May 17, we launched Real User Monitoring • Using Episodes to measure browser load time of every page view • Browser reports data to our ‘Beacon’ servers • Monitoring >1 Billion page views per week • Doubled our total inbound HTTP requests in a MONTH
  • 15. Beacon Architecture Response Time 0.15ms RUM Beacons Real User Asynchronously Browsers Billions of metrics from Servlets Capture and across the globe enqueue (in-memory) aggregate and forward Timeslices to our Collectors Over 1 Billion user sessions measured for performance in first Currently at EC2 month.
  • 16. Challenges • Data Purging • Determining what to pre-aggregate • Large Accounts • MySQL Optimization and Tuning • I/O performance - (virtualized to dedicated) ...
  • 18. 1. Keep it simple
  • 19. 2. Less is more
  • 20. 3. Trendy != Reliable
  • 21. 4. Plan for scale
  • 22. s s ode Epi New
 Ja Relic va y ub 5. Use the right technology Ngin x Je/y R Rails for a given task
  • 23. See New Relic Monitor New Relic at our booth

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. Software\n
  17. People\n
  18. \n
  19. \n
  20. \n
  21. \n