• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
NY Web Perf Meetup: Peeling the Web Performance Onion
 

NY Web Perf Meetup: Peeling the Web Performance Onion

on

  • 3,169 views

NY Web Perf Meetup presentation

NY Web Perf Meetup presentation
Web Performance Monitoring

Statistics

Views

Total Views
3,169
Views on SlideShare
2,284
Embed Views
885

Actions

Likes
1
Downloads
0
Comments
0

6 Embeds 885

http://blog.catchpoint.com 867
http://www.linkedin.com 9
https://www.linkedin.com 5
url_unknown 2
http://translate.googleusercontent.com 1
http://www.thesearchengine.net 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    NY Web Perf Meetup: Peeling the Web Performance Onion NY Web Perf Meetup: Peeling the Web Performance Onion Presentation Transcript

    • 5/19/2011
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2010 . ALL RIGHTS RESERVED
      Michael Mikikian - 1
      Peeling the Web Performance Onion
      Drit Suljoti, Co-founder Catchpoint Systems
      drit@catchpoint.com
    • Speed Matters
      Shopzilla
      Performance 6sec to 1.2 sec
      Revenue +12%
      Pageviews+25%
      Yahoo
      400ms improvement
      Traffic +9%
      Length of an eye blink is ~400ms
    • Speed Matters
      Amazon
      Latency +100ms
      Sales drop 1% / 250mil
      Google
      400ms latency
      Searches drop 0.59%
    • What Impacts Speed
      5/19/2011
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2010 . ALL RIGHTS RESERVED
      4
    • What Impacts Speed
      5/19/2011
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2010 . ALL RIGHTS RESERVED
      5
      Webpage Response (ms): 2,254
      # Requests: 102
      # Hosts: 11
    • Web Performance Optimization
      Reduce
      Pack Tight
      Merge
      Leverage Cache
      Partner Up
    • WPO Works
      5/19/2011
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2010 . ALL RIGHTS RESERVED
      For Ethan Selzer - 7
    • 80 / 20 Rule
      5/19/2011
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2011 . ALL RIGHTS RESERVED
      8
      Steve Souders:
      “80% of the end-user response time is spent
      on the front-end. Start there.”
      When things fail, a single request takes 80% of the time!
    • Great Performance is More than SPEED
      Availability can users access it?
      Reliability is it working consistently and properly?
    • Monitoring
    • Web Performance Monitoring
      Synthetic: simulates End User activity
      Measures the end to end transaction
      Continuous, 24x7
      Detect anomalies
      Track performance metrics over time
      Measures every component
    • The Foundations of the Web
      IP
    • Simple HTTP Transaction
      DNS Lookup: The client tries to resolve the domain name for the request.
      Client sends DNS Query to local ISP DNS server.
      DNS server responds with the IP address for hostname.com
      Connect: Client establishes TCP connection with the IP address of hostname.com
      Client sends SYN packet.
      Web server sends SYN-ACK packet.
      Client answers with ACK packet, concluding the three-way TCP connection establishment.
      Send: Client sends the HTTP request to the web server.
      Wait: Client waits for the server to respond to the request.
      Web server processes the request, finds the resource, and sends the response to the Client. Client receives the first byte of the first packet from the web server, which contains the HTTP Response headers and content.
      Load: Client loads the content of the response.
      Web server sends second TCP segment with the PSH flag set.
      Client sends ACK. (Client sends ACK every two segments it receives. from the host)
      Web server sends third TCP segment with HTTP_Continue.
      Close: Client sends a a FIN packet to close the TCP connection.
    • DNS
    • DNS Recommendations
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2010 . ALL RIGHTS RESERVED
      15
      • Avoid Short TTL if possible.
      • Avoid multiple CNAMEs.
      • Rely on distributed NS server or utilize companies like 3Crowd, Cotendo, Dyn, UltraDNS,...
      • Ensure DNS servers capacity.
      • Avoid using Registrars, usually unreliable and slow.
      • Do not mix Private & Public IP addresses.
    • Use Case: DNS
    • Use Case: DNS
      DNS Provider Switch
    • Connect
      DNS Lookup: The client tries to resolve the domain name for the request.
      Client sends DNS Query to local ISP DNS server.
      DNS server responds with the IP address for hostname.com
      Connect: Client establishes TCP connection with the IP address of hostname.com
      Client sends SYN packet.
      Web server sends SYN-ACK packet.
      Client answers with ACK packet, concluding the three-way TCP connection establishment.
      Send: Client sends the HTTP request to the web server.
      Wait: Client waits for the server to respond to the request.
      Web server processes the request, finds the resource, and sends the response to the Client. Client receives the first byte of the first packet from the web server, which contains the HTTP Response headers and content.
      Load: Client loads the content of the response.
      Web server sends second TCP segment with the PSH flag set.
      Client sends ACK. (Client sends ACK every two segments it receives. from the host)
      Web server sends third TCP segment with HTTP_Continue.
      Close: Client sends a a FIN packet to close the TCP connection.
    • Connect
      What impacts Connect times?
      Packet loss / network stability
      Latency
      Routers
      Load Balancers
      Network Cards
      Server Load
      Common issues:
      Bad configuration of Load Balancers, Routers, Switches and the NIC cards!
      Latency/Packet Loss/Link speed
      TCP offloading
      Capacity
    • Use Case: Connect
    • Use Case: Connect
      Capacity Problems at Datacenter
    • Wait
      DNS Lookup: The client tries to resolve the domain name for the request.
      Client sends DNS Query to local ISP DNS server.
      DNS server responds with the IP address for hostname.com
      Connect: Client establishes TCP connection with the IP address of hostname.com
      Client sends SYN packet.
      Web server sends SYN-ACK packet.
      Client answers with ACK packet, concluding the three-way TCP connection establishment.
      Send: Client sends the HTTP request to the web server.
      Wait: Client waits for the server to respond to the request.
      Web server processes the request, finds the resource, and sends the response to the Client. Client receives the first byte of the first packet from the web server, which contains the HTTP Response headers and content.
      Load: Client loads the content of the response.
      Web server sends second TCP segment with the PSH flag set.
      Client sends ACK. (Client sends ACK every two segments it receives. from the host)
      Web server sends third TCP segment with HTTP_Continue.
      Close: Client sends a a FIN packet to close the TCP connection.
    • Wait
      What impacts Wait times?
      Network factors
      The Application code
      The Application Load / Capacity
      Any other dependencies (database lookups, other servers, apis…)
      Common issues:
      Application Load - Profile your capacity / perform load testing
      Database utilization, locks…,
      Memory Leaks, Bugs…
    • Use Case: Wait
    • Use Case: Wait
    • Use Case: Wait - Database Impact
      DB Problems
    • Load/Receive
      DNS Lookup: The client tries to resolve the domain name for the request.
      Client sends DNS Query to local ISP DNS server.
      DNS server responds with the IP address for hostname.com
      Connect: Client establishes TCP connection with the IP address of hostname.com
      Client sends SYN packet.
      Web server sends SYN-ACK packet.
      Client answers with ACK packet, concluding the three-way TCP connection establishment.
      Send: Client sends the HTTP request to the web server.
      Wait: Client waits for the server to respond to the request.
      Web server processes the request, finds the resource, and sends the response to the Client. Client receives the first byte of the first packet from the web server, which contains the HTTP Response headers and content.
      Load: Client loads the content of the response.
      Web server sends second TCP segment with the PSH flag set.
      Client sends ACK. (Client sends ACK every two segments it receives. from the host)
      Web server sends third TCP segment with HTTP_Continue.
      Close: Client sends a a FIN packet to close the TCP connection.
    • Load/Receive
      What impacts Wait times?
      Network factors
      The Application code
      The Application Load / Capacity
      Any other dependencies (database lookups, other servers, apis…)
      Common issues:
      Application Load - Profile your capacity / Load testing
      Database utilization, locks…,
      Memory Leaks, Bugs…
      Bad chunked encoding!
    • Factors You Control
      Applications/System
      Architecture, Code, DB, SQL Queries, …
      Configurations
      Partners:
      DNS, CDN, ISP Links, Datacenter, Optimization, Ads…
      ISP Peering
      Hardware
      Server, CPU, RAM, I/O, Routers,…
      5/19/2011
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2010 . ALL RIGHTS RESERVED
      For Ethan Selzer - 29
    • Factors You CANNOT Control
      Internet
      Fiber cut in DC, Latency East Coast to West Coast
      Last Mile
      The internet link to the end user!
      End User Local Network
      Misconfigured router, computers downloading/uploading torrents!
      OS/Browser of the end user
      Configurations, Add-ons, installed malware…
      5/19/2011
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2010 . ALL RIGHTS RESERVED
      For Ethan Selzer - 30
    • Parallel & Persistent HTTP Connections
      • Connect – Once per connection
      • DNS – Once or less per transaction per hostname
    • Parallel & Persistent HTTP Connections
      http://www.webpagetest.org
    • HTML Content
      Each request relies on HTTP - same rules apply!
      Pay attention to bottlenecks
      Primary URL
      Inline JavaScript requests
      Front end code impacts browser processing!
    • Oops … bad httpd.conf file
      5/19/2011
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2011 . ALL RIGHTS RESERVED
      34
    • Oops … bad httpd.conf file
      5/19/2011
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2011 . ALL RIGHTS RESERVED
      35
      Apache configuration!
      No More Keep Alives
    • Monitoring Best Practices
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2011 . ALL RIGHTS RESERVED
      36
      Monitor every component:
      • Key webpages with real browser
      • Setup monitoring for every key provider / hostname / requests
      • DNS
      • Origin servers
      • if Multiple datacenters, monitor them individually
      Baseline very often, track improvement or degradation
      Keep an eye on performance during critical business hours (Volume)
      Compare data pre-release and post release
      Review Daily / Weekly /Monthly
      Alerts on % delta
    • Troubleshooting Tool Box
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2011 . ALL RIGHTS RESERVED
      37
      Standalone
      • Chrome – Developer tool
      • Firebug
      • IE 9 – Developer tool
      • HTTPWatch
      • Wireshark
      • Fiddler
      • Yslow
      • Dynatrace
      • Dig for DNS
      • MTR (Matt traceroute)
      Online:
      • WebPageTest
      • Google Page speed (http://pagespeed.googlelabs.com/)

    • Closing the Loop
      Monitor Action Plan  Optimize Monitor… do it again
      It isn’t over till it’s over.
      And it’s never over.
      It is not a sprint, it is a never ending marathon!
    • CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2011 . ALL RIGHTS RESERVED
      39
      THANK YOU.
      http://blog.catchpoint.com
      @catchpoint
    • Troubleshooting: My 6 Best Friends
      CATCHPOINT SYSTEMS . CONFIDENTIAL . © 2010 . ALL RIGHTS RESERVED
      40
      Rudyard Kipling: “I keep six honest serving-men (They taught me all I knew);Their names are What and Why and When And How and Where and Who.”
      • What is there a problem? Is it one monitoring tool or multiple that turned red? is it only one node or multiple monitoring nodes in the case of a 3rd party tool? Is it the DB?…
      • Where is the problem? is it specific to a country? a city? a customer? a datacenter?
      • Who is affected? All customers? Some customers? Just the monitoring tools?
      • When did it start? Does it always happen? is it sporadic?
      • Why?Did something change? Did we roll out something? Did the environment change? Did the customer behavior change?
      • How did this happen? Change management? Router upgrade? Did someone upload a crazy ad?