Your SlideShare is downloading. ×
Who cares about customer
         experience?
         A Complete
         Web Monitoring
         perspective




Wednesd...
“Hard” data

             Analytics                  Usability     Performability
           (what did they            (ho...
What did they do?
         Web analytics




Wednesday, November 11, 2009
http://www.flickr.com/photos/diegocupolo/3511117614/
Wednesday, November 11, 2009
Accounting                                     Optimization


                                                            ...
How did they do it?
         Web Interaction Analytics




Wednesday, November 11, 2009
http://www.flickr.com/photos/trekkyandy/189717616/

Wednesday, November 11, 2009
Why did they do it?
         Voice of the Customer




Wednesday, November 11, 2009
http://www.flickr.com/photos/karola/3623768629/
Wednesday, November 11, 2009
Could they do it?
         Performance & availability




Wednesday, November 11, 2009
Wednesday, November 11, 2009
What were they saying?
         Community monitoring




Wednesday, November 11, 2009
!
Wednesday, November 11, 2009
What were they up to?
         Competitive analysis




Wednesday, November 11, 2009
http://www.flickr.com/photos/31690139@N02/2965956581/
Wednesday, November 11, 2009
Why here? Why now?
         Online business is becoming the default.




Wednesday, November 11, 2009
!8B                                         • In 2006 !3B online retail
                                 !1.1
            ...
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Online marketers want to
                  maximize their revenues.



Wednesday, November 11, 2009
http://www.davidross.com.mx/admin/wideimagerepository/sergioZymanAmp.jpg
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Wednesday, November 11, 2009
http://www.goal-posts.net/media/fbl-181_cat_07_160_x_160.jpg
Wednesday, November 11, 2009
http://www.johnsoncontrols.com/publish/etc/medialib/jci/ps/Media_Kit.Par.25973.File.dat/complex_machinery_HIRES.JPG
Wednes...
http://www.infovisual.info/02/img_en/078%20Different%20land%20animals%201.jpg

Wednesday, November 11, 2009
http://upload.wikimedia.org/wikipedia/commons/0/0e/Soccer_Youth_Goal_Keeper.jpg
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Some sites have ungoals.
         Things you really don’t want visitors to do.




Wednesday, November 11, 2009
You really don’t want web
                    users to call you.
          US$15


          US$12


            US$9


  ...
Websites
    have a dirty
    little secret


   http://todaystatus.files.wordpress.com/2009/04/ww11-secret.jpg
Wednesday, ...
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Wednesday, November 11, 2009
http://www.inquisitr.com/2097/site-meter-causing-internet-explorer-failure/



Wednesday, November 11, 2009
http://www.flickr.com/photos/aleermakers/3455786409/
Wednesday, November 11, 2009
Wednesday, November 11, 2009
http://www.octulipfestival.com/images/Picture%20144.jpg
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Figure 3          Interactive user productivity versus computer response time for human-intensive
                        ...
10 s
                                1s

                               100 ms
                               10 ms
      ...
Wednesday, November 11, 2009
Wednesday, November 11, 2009
http://www.flickr.com/photos/spunter/393793587   http://www.flickr.com/photos/laurenclose/2217307446




Wednesday, November...
!"#$%&'()*+(,+()&#$-."#)/
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Shopzilla had another angle

         •      Big, high-traffic site          •   16 month re-engineering
         •      10...
5-12% increase in
                               revenue.


Wednesday, November 11, 2009
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Tying web latency to
                     business outcomes.


Wednesday, November 11, 2009
http://www.flickr.com/photos/spunter/393793587   http://www.flickr.com/photos/laurenclose/2217307446




                   ...
Wednesday, November 11, 2009
http://www.flickr.com/photos/mrmoorey/160654236




Wednesday, November 11, 2009
Wednesday, November 11, 2009
ATTENTION                              ENGAGEMENT     CONVERSION

                                 NEW
                   ...
Wednesday, November 11, 2009
VISITOR         STRANGELOOP                WEB
                                               ACCELERATOR               SE...
What we learned:



Wednesday, November 11, 2009
Traffic levels
                             9,000
    Total number of visits




                             6,750



    ...
Bounce rate
                           20



                           15
     Visits that bounced




                  ...
% visits marked “new”
 % of visits that had no returning cookie



                                            14



     ...
That means...
                           9000
  Value Number of visits




                           6750



            ...
Average time on site
                             31
    Time on site (minutes)




                             23



   ...
Pages per visit
                         16



                         12
    Average pages seen




                    ...
Conversion rate
                                       20
                                            and order value
    ...
This is just one case
              LOTS


            # OF
          VISITS

                               OPTIMIZED
   ...
With one outcome
              LOTS


            # OF
          VISITS
                               21.58%
            ...
With one outcome
              LOTS

                               Best 5%            Worst 5%
            # OF
         ...
Lots of different results
              LOTS             24%

                                     18%
            $ PER  ...
You have your own curve
              LOTS


            $ PER
              DAY


                  0
                   ...
“What’s a latency?”




Wednesday, November 11, 2009
!"#$%&'(')*$+#,$%&)-.)/01-
     Browser                                                                      Data center

...
Analytics site

                                              Server


                                                  D...
21--#/01-.$%1$*1)3$4'%/5&&#-%
       !"##$%&"#'(')'***+,-.%/00$#.+%"1'234+5(6+36+758
       !"##$%&"#'7')'***+,-.%/00$#.+%...
6")%$/15*3$!"##$%&'$71$+&1-78
       1#$2.$34)5#                            D"4(@4G(@(&%
       • 6789+J-#B'.C$'=-.$      ...
Wednesday, November 11, 2009
How do we measure it?




Wednesday, November 11, 2009
Test                  Testing
        config                   node

                                         Data center
 ...
Synthetic isn’t enough




Wednesday, November 11, 2009
Browser                                    Load      Web
                               Network      balancer   server
   ...
Justifying an investment in
                       performance

          Current
                    *
                  ...
Justifying an investment in
                       performance

      $10,000 *                (   0.1607 + 0.0551   )   =...
Conclusions
         • Links between performance and business KPIs
                are undeniable

         • By talking t...
Putting it all together
         The Big Picture




Wednesday, November 11, 2009
“Hard” data

             Analytics                  Usability     Performability
           (what did they            (ho...
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Wednesday, November 11, 2009
Thanks!
     alistair@bitcurrent.com               sean@httpd.org
     @acroll                               @seanpower
  ...
Upcoming SlideShare
Loading in...5
×

Web Performance: Who Cares About Customer Experience? : Sean Power and Alistair Croll

1,739

Published on

This was presented on the MeasureWorks event in Baarn on 12 November 2009 by Sean Power and Alistair Croll. More Event Info:
www.measureworks-event.nl and www.measureworks.nl

Published in: Business, Technology, Design
0 Comments
12 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,739
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
32
Comments
0
Likes
12
Embeds 0
No embeds

No notes for slide
  • here today to tell you that your website has a dirty little secret
    your website is trying really, really hard to hide things from you . . and they’re not necessarily small things either.
  • for example . . . imagine that you decided to launch a kick ass survey. you’ve bought the latest shiny tool
    you’ve carefully crafted the questions
    you hired outside help to make sure they’re worded properly
    you had them sent to a professional copy editor to get the final tone just right
    it went through legal
    you segmented your campaign according to the demographic whose voice you need to understand the most

    and as you sit precariously over the big red send button you can’t help but feel that you’ve covered all your bases.
    Satisfied, you press the button and out it goes into the world.
  • that was the case with paypal, recently.
    We don’t have insight into their numbers, so we can’t tell for sure what the particular conversion rate for this survey was, but we suspect that the pickup wasn’t as good as anticipated.

    Their web analytics and VOC don’t have the necessary functions built in to determine that their SSL cert was mismatched, cause safari and other browsers to come up with a nasty message saying “we can’t verify the identity of paypal-surveys.com”.
    After all, think about it; if it’s coming from paypal and the identity can’t be verified, would you go on the site and fill anything out?
  • we know of a case of a marketing officer who’se job was put in question because of a string of failed campaigns.
    The company jumped the gun on this one. Thanks to a friend in the web operations department, he was able to show that the network was at fault. Even though the company load tested diligently, they only did from their internal network. It turns out the problems were related to the last mile - something that was hidden until the company implemented synthetic monitoring.

    Even though overall sentiment was a little more negative than usual during the campaigns, the conversion rates skyrocketed once better transit was installed.
  • This is a scary one and a true. If you haven’t heard, sitemeter took down every single website that were a client of theirs. If you were on IE and wanted to access sites like TechCrunch, Gizmodo and so on, you were out of luck in August, because the code crashed the browser.

    Think about it - your site isn’t just vulnerable to whatever goofy code your development team throws at the Internet, it’s also vulnerable to your very own web analytics tracking codes!

    This would take hours of troubleshooting to reveal without synthetic monitoring - or one simply alert would be triggered with the proper tools in place.

    I don’t mean to pick on SiteMeter btw, I’m sure they have a great service - but these types of errors can kill substantial amounts of revenue until you catch it.
  • You may be able to tie performance to business results such as key steps in a transaction
  • Or see what people think of your site (based on surveys) by page within analytics.
  • Once upon a time, performance was a dark art. We struggled to deliver “good enough” without really knowing why.
  • We managed by anecdote. We were sure faster was better, but we couldn’t tie it to specific business outcomes.
  • The notion that speed is good for users isn’t new. The concept of “Flow” – a state of heightened engagement that we experience when we’re truly focused on something – was first proposed by mihaly csikszentmihalyi
  • It turns out that attention and engagement drop off predictably. At ten milliseconds, we actually believe something is physically accessible – think clicking a button and seeing it change color. At 100 milliseconds, we can have a conversation with someone without noticing the delay (remember old transatlantic calls?) At a second, we’re still engaged, but aware of the delay. At ten seconds, we get bored and tune out, because other things come into our minds.
  • It turns out that attention and engagement drop off predictably. At ten milliseconds, we actually believe something is physically accessible – think clicking a button and seeing it change color. At 100 milliseconds, we can have a conversation with someone without noticing the delay (remember old transatlantic calls?) At a second, we’re still engaged, but aware of the delay. At ten seconds, we get bored and tune out, because other things come into our minds.
  • It turns out that attention and engagement drop off predictably. At ten milliseconds, we actually believe something is physically accessible – think clicking a button and seeing it change color. At 100 milliseconds, we can have a conversation with someone without noticing the delay (remember old transatlantic calls?) At a second, we’re still engaged, but aware of the delay. At ten seconds, we get bored and tune out, because other things come into our minds.
  • It turns out that attention and engagement drop off predictably. At ten milliseconds, we actually believe something is physically accessible – think clicking a button and seeing it change color. At 100 milliseconds, we can have a conversation with someone without noticing the delay (remember old transatlantic calls?) At a second, we’re still engaged, but aware of the delay. At ten seconds, we get bored and tune out, because other things come into our minds.
  • It turns out that attention and engagement drop off predictably. At ten milliseconds, we actually believe something is physically accessible – think clicking a button and seeing it change color. At 100 milliseconds, we can have a conversation with someone without noticing the delay (remember old transatlantic calls?) At a second, we’re still engaged, but aware of the delay. At ten seconds, we get bored and tune out, because other things come into our minds.
  • How much was fast enough? It was anybody’s guess.
  • And guess they did.
    This is Zona’s formula for patience, the basis for the “eight second rule.” Unfortunately, things like tenacity, importance, and natural patience aren’t concrete enough for the no-nonsense folks that run web applications.
  • And guess they did.
    This is Zona’s formula for patience, the basis for the “eight second rule.” Unfortunately, things like tenacity, importance, and natural patience aren’t concrete enough for the no-nonsense folks that run web applications.
  • IT operators and marketers are completely different people. What convinces an IT person to fix performance doesn’t convince a marketer. They want to know how it will impact the business fundamentals.
  • By now, we know that everything matters. Usability, page latency, visitor mindset, and even sentiment on social media platforms all contribute to the business results you get from a site.
  • Fortunately, we’re getting better at linking performance to business outcomes.
  • One example of this is performance experimentation that Google’s done. Google’s a perfect lab. Not only do they have a lot of traffic, they also have computing resources to do back-end analysis of large data sets. Plus, they’re not afraid of experimentation – in fact, they insist on it. So they tried different levels of performance and watched what happened to visitors.
  • The results, which they presented at Velocity in May, were fascinating. There was a direct impact between delay and the number of searches a user did each day – and to make matters worse, the numbers often didn’t improve even when the delay was removed. You may think 0.7% drop isn’t significant, but for Google this represents a tremendous amount of revenue.
  • Microsoft’s Bing site is a good lab, too. They looked at key metrics, or KPIs, of their search site.
  • They showed that as performance got worse, all key metrics did, too. Not just the number of searches, but also the revenue (earned when someone clicks) and refinement of searches.
  • Shopzilla overhauled their entire site, dramatically reducing page load time, hardware requirements, and downtime.
  • They saw a significant increase in revenues
  • The site improvement increased the number of Google clicks that turned into actual visits
  • It also affected search engine scores. By improving load time, search engines (in this case Google UK) “learned” that this was a good destination. That’s right – Google actually penalizes sites that are slow by giving them a lower page ranking.
  • If we want to convince marketing, we need to measure business metrics.
  • By tying performance and availability to Key Performance Indicators – KPIs – business and operations can finally have a conversation.
  • The first kind of site is a media site.
  • Transactional sites make money when people complete a process, usually involving a purchase or subscription.
  • Collaboration sites (like wikis, news aggregators, user groups, social news aggregators) are those where the value of the site is largely determined by the content users generate.
  • Finally, Software as a Service sites are ones where a subscribers expect reliability after paying a monthly fee for a service
  • Whether those KPIs are shopping cart abandonment
  • Or visitor “bounce rate” (the number of visitors that leave immediately)
  • Or just traffic.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they’re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It’s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they’re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It’s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they’re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It’s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they’re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It’s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they’re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It’s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they’re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It’s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they’re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It’s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they’re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It’s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they’re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It’s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they’re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It’s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they’re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It’s what the business cares about.
  • Strangeloop agreed to set up an experiment using their technology which would help measure this.
  • First, traffic. Despite splitting visitors to be optimized and unoptimized evenly, we had many more optimized sessions captured by the analytics. This may be a result of slower-loading pages failing to execute the analytics script, or abandoning the visit before the page had time to load.
  • Unoptimized visitors are roughly 1% more likely to leave the site immediately, without proceeding to other pages.
  • Strangely, the unoptimized visitors consisted of more new visitors than the optimized ones did. This seems counter-intuitive and warrants further study.
  • Optimized visitors spent more time on the site
  • And looked at more pages during their visit – if you’re a media property, this means more impressions for your advertisers.
  • On a second e-commerce site running roughly the same experiment, conversions were 16 percent higher and orders were 5.5% higher.
  • slow performance hurts
  • A site that’s unresponsive or plagued by incidents and unpredictable availability has lower conversion rates.
    Sites that deliver a consistently poor end user experience are less likely to attract a loyal following. Poor site performance may also affect perception of your company’s brand or reputation.
    You may be liable for damages if you can’t handle transactions promptly, particularly if you’re in a heavily regulated industry such as finance or healthcare.
    Poor performance may cost you money. If you have a formal contract with users, you may be liable for refunds or service credits. Slow or unavailable sites also encourage customers to find other channels, such as phone support or retail outlets, that cost your organization far more than handling requests via the Web. Once visitors try those channels, they may stick with them, costing you even more money.
  • Time for a crash course in web performance
    (there are smarter people than us on this topic but we’ll try.)
  • Each time a visitor comes to your site, she must first look up the domain with a DNS service
  • Each time a visitor comes to your site, she must first look up the domain with a DNS service
  • Each time a visitor comes to your site, she must first look up the domain with a DNS service
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it’s of any real size, then there’s a thing called a load balancer at the other end.
    These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it’s of any real size, then there’s a thing called a load balancer at the other end.
    These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it’s of any real size, then there’s a thing called a load balancer at the other end.
    These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it’s of any real size, then there’s a thing called a load balancer at the other end.
    These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it’s of any real size, then there’s a thing called a load balancer at the other end.
    These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it’s of any real size, then there’s a thing called a load balancer at the other end.
    These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it’s of any real size, then there’s a thing called a load balancer at the other end.
    These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it’s of any real size, then there’s a thing called a load balancer at the other end.
    These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it’s of any real size, then there’s a thing called a load balancer at the other end.
    These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it’s of any real size, then there’s a thing called a load balancer at the other end.
    These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it’s of any real size, then there’s a thing called a load balancer at the other end.
    These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it’s of any real size, then there’s a thing called a load balancer at the other end.
    These packets are called IP, or Internet Protocol, packets.
  • That connection takes a bit of explaining.
    Think about the postal service. You don’t need to know how it works to send mail. There’s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining.
    Think about the postal service. You don’t need to know how it works to send mail. There’s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining.
    Think about the postal service. You don’t need to know how it works to send mail. There’s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining.
    Think about the postal service. You don’t need to know how it works to send mail. There’s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining.
    Think about the postal service. You don’t need to know how it works to send mail. There’s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining.
    Think about the postal service. You don’t need to know how it works to send mail. There’s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining.
    Think about the postal service. You don’t need to know how it works to send mail. There’s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining.
    Think about the postal service. You don’t need to know how it works to send mail. There’s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining.
    Think about the postal service. You don’t need to know how it works to send mail. There’s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining.
    Think about the postal service. You don’t need to know how it works to send mail. There’s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining.
    Think about the postal service. You don’t need to know how it works to send mail. There’s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You’d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn’t be able to reassemble the pile of letters that came through their door.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • So maybe you’d get clever, and start numbering the letters. That way your receiver would know what order they’re in.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • The system doesn’t work so well, however, if something’s lost along the way.
  • What you’d need is a system of numbered envelopes, plus a system that let the receiver tell you when they were missing something. With these two things working properly, you’d simulate an end-to-end connection between yourself and your receiver, and you could send long messages reliably even though only one word fit in each envelope.
  • What you’d need is a system of numbered envelopes, plus a system that let the receiver tell you when they were missing something. With these two things working properly, you’d simulate an end-to-end connection between yourself and your receiver, and you could send long messages reliably even though only one word fit in each envelope.
  • What you’d need is a system of numbered envelopes, plus a system that let the receiver tell you when they were missing something. With these two things working properly, you’d simulate an end-to-end connection between yourself and your receiver, and you could send long messages reliably even though only one word fit in each envelope.
  • What you’d need is a system of numbered envelopes, plus a system that let the receiver tell you when they were missing something. With these two things working properly, you’d simulate an end-to-end connection between yourself and your receiver, and you could send long messages reliably even though only one word fit in each envelope.
  • This is how the TCP/IP stack works. IP sends packets of data; TCP makes sure they get there.
    TCP creates an end-to-end connection between your browser and a server.
  • This is how the TCP/IP stack works. IP sends packets of data; TCP makes sure they get there.
    TCP creates an end-to-end connection between your browser and a server.
  • This is how the TCP/IP stack works. IP sends packets of data; TCP makes sure they get there.
    TCP creates an end-to-end connection between your browser and a server.
  • This is how the TCP/IP stack works. IP sends packets of data; TCP makes sure they get there.
    TCP creates an end-to-end connection between your browser and a server.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • You can try this out yourself.
  • You can try this out yourself.
  • You can try this out yourself.
  • You can try this out yourself.
  • You can try this out yourself.
  • If you’re loading static content -- an image, for example -- the web server can deliver it.
  • If you’re loading static content -- an image, for example -- the web server can deliver it.
  • If you’re loading static content -- an image, for example -- the web server can deliver it.
  • If you’re loading static content -- an image, for example -- the web server can deliver it.
  • If it’s a dynamic object, such as a custom page, then the app server responds.
  • If it’s a dynamic object, such as a custom page, then the app server responds.
  • If it’s a dynamic object, such as a custom page, then the app server responds.
  • If it’s a dynamic object, such as a custom page, then the app server responds.
  • If it’s a dynamic object, such as a custom page, then the app server responds.
  • If it’s a dynamic object, such as a custom page, then the app server responds.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection.
    On top of TCP, there’s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • The time a page takes to load is a function of all the objects it contains, which are retrieved from the server -- more objects, or a longer time to load, makes a slower page load.
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • Websites have more and more connections, too...
  • So what could possibly go wrong?
  • This makes visitors sad and lonely
  • Sites still fail in lots of ways. It’s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It’s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It’s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It’s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It’s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It’s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It’s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It’s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It’s scary how much things break. This is just a sample of pages for Canadians...
  • So how do we check for this stuff? One way to do testing is to use internal systems to run tests.
  • So how do we check for this stuff? One way to do testing is to use internal systems to run tests.
  • So how do we check for this stuff? One way to do testing is to use internal systems to run tests.
  • If you have a load balancer, you can use that too.
  • If you have a load balancer, you can use that too.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there’s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there’s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there’s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there’s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there’s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there’s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there’s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there’s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there’s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there’s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there’s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • A more advanced approach is what we call “browser puppetry” -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call “browser puppetry” -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call “browser puppetry” -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call “browser puppetry” -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call “browser puppetry” -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call “browser puppetry” -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call “browser puppetry” -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call “browser puppetry” -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call “browser puppetry” -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call “browser puppetry” -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call “browser puppetry” -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call “browser puppetry” -- which uses a real browser to request and retrieve information.
  • Synthetic tests show you whether your site is working; but to tell whether it’s BROKEN you need to look at actual visitors. Here’s a chart of visits over time. <click> see the synthetic tests? See the gaps when other things happen?
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • there’s a much larger analytics ecosystem that exists. Yikes!
  • So if your brain’s full, don’t be surprised. There’s a huge amount of data to consolidate.
  • it’s also important to correlate data. In this case, we can see that there is a clear relationship between latency and conversion.
  • and often, it’s very useful to segment your data in particular ways.
    In this case, there’s a correlation between the AVG page load time across visit and the propensity that a user will post a comment
  • In the end, this is what it should all look like.
  • Ultimately, you want a single, comprehensive view of your web presence across all of these platforms in order to make good decisions and communicate what you’re doing to the rest of the organization.
  • Thanks for the time. If you want to reach us or follow us online, here’s how. And if you want to pre-order our book (which would freak out our editors) we’d love it. We’re shameless self-promoters.
  • Transcript of "Web Performance: Who Cares About Customer Experience? : Sean Power and Alistair Croll "

    1. 1. Who cares about customer experience? A Complete Web Monitoring perspective Wednesday, November 11, 2009
    2. 2. “Hard” data Analytics Usability Performability (what did they (how did they (could they do do on the interact with what they site?) it?) wanted to?) Complete Web Monitoring Community VoC Competition (what were (what were (what are they they saying?) their up to?) motivations?) “Soft” data Wednesday, November 11, 2009
    3. 3. What did they do? Web analytics Wednesday, November 11, 2009
    4. 4. http://www.flickr.com/photos/diegocupolo/3511117614/ Wednesday, November 11, 2009
    5. 5. Accounting Optimization http://www.flickr.com/photos/sanchom/2963072255/ http://www.flickr.com/photos/thomasclaveirole/538819881/ Wednesday, November 11, 2009
    6. 6. How did they do it? Web Interaction Analytics Wednesday, November 11, 2009
    7. 7. http://www.flickr.com/photos/trekkyandy/189717616/ Wednesday, November 11, 2009
    8. 8. Why did they do it? Voice of the Customer Wednesday, November 11, 2009
    9. 9. http://www.flickr.com/photos/karola/3623768629/ Wednesday, November 11, 2009
    10. 10. Could they do it? Performance & availability Wednesday, November 11, 2009
    11. 11. Wednesday, November 11, 2009
    12. 12. What were they saying? Community monitoring Wednesday, November 11, 2009
    13. 13. ! Wednesday, November 11, 2009
    14. 14. What were they up to? Competitive analysis Wednesday, November 11, 2009
    15. 15. http://www.flickr.com/photos/31690139@N02/2965956581/ Wednesday, November 11, 2009
    16. 16. Why here? Why now? Online business is becoming the default. Wednesday, November 11, 2009
    17. 17. !8B • In 2006 !3B online retail !1.1 sales in the Netherlands !6B !1.7 • In 2011 !12B in total online sales in Benelux, with more than half of !1.9 !4B the Dutch adult population shopping online !2B !3.3 • In 2014 Forrester estimates over !8B in !0B online retail sales in the Netherlands Other Leisure flights Consumer electronics Hotels Wednesday, November 11, 2009
    18. 18. Wednesday, November 11, 2009
    19. 19. Wednesday, November 11, 2009
    20. 20. Wednesday, November 11, 2009
    21. 21. Online marketers want to maximize their revenues. Wednesday, November 11, 2009
    22. 22. http://www.davidross.com.mx/admin/wideimagerepository/sergioZymanAmp.jpg Wednesday, November 11, 2009
    23. 23. Wednesday, November 11, 2009
    24. 24. Wednesday, November 11, 2009
    25. 25. Wednesday, November 11, 2009
    26. 26. http://www.goal-posts.net/media/fbl-181_cat_07_160_x_160.jpg Wednesday, November 11, 2009
    27. 27. http://www.johnsoncontrols.com/publish/etc/medialib/jci/ps/Media_Kit.Par.25973.File.dat/complex_machinery_HIRES.JPG Wednesday, November 11, 2009
    28. 28. http://www.infovisual.info/02/img_en/078%20Different%20land%20animals%201.jpg Wednesday, November 11, 2009
    29. 29. http://upload.wikimedia.org/wikipedia/commons/0/0e/Soccer_Youth_Goal_Keeper.jpg Wednesday, November 11, 2009
    30. 30. Wednesday, November 11, 2009
    31. 31. Wednesday, November 11, 2009
    32. 32. Wednesday, November 11, 2009
    33. 33. Wednesday, November 11, 2009
    34. 34. Wednesday, November 11, 2009
    35. 35. Wednesday, November 11, 2009
    36. 36. Wednesday, November 11, 2009
    37. 37. Some sites have ungoals. Things you really don’t want visitors to do. Wednesday, November 11, 2009
    38. 38. You really don’t want web users to call you. US$15 US$12 US$9 US$6 $5.50 $3.00 US$3 $0.24 $0.45 US$0 Web self-service IVR Email Live phone Cost estimates BiT Group White Paper: “Web Self-Service Lowers Call Center Costs and Improves Customer Service” Low Average High Wednesday, November 11, 2009
    39. 39. Websites have a dirty little secret http://todaystatus.files.wordpress.com/2009/04/ww11-secret.jpg Wednesday, November 11, 2009
    40. 40. Wednesday, November 11, 2009
    41. 41. Wednesday, November 11, 2009
    42. 42. Wednesday, November 11, 2009
    43. 43. http://www.inquisitr.com/2097/site-meter-causing-internet-explorer-failure/ Wednesday, November 11, 2009
    44. 44. http://www.flickr.com/photos/aleermakers/3455786409/ Wednesday, November 11, 2009
    45. 45. Wednesday, November 11, 2009
    46. 46. http://www.octulipfestival.com/images/Picture%20144.jpg Wednesday, November 11, 2009
    47. 47. Wednesday, November 11, 2009
    48. 48. Figure 3 Interactive user productivity versus computer response time for human-intensive interactions for system A E 600 - 3 T -" INTERACTIVE USER PRODUCTIVITY (IUP) w -HUMAN-INTENSIVE COMPONENT OF IUP 7 MEASURED DATA (HUMAN-INTENSIVE E 500 - A z " COMPONENT) U E - w E 0 > - > - - 400 3 n F 2 0 0 300 - 200 - 100 - 0 0- I 1 I I I 0 1 2 3 4 5 COMPUTER RESPONSE TIME (SI (1981) A. J. Thadhani, IBM Systems Journal, Volume 20, number 4 Wednesday, November 11, 2009
    49. 49. 10 s 1s 100 ms 10 ms ! Zzz Wednesday, November 11, 2009
    50. 50. Wednesday, November 11, 2009
    51. 51. Wednesday, November 11, 2009
    52. 52. http://www.flickr.com/photos/spunter/393793587 http://www.flickr.com/photos/laurenclose/2217307446 Wednesday, November 11, 2009
    53. 53. !"#$%&'()*+(,+()&#$-."#)/ Wednesday, November 11, 2009
    54. 54. Wednesday, November 11, 2009
    55. 55. Shopzilla had another angle • Big, high-traffic site • 16 month re-engineering • 100M impressions a day • Page load from 6 seconds to 1.2 • 8,000 searches a second • 20-29M unique visitors a • Uptime from 99.65% to month 99.97% • 100M products • 10% of previous hardware needs http://en.oreilly.com/velocity2009/public/schedule/detail/7709 Wednesday, November 11, 2009
    56. 56. 5-12% increase in revenue. Wednesday, November 11, 2009
    57. 57. Wednesday, November 11, 2009
    58. 58. Wednesday, November 11, 2009
    59. 59. Tying web latency to business outcomes. Wednesday, November 11, 2009
    60. 60. http://www.flickr.com/photos/spunter/393793587 http://www.flickr.com/photos/laurenclose/2217307446 KPIs Wednesday, November 11, 2009
    61. 61. Wednesday, November 11, 2009
    62. 62. http://www.flickr.com/photos/mrmoorey/160654236 Wednesday, November 11, 2009
    63. 63. Wednesday, November 11, 2009
    64. 64. ATTENTION ENGAGEMENT CONVERSION NEW VISITORS SEARCHES GROWTH CONVERSION PAGES TIME RATE TWEETS NUMBER OF VISITS PER ON 0 MENTIONS VISIT SITE ORDER ADS SEEN LOSS VALUE BOUNCE RATE Wednesday, November 11, 2009
    65. 65. Wednesday, November 11, 2009
    66. 66. VISITOR STRANGELOOP WEB ACCELERATOR SERVER Decide whether to optimize Normal Receive Accelerated content page Insert Process scripts Optimize? segment marker Send analytics Unaccelerated GOOGLE ANALYTICS Wednesday, November 11, 2009
    67. 67. What we learned: Wednesday, November 11, 2009
    68. 68. Traffic levels 9,000 Total number of visits 6,750 4,500 8,505 2,250 4,740 0 Optimized Unoptimized Visitor experience Wednesday, November 11, 2009
    69. 69. Bounce rate 20 15 Visits that bounced 10 13.38% 14.35% 5 0 Optimized Unoptimized Visitor experience Wednesday, November 11, 2009
    70. 70. % visits marked “new” % of visits that had no returning cookie 14 11 7 13.61% 10.85% 4 0 Optimized Unoptimized Visitor experience Wednesday, November 11, 2009
    71. 71. That means... 9000 Value Number of visits 6750 4500 7,582 4,095 2250 923 645 0 Optimized Unoptimized Wednesday, November 11, 2009
    72. 72. Average time on site 31 Time on site (minutes) 23 16 30.17 23.83 8 0 Optimized Unoptimized Visitor experience Wednesday, November 11, 2009
    73. 73. Pages per visit 16 12 Average pages seen 8 15.64 11.04 4 0 Optimized Unoptimized Visitor experience Wednesday, November 11, 2009
    74. 74. Conversion rate 20 and order value Difference due to optimization 15 10 16.07 5 5.51 0 Conversion rate Order value Wednesday, November 11, 2009
    75. 75. This is just one case LOTS # OF VISITS OPTIMIZED 0 0 VISITOR LATENCY 10,000 Different visitors experienced different performance levels. Wednesday, November 11, 2009
    76. 76. With one outcome LOTS # OF VISITS 21.58% 0 BETTER 0 VISITOR LATENCY 10,000 Right now we have a single experiment, and a single resulting business impact. Wednesday, November 11, 2009
    77. 77. With one outcome LOTS Best 5% Worst 5% # OF VISITS 21.58% 0 BETTER 0 VISITOR LATENCY 10,000 Visitors who were optimized fall into a range – the 5th to 95th percentile. Wednesday, November 11, 2009
    78. 78. Lots of different results LOTS 24% 18% $ PER 14% 12% DAY 9.5% 0 0 VISITOR LATENCY 10,000 If we have several experiments, we can understand the relationship better. Wednesday, November 11, 2009
    79. 79. You have your own curve LOTS $ PER DAY 0 0 VISITOR LATENCY 10,000 Every web business has a curve like this hidden inside it. Wednesday, November 11, 2009
    80. 80. “What’s a latency?” Wednesday, November 11, 2009
    81. 81. !"#$%&'(')*$+#,$%&)-.)/01- Browser Data center Server TCP SYN (“let’s talk”) TCP SYN ACK (“Agreed: let’s talk”) TCP ACK (“OK, we’re talking) SSL (“Someone might be listening!”) SSL (“Here’s a decoder ring”) HTTP GET / (“Can I have your home page?”) HTTP 200 OK (“Sure!”) (thinks [index.html] (“Here it is!”) a bit) (Renders furiously) Bump, bump. [img js css] (“Have this too!”) TCP FIN (“Thanks! I’m done now.”) TCP FIN ACK (“You’re welcome. Have a nice day.”) Wednesday, November 11, 2009
    82. 82. Analytics site Server Data center Browser Server Snurken Mashup site Server Wednesday, November 11, 2009
    83. 83. 21--#/01-.$%1$*1)3$4'%/5&&#-% !"##$%&"#'(')'***+,-.%/00$#.+%"1'234+5(6+36+758 !"##$%&"#'7')'***+,-.%/00$#.+%"1'234+5(6+36+758 !"##$%&"#'5')'9:-#;-.$+9:+-<$0%$<&"#=+%"1'239+7>+47+4(8' !"##$%&"#'?')'=.@&%+=A-B$=C@0$+#$.'233+779+9D+598 !"##$%&"#'9')'=.@&%+=A-B$=C@0$+#$.'233+779+9D+598 !"##$%&"#'6')'***+E$$B,/0#$0+%"1'233+76(+D3+75?8 !"##$%&"#'3')'=.@&%+F$.%A-%GH+%"1'25(9+7?+>+7>8' !"##$%&"#'4')'%$.0G+%"1'25(>+34+7>?+7((8 !"##$%&"#'>')'-#+F$.%A-%GH+%"1'25(9+7?+>+7>8 !"##$%&"#'D')'%0@IH$FF+%"1'25(>+34+7>(+5?38 !"##$%&"#'7(')'***+F""FA$)@#@AH&%=+%"1'245+79+55?+7948' !"##$%&"#'77')'***+@<./0$+%"1'234+7D5+93+7D8 !"##$%&"#'75')'=.@&%+@<./0$+%"1'234+7D5+93+568 !"##$%&"#'7?')'=+%A-%G.@A$+#$.'233+779+9D+598' !"##$%&"#'79')'***+%A-%G.@A$+#$.'246+756+>5+4(8 Wednesday, November 11, 2009
    84. 84. 6")%$/15*3$!"##$%&'$71$+&1-78 1#$2.$34)5# D"4(@4G(@(&% • 6789+J-#B'.C$'=-.$ • <@(#)&9+U@B'0$:/$=.= • :19+K"/.$'.C$'<@%G$.= • 8#$"#$9+!@<@%-.HS'@;@-A@,-A-.HS'VW • ;<19+L=.@,A-=C'@'%"##$%&"# • 7#&-.$A9+R@%G$.'A"==S'X!R'&1$"/.=S' • <679+K$B-0$%.M";$0A@H'.C$'.0@N% .0@N%'=C@<-#F • 88=9+O$F"&@.$'$#%0H<&"# • DHH@(54I.)9+Y"F-%S'B$<$#B$#%-$= • >;;19+K$:/$=.'%"#.$#. • 74"(*4I.)9+Y""<-#FS'=$==-"#'&1$"/.= • >.,&+?#@4%9+P$#$0@.$'.C$'%"#.$#. • <.)&#)&9+U0"G$#S'/#0$#B$0@,A$ • 7#&-.$A+?#@4%9+Q$A-;$0'.C$'%"#.$#. • C:D9+RA/F-#S'1$1"0HS'VW • 145A#&+@.,,9+K$%";$0'E0"1'$00"0= • B$.-,#$9+R@0=$'.C$'<@F$S'F$.'1"0$' %"#.$#. • C:D,9+LT$%/.$'%"B$'"#'.C$'%A-$#. • B$.-,#$E+F89+K$#B$0'.C$'<@F$ Wednesday, November 11, 2009
    85. 85. Wednesday, November 11, 2009
    86. 86. How do we measure it? Wednesday, November 11, 2009
    87. 87. Test Testing config node Data center Testing node Website Reporting service Testing node Wednesday, November 11, 2009
    88. 88. Synthetic isn’t enough Wednesday, November 11, 2009
    89. 89. Browser Load Web Network balancer server tap User A User B User C Visit Aggregate Alerts history reports ! P1 P2 P3 Wednesday, November 11, 2009
    90. 90. Justifying an investment in performance Current * Increased + (Increased daily orders conversions order value ) ROI = (days) Cost of performance enhancement Wednesday, November 11, 2009
    91. 91. Justifying an investment in performance $10,000 * ( 0.1607 + 0.0551 ) = $2,158 23.17 = days $50,000 $50,000 Caveats: Your mileage will vary This is just how to think about it Wednesday, November 11, 2009
    92. 92. Conclusions • Links between performance and business KPIs are undeniable • By talking the same language, IT and marketing can finally agree on what to do about it • Changing from “X times faster” to “$Y more money” makes the business care • More research is needed Wednesday, November 11, 2009
    93. 93. Putting it all together The Big Picture Wednesday, November 11, 2009
    94. 94. “Hard” data Analytics Usability Performability (what did they (how did they (could they do on the interact with do what they site?) it?) wanted to?) Complete Web Monitoring Community VoC Competition (what were (what were (what are they they saying?) their up to?) motivations?) “Soft” data Wednesday, November 11, 2009
    95. 95. Wednesday, November 11, 2009
    96. 96. Wednesday, November 11, 2009
    97. 97. Wednesday, November 11, 2009
    98. 98. Thanks! alistair@bitcurrent.com sean@httpd.org @acroll @seanpower linkedin/in/alistaircroll linkedin/in/slpower bitcurrent.com www.watchingwebsites.com rednod.com www.watchingwebsites.com http://www.amazon.com/Complete-Web-Monitoring- Performance-Communities/dp/0596155131/ Wednesday, November 11, 2009

    ×