• Save
Web Performance: Who Cares About Customer Experience? : Sean Power and Alistair Croll
Upcoming SlideShare
Loading in...5
×
 

Web Performance: Who Cares About Customer Experience? : Sean Power and Alistair Croll

on

  • 2,891 views

This was presented on the MeasureWorks event in Baarn on 12 November 2009 by Sean Power and Alistair Croll. More Event Info:

This was presented on the MeasureWorks event in Baarn on 12 November 2009 by Sean Power and Alistair Croll. More Event Info:
www.measureworks-event.nl and www.measureworks.nl

Statistics

Views

Total Views
2,891
Views on SlideShare
2,733
Embed Views
158

Actions

Likes
12
Downloads
32
Comments
0

8 Embeds 158

http://www.webanalisten.nl 107
http://events.measureworks.nl 21
http://www.slideshare.net 15
http://siteusability.blogspot.com 5
http://www.linkedin.com 5
http://www.slidescene.com 2
https://www.linkedin.com 2
http://feeds.feedburner.com 1
More...

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • here today to tell you that your website has a dirty little secret <br /> your website is trying really, really hard to hide things from you . . and they&#x2019;re not necessarily small things either.
  • for example . . . imagine that you decided to launch a kick ass survey. you&#x2019;ve bought the latest shiny tool <br /> you&#x2019;ve carefully crafted the questions <br /> you hired outside help to make sure they&#x2019;re worded properly <br /> you had them sent to a professional copy editor to get the final tone just right <br /> it went through legal <br /> you segmented your campaign according to the demographic whose voice you need to understand the most <br /> <br /> and as you sit precariously over the big red send button you can&#x2019;t help but feel that you&#x2019;ve covered all your bases. <br /> Satisfied, you press the button and out it goes into the world.
  • that was the case with paypal, recently. <br /> We don&#x2019;t have insight into their numbers, so we can&#x2019;t tell for sure what the particular conversion rate for this survey was, but we suspect that the pickup wasn&#x2019;t as good as anticipated. <br /> <br /> Their web analytics and VOC don&#x2019;t have the necessary functions built in to determine that their SSL cert was mismatched, cause safari and other browsers to come up with a nasty message saying &#x201C;we can&#x2019;t verify the identity of paypal-surveys.com&#x201D;. <br /> After all, think about it; if it&#x2019;s coming from paypal and the identity can&#x2019;t be verified, would you go on the site and fill anything out?
  • we know of a case of a marketing officer who&#x2019;se job was put in question because of a string of failed campaigns. <br /> The company jumped the gun on this one. Thanks to a friend in the web operations department, he was able to show that the network was at fault. Even though the company load tested diligently, they only did from their internal network. It turns out the problems were related to the last mile - something that was hidden until the company implemented synthetic monitoring. <br /> <br /> Even though overall sentiment was a little more negative than usual during the campaigns, the conversion rates skyrocketed once better transit was installed.
  • This is a scary one and a true. If you haven&#x2019;t heard, sitemeter took down every single website that were a client of theirs. If you were on IE and wanted to access sites like TechCrunch, Gizmodo and so on, you were out of luck in August, because the code crashed the browser. <br /> <br /> Think about it - your site isn&#x2019;t just vulnerable to whatever goofy code your development team throws at the Internet, it&#x2019;s also vulnerable to your very own web analytics tracking codes! <br /> <br /> This would take hours of troubleshooting to reveal without synthetic monitoring - or one simply alert would be triggered with the proper tools in place. <br /> <br /> I don&#x2019;t mean to pick on SiteMeter btw, I&#x2019;m sure they have a great service - but these types of errors can kill substantial amounts of revenue until you catch it.
  • You may be able to tie performance to business results such as key steps in a transaction
  • Or see what people think of your site (based on surveys) by page within analytics.
  • Once upon a time, performance was a dark art. We struggled to deliver &#x201C;good enough&#x201D; without really knowing why.
  • We managed by anecdote. We were sure faster was better, but we couldn&#x2019;t tie it to specific business outcomes.
  • The notion that speed is good for users isn&#x2019;t new. The concept of &#x201C;Flow&#x201D; &#x2013; a state of heightened engagement that we experience when we&#x2019;re truly focused on something &#x2013; was first proposed by mihaly csikszentmihalyi
  • It turns out that attention and engagement drop off predictably. At ten milliseconds, we actually believe something is physically accessible &#x2013; think clicking a button and seeing it change color. At 100 milliseconds, we can have a conversation with someone without noticing the delay (remember old transatlantic calls?) At a second, we&#x2019;re still engaged, but aware of the delay. At ten seconds, we get bored and tune out, because other things come into our minds.
  • It turns out that attention and engagement drop off predictably. At ten milliseconds, we actually believe something is physically accessible &#x2013; think clicking a button and seeing it change color. At 100 milliseconds, we can have a conversation with someone without noticing the delay (remember old transatlantic calls?) At a second, we&#x2019;re still engaged, but aware of the delay. At ten seconds, we get bored and tune out, because other things come into our minds.
  • It turns out that attention and engagement drop off predictably. At ten milliseconds, we actually believe something is physically accessible &#x2013; think clicking a button and seeing it change color. At 100 milliseconds, we can have a conversation with someone without noticing the delay (remember old transatlantic calls?) At a second, we&#x2019;re still engaged, but aware of the delay. At ten seconds, we get bored and tune out, because other things come into our minds.
  • It turns out that attention and engagement drop off predictably. At ten milliseconds, we actually believe something is physically accessible &#x2013; think clicking a button and seeing it change color. At 100 milliseconds, we can have a conversation with someone without noticing the delay (remember old transatlantic calls?) At a second, we&#x2019;re still engaged, but aware of the delay. At ten seconds, we get bored and tune out, because other things come into our minds.
  • It turns out that attention and engagement drop off predictably. At ten milliseconds, we actually believe something is physically accessible &#x2013; think clicking a button and seeing it change color. At 100 milliseconds, we can have a conversation with someone without noticing the delay (remember old transatlantic calls?) At a second, we&#x2019;re still engaged, but aware of the delay. At ten seconds, we get bored and tune out, because other things come into our minds.
  • How much was fast enough? It was anybody&#x2019;s guess.
  • And guess they did. <br /> This is Zona&#x2019;s formula for patience, the basis for the &#x201C;eight second rule.&#x201D; Unfortunately, things like tenacity, importance, and natural patience aren&#x2019;t concrete enough for the no-nonsense folks that run web applications.
  • And guess they did. <br /> This is Zona&#x2019;s formula for patience, the basis for the &#x201C;eight second rule.&#x201D; Unfortunately, things like tenacity, importance, and natural patience aren&#x2019;t concrete enough for the no-nonsense folks that run web applications.
  • IT operators and marketers are completely different people. What convinces an IT person to fix performance doesn&#x2019;t convince a marketer. They want to know how it will impact the business fundamentals.
  • By now, we know that everything matters. Usability, page latency, visitor mindset, and even sentiment on social media platforms all contribute to the business results you get from a site.
  • Fortunately, we&#x2019;re getting better at linking performance to business outcomes.
  • One example of this is performance experimentation that Google&#x2019;s done. Google&#x2019;s a perfect lab. Not only do they have a lot of traffic, they also have computing resources to do back-end analysis of large data sets. Plus, they&#x2019;re not afraid of experimentation &#x2013; in fact, they insist on it. So they tried different levels of performance and watched what happened to visitors.
  • The results, which they presented at Velocity in May, were fascinating. There was a direct impact between delay and the number of searches a user did each day &#x2013; and to make matters worse, the numbers often didn&#x2019;t improve even when the delay was removed. You may think 0.7% drop isn&#x2019;t significant, but for Google this represents a tremendous amount of revenue.
  • Microsoft&#x2019;s Bing site is a good lab, too. They looked at key metrics, or KPIs, of their search site.
  • They showed that as performance got worse, all key metrics did, too. Not just the number of searches, but also the revenue (earned when someone clicks) and refinement of searches.
  • Shopzilla overhauled their entire site, dramatically reducing page load time, hardware requirements, and downtime.
  • They saw a significant increase in revenues
  • The site improvement increased the number of Google clicks that turned into actual visits
  • It also affected search engine scores. By improving load time, search engines (in this case Google UK) &#x201C;learned&#x201D; that this was a good destination. That&#x2019;s right &#x2013; Google actually penalizes sites that are slow by giving them a lower page ranking.
  • If we want to convince marketing, we need to measure business metrics.
  • By tying performance and availability to Key Performance Indicators &#x2013; KPIs &#x2013; business and operations can finally have a conversation.
  • The first kind of site is a media site.
  • Transactional sites make money when people complete a process, usually involving a purchase or subscription.
  • Collaboration sites (like wikis, news aggregators, user groups, social news aggregators) are those where the value of the site is largely determined by the content users generate.
  • Finally, Software as a Service sites are ones where a subscribers expect reliability after paying a monthly fee for a service
  • Whether those KPIs are shopping cart abandonment
  • Or visitor &#x201C;bounce rate&#x201D; (the number of visitors that leave immediately)
  • Or just traffic.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they&#x2019;re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It&#x2019;s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they&#x2019;re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It&#x2019;s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they&#x2019;re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It&#x2019;s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they&#x2019;re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It&#x2019;s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they&#x2019;re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It&#x2019;s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they&#x2019;re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It&#x2019;s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they&#x2019;re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It&#x2019;s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they&#x2019;re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It&#x2019;s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they&#x2019;re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It&#x2019;s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they&#x2019;re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It&#x2019;s what the business cares about.
  • So what KPIs would we like to learn about? This is what web analytics folks work by, whether they&#x2019;re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It&#x2019;s what the business cares about.
  • Strangeloop agreed to set up an experiment using their technology which would help measure this.
  • First, traffic. Despite splitting visitors to be optimized and unoptimized evenly, we had many more optimized sessions captured by the analytics. This may be a result of slower-loading pages failing to execute the analytics script, or abandoning the visit before the page had time to load.
  • Unoptimized visitors are roughly 1% more likely to leave the site immediately, without proceeding to other pages.
  • Strangely, the unoptimized visitors consisted of more new visitors than the optimized ones did. This seems counter-intuitive and warrants further study.
  • Optimized visitors spent more time on the site
  • And looked at more pages during their visit &#x2013; if you&#x2019;re a media property, this means more impressions for your advertisers.
  • On a second e-commerce site running roughly the same experiment, conversions were 16 percent higher and orders were 5.5% higher.
  • slow performance hurts
  • A site that&#x2019;s unresponsive or plagued by incidents and unpredictable availability has lower conversion rates. <br /> Sites that deliver a consistently poor end user experience are less likely to attract a loyal following. Poor site performance may also affect perception of your company&#x2019;s brand or reputation. <br /> You may be liable for damages if you can&#x2019;t handle transactions promptly, particularly if you&#x2019;re in a heavily regulated industry such as finance or healthcare. <br /> Poor performance may cost you money. If you have a formal contract with users, you may be liable for refunds or service credits. Slow or unavailable sites also encourage customers to find other channels, such as phone support or retail outlets, that cost your organization far more than handling requests via the Web. Once visitors try those channels, they may stick with them, costing you even more money.
  • Time for a crash course in web performance <br /> (there are smarter people than us on this topic but we&#x2019;ll try.)
  • Each time a visitor comes to your site, she must first look up the domain with a DNS service
  • Each time a visitor comes to your site, she must first look up the domain with a DNS service
  • Each time a visitor comes to your site, she must first look up the domain with a DNS service
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it&#x2019;s of any real size, then there&#x2019;s a thing called a load balancer at the other end. <br /> These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it&#x2019;s of any real size, then there&#x2019;s a thing called a load balancer at the other end. <br /> These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it&#x2019;s of any real size, then there&#x2019;s a thing called a load balancer at the other end. <br /> These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it&#x2019;s of any real size, then there&#x2019;s a thing called a load balancer at the other end. <br /> These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it&#x2019;s of any real size, then there&#x2019;s a thing called a load balancer at the other end. <br /> These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it&#x2019;s of any real size, then there&#x2019;s a thing called a load balancer at the other end. <br /> These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it&#x2019;s of any real size, then there&#x2019;s a thing called a load balancer at the other end. <br /> These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it&#x2019;s of any real size, then there&#x2019;s a thing called a load balancer at the other end. <br /> These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it&#x2019;s of any real size, then there&#x2019;s a thing called a load balancer at the other end. <br /> These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it&#x2019;s of any real size, then there&#x2019;s a thing called a load balancer at the other end. <br /> These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it&#x2019;s of any real size, then there&#x2019;s a thing called a load balancer at the other end. <br /> These packets are called IP, or Internet Protocol, packets.
  • Then the browser sends packets of data across the Internet, bouncing from router to router, until they get to the website. If it&#x2019;s of any real size, then there&#x2019;s a thing called a load balancer at the other end. <br /> These packets are called IP, or Internet Protocol, packets.
  • That connection takes a bit of explaining. <br /> Think about the postal service. You don&#x2019;t need to know how it works to send mail. There&#x2019;s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining. <br /> Think about the postal service. You don&#x2019;t need to know how it works to send mail. There&#x2019;s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining. <br /> Think about the postal service. You don&#x2019;t need to know how it works to send mail. There&#x2019;s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining. <br /> Think about the postal service. You don&#x2019;t need to know how it works to send mail. There&#x2019;s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining. <br /> Think about the postal service. You don&#x2019;t need to know how it works to send mail. There&#x2019;s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining. <br /> Think about the postal service. You don&#x2019;t need to know how it works to send mail. There&#x2019;s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining. <br /> Think about the postal service. You don&#x2019;t need to know how it works to send mail. There&#x2019;s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining. <br /> Think about the postal service. You don&#x2019;t need to know how it works to send mail. There&#x2019;s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining. <br /> Think about the postal service. You don&#x2019;t need to know how it works to send mail. There&#x2019;s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining. <br /> Think about the postal service. You don&#x2019;t need to know how it works to send mail. There&#x2019;s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • That connection takes a bit of explaining. <br /> Think about the postal service. You don&#x2019;t need to know how it works to send mail. There&#x2019;s a separation of concerns there. As long as you follow some rules -- addressing, putting it in the mailbox by 5PM -- everything works fine.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • But imagine you could only put one word in each envelope. Trying to send someone a message would be a bit more complicated. You&#x2019;d compose your sentence, and send it on its way. Unfortunately, the receiver wouldn&#x2019;t be able to reassemble the pile of letters that came through their door.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • So maybe you&#x2019;d get clever, and start numbering the letters. That way your receiver would know what order they&#x2019;re in.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • The system doesn&#x2019;t work so well, however, if something&#x2019;s lost along the way.
  • What you&#x2019;d need is a system of numbered envelopes, plus a system that let the receiver tell you when they were missing something. With these two things working properly, you&#x2019;d simulate an end-to-end connection between yourself and your receiver, and you could send long messages reliably even though only one word fit in each envelope.
  • What you&#x2019;d need is a system of numbered envelopes, plus a system that let the receiver tell you when they were missing something. With these two things working properly, you&#x2019;d simulate an end-to-end connection between yourself and your receiver, and you could send long messages reliably even though only one word fit in each envelope.
  • What you&#x2019;d need is a system of numbered envelopes, plus a system that let the receiver tell you when they were missing something. With these two things working properly, you&#x2019;d simulate an end-to-end connection between yourself and your receiver, and you could send long messages reliably even though only one word fit in each envelope.
  • What you&#x2019;d need is a system of numbered envelopes, plus a system that let the receiver tell you when they were missing something. With these two things working properly, you&#x2019;d simulate an end-to-end connection between yourself and your receiver, and you could send long messages reliably even though only one word fit in each envelope.
  • This is how the TCP/IP stack works. IP sends packets of data; TCP makes sure they get there. <br /> TCP creates an end-to-end connection between your browser and a server.
  • This is how the TCP/IP stack works. IP sends packets of data; TCP makes sure they get there. <br /> TCP creates an end-to-end connection between your browser and a server.
  • This is how the TCP/IP stack works. IP sends packets of data; TCP makes sure they get there. <br /> TCP creates an end-to-end connection between your browser and a server.
  • This is how the TCP/IP stack works. IP sends packets of data; TCP makes sure they get there. <br /> TCP creates an end-to-end connection between your browser and a server.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • You can try this out yourself.
  • You can try this out yourself.
  • You can try this out yourself.
  • You can try this out yourself.
  • You can try this out yourself.
  • If you&#x2019;re loading static content -- an image, for example -- the web server can deliver it.
  • If you&#x2019;re loading static content -- an image, for example -- the web server can deliver it.
  • If you&#x2019;re loading static content -- an image, for example -- the web server can deliver it.
  • If you&#x2019;re loading static content -- an image, for example -- the web server can deliver it.
  • If it&#x2019;s a dynamic object, such as a custom page, then the app server responds.
  • If it&#x2019;s a dynamic object, such as a custom page, then the app server responds.
  • If it&#x2019;s a dynamic object, such as a custom page, then the app server responds.
  • If it&#x2019;s a dynamic object, such as a custom page, then the app server responds.
  • If it&#x2019;s a dynamic object, such as a custom page, then the app server responds.
  • If it&#x2019;s a dynamic object, such as a custom page, then the app server responds.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • So TCP creates this end-to-end link between your computer and the web server on the other side of the connection. <br /> On top of TCP, there&#x2019;s a similar set of rules for encrypting data (if needed) and finally a set of rules for requesting web objects, known as HTTP.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • In the end, this is what happens.
  • The time a page takes to load is a function of all the objects it contains, which are retrieved from the server -- more objects, or a longer time to load, makes a slower page load.
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • This creates a huge amount of latency
  • Websites have more and more connections, too...
  • So what could possibly go wrong?
  • This makes visitors sad and lonely
  • Sites still fail in lots of ways. It&#x2019;s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It&#x2019;s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It&#x2019;s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It&#x2019;s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It&#x2019;s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It&#x2019;s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It&#x2019;s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It&#x2019;s scary how much things break. This is just a sample of pages for Canadians...
  • Sites still fail in lots of ways. It&#x2019;s scary how much things break. This is just a sample of pages for Canadians...
  • So how do we check for this stuff? One way to do testing is to use internal systems to run tests.
  • So how do we check for this stuff? One way to do testing is to use internal systems to run tests.
  • So how do we check for this stuff? One way to do testing is to use internal systems to run tests.
  • If you have a load balancer, you can use that too.
  • If you have a load balancer, you can use that too.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there&#x2019;s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there&#x2019;s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there&#x2019;s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there&#x2019;s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there&#x2019;s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there&#x2019;s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there&#x2019;s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there&#x2019;s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there&#x2019;s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there&#x2019;s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • While testing vendors vary by the number of testing points, and the detail of their reports, there&#x2019;s one more important dimension to consider: How the test is done. Simple testing services use scripts that simulate a browser.
  • A more advanced approach is what we call &#x201C;browser puppetry&#x201D; -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call &#x201C;browser puppetry&#x201D; -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call &#x201C;browser puppetry&#x201D; -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call &#x201C;browser puppetry&#x201D; -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call &#x201C;browser puppetry&#x201D; -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call &#x201C;browser puppetry&#x201D; -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call &#x201C;browser puppetry&#x201D; -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call &#x201C;browser puppetry&#x201D; -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call &#x201C;browser puppetry&#x201D; -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call &#x201C;browser puppetry&#x201D; -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call &#x201C;browser puppetry&#x201D; -- which uses a real browser to request and retrieve information.
  • A more advanced approach is what we call &#x201C;browser puppetry&#x201D; -- which uses a real browser to request and retrieve information.
  • Synthetic tests show you whether your site is working; but to tell whether it&#x2019;s BROKEN you need to look at actual visitors. Here&#x2019;s a chart of visits over time. see the synthetic tests? See the gaps when other things happen?
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • But the most common way to do this testing is to use a monitoring service to perform synthetic, scripted tests.
  • there&#x2019;s a much larger analytics ecosystem that exists. Yikes!
  • So if your brain&#x2019;s full, don&#x2019;t be surprised. There&#x2019;s a huge amount of data to consolidate.
  • it&#x2019;s also important to correlate data. In this case, we can see that there is a clear relationship between latency and conversion.
  • and often, it&#x2019;s very useful to segment your data in particular ways. <br /> In this case, there&#x2019;s a correlation between the AVG page load time across visit and the propensity that a user will post a comment
  • In the end, this is what it should all look like.
  • Ultimately, you want a single, comprehensive view of your web presence across all of these platforms in order to make good decisions and communicate what you&#x2019;re doing to the rest of the organization.
  • Thanks for the time. If you want to reach us or follow us online, here&#x2019;s how. And if you want to pre-order our book (which would freak out our editors) we&#x2019;d love it. We&#x2019;re shameless self-promoters.

Web Performance: Who Cares About Customer Experience? : Sean Power and Alistair Croll Web Performance: Who Cares About Customer Experience? : Sean Power and Alistair Croll Presentation Transcript

  • Who cares about customer experience? A Complete Web Monitoring perspective Wednesday, November 11, 2009
  • “Hard” data Analytics Usability Performability (what did they (how did they (could they do do on the interact with what they site?) it?) wanted to?) Complete Web Monitoring Community VoC Competition (what were (what were (what are they they saying?) their up to?) motivations?) “Soft” data Wednesday, November 11, 2009
  • What did they do? Web analytics Wednesday, November 11, 2009
  • http://www.flickr.com/photos/diegocupolo/3511117614/ Wednesday, November 11, 2009
  • Accounting Optimization http://www.flickr.com/photos/sanchom/2963072255/ http://www.flickr.com/photos/thomasclaveirole/538819881/ Wednesday, November 11, 2009
  • How did they do it? Web Interaction Analytics Wednesday, November 11, 2009
  • http://www.flickr.com/photos/trekkyandy/189717616/ Wednesday, November 11, 2009
  • Why did they do it? Voice of the Customer Wednesday, November 11, 2009
  • http://www.flickr.com/photos/karola/3623768629/ Wednesday, November 11, 2009
  • Could they do it? Performance & availability Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • What were they saying? Community monitoring Wednesday, November 11, 2009
  • ! Wednesday, November 11, 2009
  • What were they up to? Competitive analysis Wednesday, November 11, 2009
  • http://www.flickr.com/photos/31690139@N02/2965956581/ Wednesday, November 11, 2009
  • Why here? Why now? Online business is becoming the default. Wednesday, November 11, 2009
  • !8B • In 2006 !3B online retail !1.1 sales in the Netherlands !6B !1.7 • In 2011 !12B in total online sales in Benelux, with more than half of !1.9 !4B the Dutch adult population shopping online !2B !3.3 • In 2014 Forrester estimates over !8B in !0B online retail sales in the Netherlands Other Leisure flights Consumer electronics Hotels Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Online marketers want to maximize their revenues. Wednesday, November 11, 2009
  • http://www.davidross.com.mx/admin/wideimagerepository/sergioZymanAmp.jpg Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • http://www.goal-posts.net/media/fbl-181_cat_07_160_x_160.jpg Wednesday, November 11, 2009
  • http://www.johnsoncontrols.com/publish/etc/medialib/jci/ps/Media_Kit.Par.25973.File.dat/complex_machinery_HIRES.JPG Wednesday, November 11, 2009
  • http://www.infovisual.info/02/img_en/078%20Different%20land%20animals%201.jpg Wednesday, November 11, 2009
  • http://upload.wikimedia.org/wikipedia/commons/0/0e/Soccer_Youth_Goal_Keeper.jpg Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Some sites have ungoals. Things you really don’t want visitors to do. Wednesday, November 11, 2009
  • You really don’t want web users to call you. US$15 US$12 US$9 US$6 $5.50 $3.00 US$3 $0.24 $0.45 US$0 Web self-service IVR Email Live phone Cost estimates BiT Group White Paper: “Web Self-Service Lowers Call Center Costs and Improves Customer Service” Low Average High Wednesday, November 11, 2009
  • Websites have a dirty little secret http://todaystatus.files.wordpress.com/2009/04/ww11-secret.jpg Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • http://www.inquisitr.com/2097/site-meter-causing-internet-explorer-failure/ Wednesday, November 11, 2009
  • http://www.flickr.com/photos/aleermakers/3455786409/ Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • http://www.octulipfestival.com/images/Picture%20144.jpg Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Figure 3 Interactive user productivity versus computer response time for human-intensive interactions for system A E 600 - 3 T -" INTERACTIVE USER PRODUCTIVITY (IUP) w -HUMAN-INTENSIVE COMPONENT OF IUP 7 MEASURED DATA (HUMAN-INTENSIVE E 500 - A z " COMPONENT) U E - w E 0 > - > - - 400 3 n F 2 0 0 300 - 200 - 100 - 0 0- I 1 I I I 0 1 2 3 4 5 COMPUTER RESPONSE TIME (SI (1981) A. J. Thadhani, IBM Systems Journal, Volume 20, number 4 Wednesday, November 11, 2009
  • 10 s 1s 100 ms 10 ms ! Zzz Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • http://www.flickr.com/photos/spunter/393793587 http://www.flickr.com/photos/laurenclose/2217307446 Wednesday, November 11, 2009
  • !"#$%&'()*+(,+()&#$-."#)/ Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Shopzilla had another angle • Big, high-traffic site • 16 month re-engineering • 100M impressions a day • Page load from 6 seconds to 1.2 • 8,000 searches a second • 20-29M unique visitors a • Uptime from 99.65% to month 99.97% • 100M products • 10% of previous hardware needs http://en.oreilly.com/velocity2009/public/schedule/detail/7709 Wednesday, November 11, 2009
  • 5-12% increase in revenue. Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Tying web latency to business outcomes. Wednesday, November 11, 2009
  • http://www.flickr.com/photos/spunter/393793587 http://www.flickr.com/photos/laurenclose/2217307446 KPIs Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • http://www.flickr.com/photos/mrmoorey/160654236 Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • ATTENTION ENGAGEMENT CONVERSION NEW VISITORS SEARCHES GROWTH CONVERSION PAGES TIME RATE TWEETS NUMBER OF VISITS PER ON 0 MENTIONS VISIT SITE ORDER ADS SEEN LOSS VALUE BOUNCE RATE Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • VISITOR STRANGELOOP WEB ACCELERATOR SERVER Decide whether to optimize Normal Receive Accelerated content page Insert Process scripts Optimize? segment marker Send analytics Unaccelerated GOOGLE ANALYTICS Wednesday, November 11, 2009
  • What we learned: Wednesday, November 11, 2009
  • Traffic levels 9,000 Total number of visits 6,750 4,500 8,505 2,250 4,740 0 Optimized Unoptimized Visitor experience Wednesday, November 11, 2009
  • Bounce rate 20 15 Visits that bounced 10 13.38% 14.35% 5 0 Optimized Unoptimized Visitor experience Wednesday, November 11, 2009
  • % visits marked “new” % of visits that had no returning cookie 14 11 7 13.61% 10.85% 4 0 Optimized Unoptimized Visitor experience Wednesday, November 11, 2009
  • That means... 9000 Value Number of visits 6750 4500 7,582 4,095 2250 923 645 0 Optimized Unoptimized Wednesday, November 11, 2009
  • Average time on site 31 Time on site (minutes) 23 16 30.17 23.83 8 0 Optimized Unoptimized Visitor experience Wednesday, November 11, 2009
  • Pages per visit 16 12 Average pages seen 8 15.64 11.04 4 0 Optimized Unoptimized Visitor experience Wednesday, November 11, 2009
  • Conversion rate 20 and order value Difference due to optimization 15 10 16.07 5 5.51 0 Conversion rate Order value Wednesday, November 11, 2009
  • This is just one case LOTS # OF VISITS OPTIMIZED 0 0 VISITOR LATENCY 10,000 Different visitors experienced different performance levels. Wednesday, November 11, 2009
  • With one outcome LOTS # OF VISITS 21.58% 0 BETTER 0 VISITOR LATENCY 10,000 Right now we have a single experiment, and a single resulting business impact. Wednesday, November 11, 2009
  • With one outcome LOTS Best 5% Worst 5% # OF VISITS 21.58% 0 BETTER 0 VISITOR LATENCY 10,000 Visitors who were optimized fall into a range – the 5th to 95th percentile. Wednesday, November 11, 2009
  • Lots of different results LOTS 24% 18% $ PER 14% 12% DAY 9.5% 0 0 VISITOR LATENCY 10,000 If we have several experiments, we can understand the relationship better. Wednesday, November 11, 2009
  • You have your own curve LOTS $ PER DAY 0 0 VISITOR LATENCY 10,000 Every web business has a curve like this hidden inside it. Wednesday, November 11, 2009
  • “What’s a latency?” Wednesday, November 11, 2009
  • !"#$%&'(')*$+#,$%&)-.)/01- Browser Data center Server TCP SYN (“let’s talk”) TCP SYN ACK (“Agreed: let’s talk”) TCP ACK (“OK, we’re talking) SSL (“Someone might be listening!”) SSL (“Here’s a decoder ring”) HTTP GET / (“Can I have your home page?”) HTTP 200 OK (“Sure!”) (thinks [index.html] (“Here it is!”) a bit) (Renders furiously) Bump, bump. [img js css] (“Have this too!”) TCP FIN (“Thanks! I’m done now.”) TCP FIN ACK (“You’re welcome. Have a nice day.”) Wednesday, November 11, 2009
  • Analytics site Server Data center Browser Server Snurken Mashup site Server Wednesday, November 11, 2009
  • 21--#/01-.$%1$*1)3$4'%/5&&#-% !"##$%&"#'(')'***+,-.%/00$#.+%"1'234+5(6+36+758 !"##$%&"#'7')'***+,-.%/00$#.+%"1'234+5(6+36+758 !"##$%&"#'5')'9:-#;-.$+9:+-<$0%$<&"#=+%"1'239+7>+47+4(8' !"##$%&"#'?')'=.@&%+=A-B$=C@0$+#$.'233+779+9D+598 !"##$%&"#'9')'=.@&%+=A-B$=C@0$+#$.'233+779+9D+598 !"##$%&"#'6')'***+E$$B,/0#$0+%"1'233+76(+D3+75?8 !"##$%&"#'3')'=.@&%+F$.%A-%GH+%"1'25(9+7?+>+7>8' !"##$%&"#'4')'%$.0G+%"1'25(>+34+7>?+7((8 !"##$%&"#'>')'-#+F$.%A-%GH+%"1'25(9+7?+>+7>8 !"##$%&"#'D')'%0@IH$FF+%"1'25(>+34+7>(+5?38 !"##$%&"#'7(')'***+F""FA$)@#@AH&%=+%"1'245+79+55?+7948' !"##$%&"#'77')'***+@<./0$+%"1'234+7D5+93+7D8 !"##$%&"#'75')'=.@&%+@<./0$+%"1'234+7D5+93+568 !"##$%&"#'7?')'=+%A-%G.@A$+#$.'233+779+9D+598' !"##$%&"#'79')'***+%A-%G.@A$+#$.'246+756+>5+4(8 Wednesday, November 11, 2009
  • 6")%$/15*3$!"##$%&'$71$+&1-78 1#$2.$34)5# D"4(@4G(@(&% • 6789+J-#B'.C$'=-.$ • <@(#)&9+U@B'0$:/$=.= • :19+K"/.$'.C$'<@%G$.= • 8#$"#$9+!@<@%-.HS'@;@-A@,-A-.HS'VW • ;<19+L=.@,A-=C'@'%"##$%&"# • 7#&-.$A9+R@%G$.'A"==S'X!R'&1$"/.=S' • <679+K$B-0$%.M";$0A@H'.C$'.0@N% .0@N%'=C@<-#F • 88=9+O$F"&@.$'$#%0H<&"# • DHH@(54I.)9+Y"F-%S'B$<$#B$#%-$= • >;;19+K$:/$=.'%"#.$#. • 74"(*4I.)9+Y""<-#FS'=$==-"#'&1$"/.= • >.,&+?#@4%9+P$#$0@.$'.C$'%"#.$#. • <.)&#)&9+U0"G$#S'/#0$#B$0@,A$ • 7#&-.$A+?#@4%9+Q$A-;$0'.C$'%"#.$#. • C:D9+RA/F-#S'1$1"0HS'VW • 145A#&+@.,,9+K$%";$0'E0"1'$00"0= • B$.-,#$9+R@0=$'.C$'<@F$S'F$.'1"0$' %"#.$#. • C:D,9+LT$%/.$'%"B$'"#'.C$'%A-$#. • B$.-,#$E+F89+K$#B$0'.C$'<@F$ Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • How do we measure it? Wednesday, November 11, 2009
  • Test Testing config node Data center Testing node Website Reporting service Testing node Wednesday, November 11, 2009
  • Synthetic isn’t enough Wednesday, November 11, 2009
  • Browser Load Web Network balancer server tap User A User B User C Visit Aggregate Alerts history reports ! P1 P2 P3 Wednesday, November 11, 2009
  • Justifying an investment in performance Current * Increased + (Increased daily orders conversions order value ) ROI = (days) Cost of performance enhancement Wednesday, November 11, 2009
  • Justifying an investment in performance $10,000 * ( 0.1607 + 0.0551 ) = $2,158 23.17 = days $50,000 $50,000 Caveats: Your mileage will vary This is just how to think about it Wednesday, November 11, 2009
  • Conclusions • Links between performance and business KPIs are undeniable • By talking the same language, IT and marketing can finally agree on what to do about it • Changing from “X times faster” to “$Y more money” makes the business care • More research is needed Wednesday, November 11, 2009
  • Putting it all together The Big Picture Wednesday, November 11, 2009
  • “Hard” data Analytics Usability Performability (what did they (how did they (could they do on the interact with do what they site?) it?) wanted to?) Complete Web Monitoring Community VoC Competition (what were (what were (what are they they saying?) their up to?) motivations?) “Soft” data Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Wednesday, November 11, 2009
  • Thanks! alistair@bitcurrent.com sean@httpd.org @acroll @seanpower linkedin/in/alistaircroll linkedin/in/slpower bitcurrent.com www.watchingwebsites.com rednod.com www.watchingwebsites.com http://www.amazon.com/Complete-Web-Monitoring- Performance-Communities/dp/0596155131/ Wednesday, November 11, 2009