Watching websites

Slide 1: Watching websites

Slide 2: What we’re going to cover • Why watch your website? • How to do it (well & affordably)

Slide 3: STARTUP 101

Slide 4: New idea Execution Feedback Success? End Money left? Risk End

Slide 5: Feedback, the old way.

Slide 6: Are people doing what we want?

Slide 7: Are we doing dumb stuff?

Slide 8: http://flickr.com/photos/ikhlasulamal/2443194039/ Do we understand our users?

Slide 9: Is it easy and intuitive? http://flickr.com/photos/jmecelab/2323995433/

Slide 10: Slow, unmeasured trial & error.

Slide 11: The Internet lets us make mistakes faster.

Slide 12: The weird part: Mistakes are good • Most startups don’t succeed doing what they set out to do – Amazon was just a bookstore – eBay sold Pez – F5 made hurricane modeling software • But mistake speed is critical

Slide 13: New idea New idea Execution Execution Feedback Feedback Success? End Feedback Success? Success? End Feedback Money left? Success? End Money End left?

Slide 14: We do this by watching the web.

Slide 15: First: What business are you in?

Slide 16: Media

Slide 17: Transactions

Slide 18: Collaboration

Slide 19: Applications

Slide 20: Then: Know what we want to happen

Slide 21: Users do what we wanted • Enrolment: They sign up • Purchases: They buy stuff • Invitations: They tell their friends • Stickiness: They stay for longer • Loyalty: They come back • Contribution: They add content • Renewal: They buy another subscription • Upselling: They increase their payments

Slide 22: The app is fast & reliable • Uptime: It’s reachable and reliable • Latency: It responds to usage fast • SLAs: Its reliability meets contractual goals • Correctness: It functions as intended • Well maintained: Problems are found & fixed

Slide 23: We understand our visitors • Intentions: We know what they want to do • Motivations: We know why they came to us

Slide 24: The app is easy to use • Easy: Easy to learn, fast to use • Clear: Less confusion • Intuitive: Right stuff in the right place

Slide 25: Our victim…

Slide 26: Analytics receiver Synthetic tester Proxy Browser Data center Client-side Server interpreter Passive capture Survey site Our eyes…

Slide 27: The four big questions What did they do? Could they do it? Why did they do it? How did they do it?

Slide 28: The four big questions What did they do? Could they do it? Web analytics Why did they do it? How did they do it?

Slide 29: What matters in analytics Where did they come from? Acquisition What attracts them best? Did they do what we wanted? Usage Where did they drop out? What appealed most? Where did we send them? Referral

Slide 30: Browser Data center Server IT Analytics receiver

Slide 31: Analytics receiver Browser Data center Client-side Server interpreter

Slide 43: Level 1: Level 2: Level 3: Level 4: Level 5: Table stakes Fix the site Improve traffic Complete view MBA Page views, visits, Path analysis Merchandising Multichannel Multichannel sales visitors Funnel reports Segmentation aggregation reporting Top ten lists A/B testing SEO Cost-shifting Activity-based Demographics KPIs Campaign analysis costing Technographics Dashboards optimization Lifetime value Balanced scorecards Top entry/exit Personas Personalization Strategic planning pages KPI alerts Analytics-based Predictive analytics Performance content serving Integrated user Capacity Process analysis experience Security Strategic web business 330° view of customer (30° privacy) Optimize the channel Business driven, working on metrics, accuracy and process IT-driven, “feel good” information, few decisions Adapted from Stephane Hamel and Bill Gassman

Slide 44: Analytics pros & cons Pros Cons • Trivial to implement with • Requires connection to the <script> tags Internet (for hosted ones) • Can slow page load • Cheap to get started • Tagging pages with content • Absolutely essential data time-consuming • May incite privacy riots • Won’t work when not interpreted – RSS feeds – XML data – Mobile devices – Clicked through before loading

Slide 45: The four big questions What did they do? Could they do it? Web analytics User Experience Management Why did they do it? How did they do it?

Slide 46: What matters in UEM? Could they get to the site? Reachability From everywhere? What regions were worst? What was their experience like? Reliability Did the app break? Was it fast enough? Latency What things were slowest?

Slide 47: The trivial web transaction Browser Data center Server TCP SYN (“let’s talk”) TCP SYN ACK (“Agreed: let’s talk”) TCP ACK (“OK, we’re talking) SSL (“Someone might be listening!”) SSL (“Here’s a decoder ring”) HTTP GET / (“Can I have your home page?”) HTTP 200 OK (“Sure!”) (Thinks [index.html] (“Here it is!”) a bit) (Renders furiously) Bump, bump. [img js css] (“Have this too!”) TCP FIN (“Thanks! I’m done now.”) TCP FIN ACK (“You’re welcome. Have a nice day.”)

Slide 48: What could possibly go wrong? Performance Availability • DNS: Find the site • Client: Bad requests • IP: Route the packets • Server: Capacity, availability, OS • TCP: Establish a connection • Network: Packet loss, TCP • CDN: Redirect/overlay the traffic timeouts, traffic shaping • SSL: Negotiate encryption • Application: Logic, dependencies • HTTP: Request content • Navigation: Looping, session timeouts • Host delay: Generate the content • Content: Broken, unrenderable • Network delay: Deliver the content • RIA: Plugin, memory, OS • Packet loss: Recover from errors • Browser: Parse the page, get more content • RIAs: Execute code on the client • Browser, OS: Render the page

Slide 49: 2 complementary technologies Was it Was it working? broken? Synthetic testing of User monitoring of key functions from every transaction around the Internet

Slide 50: Synthetic testing Was it Was it working? broken? Synthetic testing of User monitoring of key functions from every transaction around the Internet

Slide 51: Synthetic Synthetic … Synthetic tester tester tester Browser Data center Server

Slide 57: http://www.gomez.com/info_center/instant_test.php

Slide 61: Synthetic pros & cons Pros Cons • Easy to set up • Brittle • Only way to test without • Detects macro outages, not traffic user events • Can compare to • Good geographic & network competitors coverage costs money, • Easy baseline establishment generates load

Slide 62: User experience monitoring Was it Was it working? broken? Synthetic testing of User monitoring of key functions from every transaction around the Internet

Slide 63: Browser Data center Server Passive capture

Slide 71: EUEM pros & cons Pros Cons • Maximum visibility for all • Expensive! protocols • Requires physical access* • No load on client • No extra network traffic • Forensics on hand

Slide 72: The four big questions What did they do? Could they do it? Web analytics User Experience Management Why did they do it? How did they do it? Voice of the Customer

Slide 74: What matters in VoC? Why did they visit? Motivation Did they accomplish it? Success Why or why not? Reasons

Slide 75: Browser Data center Client-side Server interpreter Random selection Survey site

Slide 78: EUEM pros & cons Pros Cons • Only way to know what • Depends on survey they really wanted “goodwill” (1-5% of • Good qualitative data respondents) complements quantitative • Popunder techniques • Subjective insights needed • Privacy and personal data collection concerns

Slide 79: The four big questions What did they do? Could they do it? Web analytics User Experience Management Why did they do it? How did they do it? Voice of the Customer Web Interaction Analytics

Slide 80: What matters in WIA? Where did they go? Navigation How did they use the pages? Interaction What did they do wrong? Usability

Slide 81: Operator display Analytics receiver The stage Browser Data center Mouse/key Server capture

Slide 85: So how do I see what the user saw?

Slide 86: Display Analytics receiver Sample sessions’ The stage stored pages Browser Data center Mouse/key & page Server capture

Slide 90: How do I reduce client burden?

Slide 91: Browser Data center Mouse/key Server capture Passive capture Display All sessions stored The stage pages

Slide 93: WIA pros & cons Pros Cons • Realtime usability analysis • Doing it right requires • Easy(ish) to implement dedicated equipment • Great for A/B testing of • Using client to record the layouts stage error-prone – Lousy for long-term dynamic • SSL and secure content sites concerns • Embedded elements (Flash) mislead the recorder

Slide 94: (Just one more) Proxy communications

Slide 95: Proxy Browser Data center Client-side Server interpreter

Slide 100: Proxy pros & cons Pros Cons • Easy way to aggregate • No longer master of your several things own domain • If proxy is a search engine, • May have downtime, delay may inform SEO you’re not aware of

Slide 101: Recap: Four big questions What did they do? Could they do it? Why did they do it? How did they do it?

Slide 103: NO SITE IS AN ISLAND

Slide 106: NON-HTML COMPONENTS

Slide 108: DYNAMIC PAGE NAMES

Slide 110: POP-UPS AND SITE DESIGN

Slide 113: PERFORMANCE WITH LOAD IN MIND

Slide 115: PRIVATE SITES

Slide 116: Analytics receiver Synthetic tester Proxy Data center Server Passive Private capture Survey agents site Browser OS Privacy limits tools agents

Slide 117: CLOUD COMPUTING PLATFORMS

Slide 118: Analytics receiver Synthetic tester Proxy Browser Data center Server Passive capture Survey site Cloud limits server access

Slide 119: AVERAGES LIE

Slide 120: Average varies wildly, making it hard to threshold properly or see a real slow-down. Setting a useful threshold on percentiles gives less false positives and more real alerts 80th percentile only spikes once for a legitimate slow-down (20% of users affected)

Slide 121: GETTING WHAT YOU PAY FOR

Slide 123: STREAMING (COMET/BAYEUX, ADOBE)

Slide 124: How realtime web protocols work Browser Data center API in COMET framework server Here’s a channel to send me updates Got it Subscribe to CSCO HTTP 200 OK (“Sure!”) CSCO: $21 CSCO: $23 Subscribe to GOOG HTTP 200 OK (“Added!”) CSCO: $23 GOOG: $450 Remove CSCO HTTP 200 OK (“Removed!”) …

Slide 125: WATCHING BECOMES THE PROBLEM

Slide 126: Connections to load Bitcurrent Connection 0 - www.bitcurrent.com (67.205.65.12) Connection 1 - www.bitcurrent.com (67.205.65.12) Connection 2 - 4qinvite.4q.iperceptions.com (64.18.71.70) Connection 3 - static.slideshare.net (66.114.49.24) Connection 4 - static.slideshare.net (66.114.49.24) Connection 5 - www.feedburner.com (66.150.96.123) Connection 6 - static.getclicky.com (204.13.8.18) Connection 7 - cetrk.com (208.67.183.100) Connection 8 - in.getclicky.com (204.13.8.18) Connection 9 - crazyegg.com (208.67.180.236) Connection 10 - www.google-analytics.com (72.14.223.147) Connection 11 - www.apture.com (67.192.46.19) Connection 12 - static.apture.com (67.192.46.25) Connection 13 - s.clicktale.net (66.114.49.24) Connection 14 - www.clicktale.net (75.125.82.70)

Slide 127: Leftovers (other places to watch)

Slide 134: Cheat sheet • What they did • Extra credit – Free: Google Analytics, Clicky – Feedburner for RSS – Premium: Omniture, Mint – Google Alerts, Serph for content • Could they do it (Synth)? monitoring – Free: Webperform, Alertsite – Stake out: Facebook, Drop.io, Twitter – Premium: Gomez, Keynote, Webmetrics – Site cred: Compete.com, Technorati, Pagerank • Could they do it (EUEM)? – Free: Log parsers, Wireshark/Ethereal – Premium: Coradiant, Tealeaf, CA, HP • Why did they do it? – Wufoo or Google Forms with Javascript – iPerceptions, Opinionlabs • How did they do it? – Free: Crazyegg, Clicktale, Robotreplay, Tapefailure, GA overlay view – Premium: Tealeaf, Clicktale Pro

Slide 135: Join the conversation

Slide 136: Questions? (alistair at bitcurrent.com)