Watching websites

What we’re going to cover

Why watch your website?

How to do it (well & affordably)

STARTUP 101

New idea End Execution Feedback Success? End Money left? Risk

Feedback, the old way.

Are people doing what we want?

Are we doing dumb stuff?

Do we understand our users? http://flickr.com/photos/ikhlasulamal/2443194039/

Is it easy and intuitive? http://flickr.com/photos/jmecelab/2323995433/

Slow, unmeasured trial & error.

The Internet lets us make mistakes faster.

The weird part: Mistakes are good

Most startups don’t succeed doing what they set out to do

Amazon was just a bookstore

eBay sold Pez

F5 made hurricane modeling software

But mistake speed is critical

New idea Execution Money left? End Feedback Success? New idea Execution Money left? Feedback Success? End Feedback Success? End Feedback Success? End

We do this by watching the web.

First: What business are you in?

Media

Transactions

Collaboration

Applications

Then: Know what we want to happen

Users do what we wanted

Enrolment: They sign up

Purchases: They buy stuff

Invitations: They tell their friends

Stickiness: They stay for longer

Loyalty: They come back

Contribution: They add content

Renewal: They buy another subscription

Upselling: They increase their payments

The app is fast & reliable

Uptime: It’s reachable and reliable

Latency: It responds to usage fast

SLAs: Its reliability meets contractual goals

Correctness: It functions as intended

Well maintained: Problems are found & fixed

We understand our visitors

Intentions: We know what they want to do

Motivations: We know why they came to us

The app is easy to use

Easy: Easy to learn, fast to use

Clear: Less confusion

Intuitive: Right stuff in the right place

Our victim…

Our eyes… Browser Data center Synthetic tester Analytics receiver Proxy Survey site Client-side interpreter Server Passive capture

The four big questions What did they do? Could they do it? Why did they do it? How did they do it?

The four big questions Web analytics What did they do? Could they do it? Why did they do it? How did they do it?

What matters in analytics Where did they come from? What attracts them best? Did they do what we wanted? Where did they drop out? What appealed most? Where did we send them? Acquisition Usage Referral

Data center Browser IT Analytics receiver Server

Data center Browser Client-side interpreter Server Analytics receiver

Page views, visits, visitors Top ten lists Demographics Technographics Top entry/exit pages Performance Capacity Security Path analysis Funnel reports A/B testing KPIs Dashboards Merchandising Segmentation SEO Campaign optimization Personas KPI alerts Multichannel aggregation Cost-shifting analysis Lifetime value Personalization Analytics-based content serving Process analysis Multichannel sales reporting Activity-based costing Balanced scorecards Strategic planning Predictive analytics Integrated user experience Adapted from Stephane Hamel and Bill Gassman IT-driven, “feel good” information, few decisions Business driven, working on metrics, accuracy and process Optimize the channel 330° view of customer (30° privacy) Strategic web business Level 1: Table stakes Level 2 : Fix the site Level 3 : Improve traffic Level 4: Complete view Level 5: MBA

Analytics pros & cons

Pros

Trivial to implement with <script> tags

Cheap to get started

Absolutely essential

Cons

Requires connection to the Internet (for hosted ones)

Can slow page load

Tagging pages with content data time-consuming

May incite privacy riots

Won’t work when not interpreted

RSS feeds

XML data

Mobile devices

Clicked through before loading

The four big questions Web analytics User Experience Management What did they do? Could they do it? Why did they do it? How did they do it?

What matters in UEM? Could they get to the site? From everywhere? What regions were worst? What was their experience like? Did the app break? Was it fast enough? What things were slowest? Reachability Reliability Latency

The trivial web transaction Data center Browser TCP SYN (“let’s talk”) TCP SYN ACK (“Agreed: let’s talk”) TCP ACK (“OK, we’re talking) HTTP 200 OK (“Sure!”) TCP FIN (“Thanks! I’m done now.”) [index.html] (“Here it is!”) TCP FIN ACK (“You’re welcome. Have a nice day.”) HTTP GET / (“Can I have your home page?”) Server SSL (“Someone might be listening!”) SSL (“Here’s a decoder ring”) [img js css] (“Have this too!”) Bump, bump. (Renders furiously) (Thinks a bit)

What could possibly go wrong?

Performance

DNS: Find the site

IP: Route the packets

TCP: Establish a connection

CDN: Redirect/overlay the traffic

SSL: Negotiate encryption

HTTP: Request content

Host delay: Generate the content

Network delay: Deliver the content

Packet loss: Recover from errors

Browser: Parse the page, get more content

RIAs: Execute code on the client

Browser, OS: Render the page

Availability

Client: Bad requests

Server: Capacity, availability, OS

Network: Packet loss, TCP timeouts, traffic shaping

Application: Logic, dependencies

Navigation: Looping, session timeouts

Content: Broken, unrenderable

RIA: Plugin, memory, OS

Synthetic testing of key functions from around the Internet User monitoring of every transaction 2 complementary technologies Was it working ? Was it broken ?

Synthetic testing Synthetic testing of key functions from around the Internet User monitoring of every transaction Was it working ? Was it broken ?

Data center Browser Synthetic tester Synthetic tester … Server Synthetic tester

http://www.gomez.com/info_center/instant_test.php

Synthetic pros & cons

Pros

Easy to set up

Only way to test without traffic

Can compare to competitors

Easy baseline establishment

Cons

Brittle

Detects macro outages, not user events

Good geographic & network coverage costs money, generates load

User experience monitoring Synthetic testing of key functions from around the Internet User monitoring of every transaction Was it working? Was it broken?

Data center Browser Server Passive capture

EUEM pros & cons

Pros

Maximum visibility for all protocols

No load on client

No extra network traffic

Forensics on hand

Cons

Expensive!

Requires physical access *

The four big questions Web analytics Voice of the Customer User Experience Management What did they do? Could they do it? Why did they do it? How did they do it?

What matters in VoC? Why did they visit? Did they accomplish it? Why or why not? Motivation Success Reasons

Data center Browser Client-side interpreter Server Random selection Survey site

EUEM pros & cons

Pros

Only way to know what they really wanted

Good qualitative data complements quantitative

Subjective insights

Cons

Depends on survey “goodwill” (1-5% of respondents)

Popunder techniques needed

Privacy and personal data collection concerns

The four big questions Web analytics Voice of the Customer User Experience Management Web Interaction Analytics What did they do? Could they do it? Why did they do it? How did they do it?

What matters in WIA? Where did they go? How did they use the pages? What did they do wrong? Navigation Interaction Usability

Data center Browser Operator display The stage Mouse/key capture Server Analytics receiver

So how do I see what the user saw?

Data center Browser Display The stage Sample sessions’ stored pages Mouse/key & page capture Server Analytics receiver

How do I reduce client burden?

Data center Browser Display The stage All sessions stored pages Mouse/key capture Server Passive capture

WIA pros & cons

Pros

Realtime usability analysis

Easy(ish) to implement

Great for A/B testing of layouts

Lousy for long-term dynamic sites

Cons

Doing it right requires dedicated equipment

Using client to record the stage error-prone

SSL and secure content concerns

Embedded elements (Flash) mislead the recorder

(Just one more) Proxy communications

Browser Data center Proxy Client-side interpreter Server

Proxy pros & cons

Pros

Easy way to aggregate several things

If proxy is a search engine, may inform SEO

Cons

No longer master of your own domain

May have downtime, delay you’re not aware of

Recap: Four big questions What did they do? Could they do it? Why did they do it? How did they do it?

NO SITE IS AN ISLAND

NON-HTML COMPONENTS

DYNAMIC PAGE NAMES

POP-UPS AND SITE DESIGN

PERFORMANCE WITH LOAD IN MIND

PRIVATE SITES

Privacy limits tools Data center Browser Synthetic tester Analytics receiver Proxy Survey site Server Private agents Passive capture OS agents

CLOUD COMPUTING PLATFORMS

Cloud limits server access Data center Browser Synthetic tester Analytics receiver Proxy Survey site Server Passive capture

AVERAGES LIE

80 th percentile only spikes once for a legitimate slow-down (20% of users affected) Average varies wildly, making it hard to threshold properly or see a real slow-down. Setting a useful threshold on percentiles gives less false positives and more real alerts

GETTING WHAT YOU PAY FOR

STREAMING (COMET/BAYEUX, ADOBE)

How realtime web protocols work Data center Browser … HTTP 200 OK (“Sure!”) Here’s a channel to send me updates CSCO: $21 Subscribe to CSCO COMET server API in framework CSCO: $23 HTTP 200 OK (“Added!”) Subscribe to GOOG CSCO: $23 GOOG: $450 HTTP 200 OK (“Removed!”) Remove CSCO Got it

WATCHING BECOMES THE PROBLEM