Complete Web Monitoring at Web 2.0 Expo - Part 1 - Analytics

Complete Web Monitoring A report from the frontlines of the web

I betook myself to linking...

...tis some visitor entreating entrance at my chamber door

What kind of site are you?

About your company

What’s your job? Webops (it’s up, and it’s fast) User experience (it’s easy to use) Community management and monitoring Market research (what people think and why) Support Other

What we’ll cover Analytics, interaction, UX, Voice of the Customer EUEM, synthetic tests, RUM Online communities, internal communities Competitive analysis Integrating data sources

Which pretty much means We’re going to waste some of your time.

What’s complete web monitoring?

Smart companies make mistakes faster

New idea Execution Feedback

New idea Execution Feedback Success?

New idea Execution Feedback Success? Money left?

New idea Execution Feedback Success? Money left? End

New idea Execution Feedback End Success? Money left? End

New idea Execution Feedback End Success? Money left? Risk End

New idea Execution Feedback End Success? Money left? End

New idea New idea Execution Execution Feedback Feedback End Success? Money left? End

New idea New idea Execution Execution Feedback Feedback End Success? End Success? Money left? End

New idea New idea Execution Execution Feedback Feedback End Success? End Success? End Success? Money left? End

New idea New idea Execution Execution Feedback Feedback End Success? End Success? End Success? End Success? Money left? End

New idea New idea Execution Execution Feedback Feedback End Success? End Success? End Success? End Success? Money left? Money left? End

New idea New idea Execution Execution Feedback Feedback End Success? End Success? End Success? End Success? Money left? Money left? End End

Mistakes are good Most startups don’t win at what they set out to do Amazon was just a bookstore eBay sold Pez F5 made hurricane modeling software Youtube was video hot-or-not Mistake speed is critical

Who you are drives what you watch

Media sites

Transactional sites

Collaborative sites

SaaS sites

Users do what we wanted Enrolment: They sign up Purchases: They buy stuff Invitations: They tell their friends Stickiness: They stay for longer Loyalty: They come back Contribution: They add content

What could we watch? What we’d like to know Tool set How much did visitors beneﬁt my business? Internal analytics Where is my trafﬁc coming from? External analytics What’s working best (and worst?) Usability testing How good’s my relationship with my market? Customer surveys, community How healthy is my infrastructure? Performance monitoring How am I doing against my competitors? Search, external testing Where are my risks? Search, alerting What are people saying about me? Search, community monitoring How is my content being used elsewhere? Search, external analytics

How much did visitors beneﬁt my business? Internal analytics Conversion and Billing and account use abandonment Click-throughs Ofﬂine activity User-generated content Subscriptions

Where’s my trafﬁc coming from? External analytics Referring websites Inbound links from social networks Visitor motivation

What’s working best (and worst)? Usability testing, A/B testing Site effectiveness Trouble ticketing and escalation Upselling effectiveness Content popularity Ad and campaign effectiveness Usability Findability and search User productivity effectiveness Community ranking and rewards

How good is my relationship with my market? Customer surveys, community monitoring Loyalty Enrollment Reach and rewards

How healthy is my infrastructure? Performance monitoring Availability and Impact of performance performance on outcomes SLA compliance Content delivery Capacity and ﬂash trafﬁc

How am I doing against my competitors? Performance monitoring Site popularity and ranking How are people ﬁnding my competitors? Relative site performance Competitor activity

Where are my risks? Search, alerting Trolling and spamming Copyright and legal liability Fraud, privacy, and account sharing

What are people saying about me? Search, community monitoring Site reputation Trends Social network activity

How is my content being used elsewhere? Search, external analytics API access and usage Mashups, stolen content, and illegal syndication Integration with legacy systems

The tools at our disposal...

Collection Server (logﬁles)

Server collection Browser Data center Server Analytics receiver

Server collection Browser Data center Server IT Analytics receiver

Collection Server (logﬁles) Inline (snifﬁng)

Sniffer collection Browser Data center Server Passive capture

Collection Server (logﬁles) Inline (snifﬁng) Browser (JS)

Analytics receiver Marketing Browser Data center Server Browser collection

Analytics receiver Marketing Browser Data center Client-side Server interpreter Browser collection

Collection Server (logﬁles) Inline (snifﬁng) Browser (JS) Interception

Interception (survey) collection Browser Data center Server Survey site

Interception (survey) collection Browser Data center Client-side Server interpreter Survey site

Interception (survey) collection Browser Data center Client-side Server interpreter Random selection Survey site

Collection Server (logﬁles) Inline (snifﬁng) Browser (JS) Interception

Collection Search & crawlers Server (logﬁles) Search engines Inline (snifﬁng) Platform search Browser (JS) Interception

Collection Search & crawlers Testing Server (logﬁles) Search engines Test scripts Inline (snifﬁng) Platform search Browser (JS) Interception

Synthetic tester Browser Data center Server Synthetic testing

Synthetic … Synthetic Synthetic tester tester tester Browser Data center Server Synthetic testing

Analyzing data properly Always compare Segment everything Don’t settle for averages

A complete web monitoring maturity model Level 1: Technical details Level 2: Minding your own house Level 3: Engaging the Internet Level 4: Building relationships Level 5: Web business strategy

Web analytics What did they do?

Arts & crafts time!

≥≥≥

Web Analytics is the Mother Ship

A brief history of analytics

Major Issues: WWW needed to become mainstream Analytics needed to stop relying on HITS We needed the ability to segment

blah blah blah ... A visitor arrives at your website, possibly after following a link that referred them. They land on a web page, and either bounce (leave immediately) or request additional pages. In time, they may complete an transaction that’s good for your business, converting them from a mere buyer into something more—a customer, a user, a member, or a contributor—depending on the kind of site you’re running. On the other hand, they may abandon that transaction and ultimately exit the website. That visitor has many external attributes—such as the browser they’re using, or where they’re surﬁng from—that let you group them into segments. They may also see different offers or pages during their visit, which are the basis for further segmentation. The goal of analytics, then, is to maximize conversions by optimizing your website, often by experimenting with different content, layout, and campaigns, and analyzing the results of those experiments on various internal and external segments.

Questions?

let’s get practical

The Three Stages of Analytics Finding the website Using the website Leaving the website

How did they get there

Direct Trafﬁc

Type-in Trafﬁc

Bookmarking

Desktop client

Javascript redirection

Browser inconsistency

Bots, spiders and probes

Organic search

Paid search

Referrals

Linkbacks

Places & Tasks

Landing page: View one story

Landing page: View one story Task: Log in Enter credentials Verify Recovery

Landing page: View one story Task: Log in Enter credentials Verify Recovery Task: Forward a story Enter recipients Enter message Send

Landing page: Task: View one story Create account Task: Log in Pick name Enter credentials Check if free Verify Set Password Recovery CAPTCHA Send mail Get conﬁrm Task: Forward a story Enter recipients Enter message Send

Landing page: Task: View one story Create account Task: Log in Pick name Enter credentials Check if free Verify Set Password Recovery CAPTCHA Send mail Get conﬁrm Task: Forward a story Task: Submit Enter recipients a new story Enter message Send Enter URL Describe Deduplicate Post it

Landing page: Task: View one story Create account Task: Log in Place: View stories Pick name Enter credentials Check if free Vote up Next 25 Verify Set Password Vote down Last 25 Recovery CAPTCHA Send mail Get conﬁrm Task: Forward a story Task: Submit Enter recipients a new story Enter message Send Enter URL Describe Deduplicate Post it

Landing page: Task: View one story Create account Task: Log in Place: View stories Pick name Enter credentials Check if free Vote up Next 25 Verify Set Password Vote down Last 25 Recovery CAPTCHA Send mail Place: Read Get conﬁrm poster comments Vote up Next 25 Task: Vote down Last 25 Forward a story Task: Submit Enter recipients a new story Enter message Send Enter URL Describe Deduplicate Post it

Landing page: Task: View one story Create account Task: Log in Place: View stories Pick name Enter credentials Check if free Vote up Next 25 Verify Set Password Vote down Last 25 Recovery CAPTCHA Send mail Place: Read Get conﬁrm poster comments Vote up Next 25 Task: Vote down Last 25 Forward a story Task: Submit Enter recipients a new story Place: My Enter message account Send Enter URL Change My Describe address comments Deduplicate Change PW See karma Post it

Landing page: Create acct. View one story Task: Log in Place: View stories Place: Read poster comments Task: Forward a story Task: Submit a new story Place: My account

Landing page: Create acct. Create acct. View one story Form uptime Task: Log in Place: View stories # started Bad form Place: Read # CAPTCHA poster comments Mail uptime Task: Forward a story Mail bounced Task: Submit a new story Place: My Conﬁrm & return account Return 3x

Landing page: Create acct. View one story Task: Log in Place: View stories Place: View stories Stories/visit # up/down Place: Read poster comments Time/story Top stories Task: Forward a story Refresh time Views/page Task: Submit a new story Place: My account

Landing page: Create acct. View one story Task: Log in Place: View stories Place: Read poster comments Task: Forward a story Task: Submit a new story Place: My account

Places Efﬁciency matters How quickly, how many, productivity Learning curve OK Leave when they’re bored Collect “aha” feedback A/B test content for pages/session, exits

Tasks Effectiveness matters Completion, abandonment Intuitiveness rules Leave when they change their mind or it breaks Collect “motivation” feedback A/B test layouts for conversion

What did they do?

Parting is such sweet sorrow

Undesirable outcomes

Abandonment & bounce rate

Attrition

Desirable outcomes

Implementing analytics

Deﬁning your site’s goals Setting up data capture Setting up ﬁltering Identifying segments Tagging page content Verifying taht everything is working

Identifying segments you want to analyze

Determine your goals, challenge your assumptions

Know what normal is

Make a prediction

Test against the original

Run the tests for long enough

Make sure both A and B had a fair shake

Web Interaction Analytics Why did they do it?

http://www.ﬂickr.com/photos/americanlady/3118301118 consume http:// give data navigate

Four kinds of issues

Visitors don’t see what you wanted them to

Visitors don’t interact as you intended

Visitors don’t input edata correctly or completely

We have no idea what’s happening on the site

But your mileage will vary

Usability and affordance

Why you should care about mouse interactions

segmenting clicks is also important

All of these aggregate views help you to understand what’s most popular, which links don’t lead to desired outcomes, and places where visitors are clicking mistakenly. In other words, they conﬁrm—or contest—the de made when you ﬁrst built the website.

when people abandon forms (oh noes!)

How many visitors arrived at the form and were shown it correctly? How many visitors started to fill out the form? How far down the form did they get? How long did they linger at each field? What was the abandonment for each field? How many of them submitted the form? How many of them submitted the form but missed required fields?

Stalking efﬁciently: replay

Are my designs working for real visitors? Why aren’t conversions as good as they should be? Why is this visitor having issues? Why is this problem happening? What are the steps needed to test this part of the application? How do I show them it’s not my fault?

... so .. is my design working?

how can get some of that?

ﬁrst .. ask yourself a few questions

paths that drive my business

places that drive users to ask questions (support)

Places that drive productivity

Issues and concerns

What you wont see with WIA

Voice of the Customer Why did they do it?

they aren’t doin’ what you think...

the travel industry’s dilemma

they really aren’t doin’ what you think...

so what’s this “VOC” thing?

get new ideas

evaluate things you can’t ﬁnd out elsewhere

see if the cool shiny improvements you just

collect demographics data

“customer insights do not come from quantitative market research. You cannot generate insight out of numbers. Numbers help you to validate insights. A

What’s in it for you?: insight and clues

What’s in it for you?: subjective scoring

What’s in it for you?: demographics

What’s in it for you?: surfographics

What’s in it for you?: collection of visitor data

What VOC isn’t (or shouldn’t be)

not a substitute for other forms of collection

not representative of your user base

not an alternative to a community

not a substitute for enrollment

Kickin’ it off

Planning the study

Before/after comparison

Demographics comparison

A/B comparison of two choices

Comparing the impact of website latency on

Comparing experience based on actions

What kind of questions to ask

Evaluation

Recall

General feedback and exploration

Mindset

Designing the study’s navigation

Why surveys fail

never ever ever ever ever ever ever ever ever ever ever

VOC in action

but the downsides are ...

How many people should I ask?

Encouraging participation

overlays

If things aren’t going right ...

high recruitment and bounce rates

low recruitment response rates

poor interception response rates

poor start rates

poor completion rates

http://www.yucezerey.com	49
http://www.slideshare.net	16
http://pathtorevenue.wordpress.com	13
http://claudelessard.wordpress.com	10
http://djomba.blogspot.com	2
http://balakbangalore.blogspot.in	2
http://peterjohnin.wordpress.com	2
http://balakbangalore.blogspot.com	2
http://clodlessard.wordpress.com	1
http://hvrn.dropmark.com	1
http://road3.com	1
http://www.linkedin.com	1

Complete Web Monitoring at Web 2.0 Expo - Part 1 - Analytics

by Alistair Croll and Sean Power on Apr 05, 2009

Accessibility

Categories

Upload Details

Usage Rights

12 Embeds 100

Statistics

Complete Web Monitoring at Web 2.0 Expo - Part 1 - Analytics Presentation Transcript