Capture Real-Time User Feedback with Website Intercept Surveys

Research Rewind April 2018
Presented by Ben Yurchak, CEO/Founder at KnowClick
Capture Real-Time User Feedback with
Website Intercept Surveys

Confidential. Copyright © KnowClick 2018
Agenda
1. Background: Role & Types Of Website Surveys
2. ‘How to’ Guide
3. Maximizing Actionable Insight
4. Getting Started

What is a Website Intercept Survey?
Feedback
User-Initiated
Survey
x
Page Intercept
Survey
Website Intercept
Survey

What Unique Role Do Site Intercept Surveys Play in User Research?
Attitudinal
Qualitative
Quantitative
Behavioral
Web Analytics
A/B Testing
Email Surveys
Website Intercept
Surveys
User Research Landscape: modelled on Mulder,
S. and Yaar, Z. (2010 The User is Always Right)
Field Studies
Diary/Journal
Card Sorting
User
Interviews
Focus Groups
Usability
Testing
Typical Recruiting
• By demographic or
customer list
• Actual Visitor “In
the Moment”
Website Intercept
Surveys + Behavior

True or False
True or False? Comments
Response: Very few people will ever take a website survey False (Mostly)
Ranges from <0.1% to
60%+
Bias: survey respondents are not representative True
Extent of bias varies by
survey
Engagement: invitations usually hurt visitor engagement False
They can. Monitor and
optimize.
Inaccuracy: what they say <> what they do True (Mostly)
Capture attitudes &
behavior

What should they measure?
Users
Profile, goals
& outcomes
Business Goals
Performance on
business goals
Diagnostics
Why aren’t users
achieving goal

CASE STUDY 1
Industry: Medical Devices. B2B
Target Audience: Clinicians
Result of KnowClick Insight: 85% Lift in Lead generation
Case Study 1
Goals Existing Performance
1) Lead Generation  1% Lead Gen
2) Persuasion  ~3 Pageviews/visit (no
good measures available)

Case Study 1
“In the moment” site intercept reveals real audience profile & needs
.
Client’s Focus
Groups & 1:1
Website
Audience
[User Research]
Client’s Assumed
Users
User Goals • Simple overview
• Easy to contact a rep
Office staff Clinicians
[Site Intercept]
Actual User & Needs
• In-depth content
• Clinical studies
Patients (non-target)

User Outcome
Case Study 1

Dollarizing Impact
$3M to $8M
Lost Sales
Purchase
Intent At
Arrival
27%
Purchase
Intent At
Exit
18%
Business Outcome
Site decreases purchase intent by 1/3.
Case Study 1
Result: Client agreed to redesign website. 85% lift in lead generation rate.

Website Diagnostics
Resources and Support Section is the Problem.
Case Study 1
Content Persuasion (which content
works/fails)
- +Impact on Purchase Intent
Resources and Support
Testimonials
-52%
Product Overview
Product Features

Agenda
1. Background & Overview
2. “How to” Guide
4. Getting Started

Getting Response: Site Intercept Invitation
A
[website]
B
Wins
1) Appeals to everyone, not just existing customers.
2) Doesn’t “feel” like a typical survey
Loses
Why would you take a “customer
satisfaction survey” if you never bought
from them before?

What users think when they hear “website survey”
Tip: be very different from these invitations
website
>25 questions
All required! 39 Pages!

Getting Response: Use multiple pages and simple initial questions
Respondents are immediately
overwhelmed with many questions
and choices.
Additionally, the questions don’t
make it feel like it is about them and
their needs.
Tip: start with user focused
questions like: Why did they visit?
Did they find what they wanted?

Getting Response: Page Intercept (Auto expanded)
A B
x
x
3% to 15% 0.5% to 3%

Page Intercept: Response Rates from a KnowClick Client
Note: This KnowClick client had products that helped modify/control negative behaviors.

Page Intercept: Measuring the Impact of Invitations on Behavior
Display Impact

Getting Response: Feedback Button
Feedback Button
at Homepage
<0.05% Response Rates
(Typically)

Response Rates w/ Effective Execution
x
Page Intercept
Survey
5% to 60%
Feedback
User-initiated
Survey
Typically <0.05%
Website Intercept
Survey
4% to 10% (Non Bouncers)
Bouncers rarely take an exit survey.

Survey Questions: What to Ask
Example website with a goal to drive sales
Are you driving the right visitors?
 Who is visiting?
 Journey & competitors considering
Are you meeting visitor needs?
 Visit purpose & needs
 Satisfaction
Are you persuading them to want you?
 Perceptions vs competition
 Awareness of value proposition
Are they confident in taking action now?
 Able to decide on product
 Barriers to purchase
Audience
Needs &
Satisfaction
Persuasion
Decision

Survey Questions: Net Promotor Scope (NPS). Use with Caution.
“Considering your experience on this website today, how likely are you to recommend our website to your friends and colleagues?”
Net Promoter Score (NPS) has been extensively used and validated as a great measure of
loyalty among a business’s existing customers.
Using it for website visitors is typically NOT effective.

Survey Question Predictivity: Purchase Intent is far more predictive of outcomes
for Prospects than NPS.
11%
23% 24%
Detractor 0-6 Passive 7-8 Promoter 9-10
Total Purchase Rate for Brand X
by Net Promoter
Score
0%
3%
7%
24%
39%
Very
unlikely
Unlikely Maybe Likely Very
likely
Total Purchase Rate for
Brand X
by Purchase Intent
Limited predictive
power
1%
8%
20%
No Partially Yes
Total Purchase Rate for Brand
X
by Task Completion
at Website

Survey Questions: Net Promotor Scope (NPS). Use with Caution.
6%
2%
4% 5% 5%
48%
7% 8% 7%
3%
5%
0 1 2 3 4 5 6 7 8 9 10
Net Promoter Scope Question
Responses
(among Brand X Prospects)
How likely would you be to recommend [website] to a friend or family member?
Half of website visitors answer “5”
because they never used the
product and don’t know enough to
recommend it or not.

Survey Questions: Make sure they have accurately answer every question
“Did you buy from
[site.com] today?
o Yes
o No
10% said Yes…
… but only 2% of visitors
converted
Why?
- Response bias?
- Question wording?
What they really meant:
I bought at [site.com]
today  2%
I bought at [site.com] in
past week  2%
I bought this product
elsewhere recently  6%
Tip: Change answer options to: Yes, today from the website | No, but I bought the product recently | No

Sample Survey Questions
Purpose
What are you visiting [website.com] to do?
o Get product information
o Order
o …
[If seeking product info…]
What product information are you seeking? (Check all that apply.)
 Features
 Pricing
 …
Did you find the information you wanted? | Were you able to complete the purpose of your visit?
o Yes
o Partially
o No
[If No | Partially]
What were you looking for that you couldn’t find?
_________________
_________________
Satisfaction

Improvements
What change would be most helpful to you?
_________________
_________________
Which changes would be most helpful to you? (Select up to two.)
 Add more product details
 Add ability to compare products
 …
Which best describes your relationship with [company/website]?
o I’ve never bought from [company]
o I bought from company 1-2 times
o I’ve bought from company 3+ times
Brand
Experience

Goals/
Outcomes
How likely are you to [take action X] in the next [Y] days?
o 1 Not at all likely
o 2
o 3
o 4
o 5 Very likely
How did this visit impact your interest in [taking action X]?
o Increased
o No change
o Decreased
Examples of “[take action X]” by industry
Brand  Buy brand X
eCommerce/Retail  Buy from company X
Financial Services  Talk to an advisor | Open an account
Informational website  Return to website X the next time you need this
Pharmaceuticals  Ask your doctor about brand X

Agenda
1. Background & Overview
2. “How to” Guide
– Connecting Survey Response to Behavior
– Measuring Offline Impact
– “Dollarizing” Impact
4. Getting Started

Survey:
Audience & Attitudes
Feedback Intentions
Profile &
Journey
Offline
Actions
Competitor
Purchase
CallOffline
Purchase
Nothing
+
Website
Behavior
Content
Viewed
Traffic
Source
Conversion
Funnel
+
Capturing Entire Journey & the Impact of Every Interaction

64%
36%
Task Completion
Website Satisfaction
Dissatisfied
Satisfied
Survey Only
Actions 
?
25%
31%
60% 62%
74%
Product
Section C
Product
Section B
Product
Section A
Low Price
Guarantee
Buying Guide
%CompletingTask
Satisfaction by Content Viewed
Survey Integrated with Behavior
Actions 
1. Prioritize improvements to Sections B & C
2. Add product comparisons to Sections B & C
3. Increase prominence of Buying Guide
Satisfaction by Content: Reveals the cause.

Case Study 2
PROBLEM
Conversion Rates: Mobile
is 1/3 of Desktop
1.6%5.2%
CAUSE
1) Intent?
2) Mobile Experience?
CASE STUDY 2: Mobile vs Desktop
Result: Client made changes to mobile that increased only revenue by $15MM a year.
Purpose = “Buy Online Now”
Converted
Entry
36% 13%
The mobile
experience IS
the problem

CASE STUDY: Retailer w/ online appointments.
Client Assumptions
Price too high?
Not intending to buy?
Existing Research
Surveys User
Testing
Web
Analytics
WHY?
90% Abandon.
WHY?!
Case Study 3

[Survey]
Want appointment
today*
$1.5M Sales
per year
[Survey]
Want appointment
tomorrow
$6.7M Sales
per year
Dollarizing Impact
$5.2M
Lost Sales
CASE STUDY 5
Cause 1: Lack of same day appointments costs ~$5M.
*Same day appointments not supported
Case Study 3

CASE STUDY 5
The product page says it is in-
stock, but many don’t notice.
The Cart page doesn’t confirm in-
stock status.
40% abandon due to uncertainty whether
the product is in stock.
Cause #2: Uncertainty that product is really in stock.
Case Study 3
Cart abandoners

Dollarizing Impact
$6M to $12M
Lost Sales
CASE STUDY 5
Bought offline
from company
Bought from
competitor
Did nothing
Competitors win half of these abandoners, costing $6M to $12M.
Cause #2: Uncertainty that product is really in stock.
Case Study 3
Cart abandoners

CASE STUDY 5
Results from changes based on these insights:
3X lift in online conversions. +$29 million lift in sales.
Case Study 3

Choosing Right Tactic
x
Page Intercept
Survey
Feedback
User-initiated
Survey
Website Intercept
Survey
Use
For:
High frustration problems
at a page (technical error, etc)
Monitoring
Minor Optimization
Page audience,
performance,
improvements.
Optimization.
Website audience,
performance,
improvements.
Redesign
Optimization
Planning

When to Use
Website Redesign Site Intercept. Run before to inform & after to measure impact.
Goal/Project Tactic
Website Optimization Site Intercept & Page Surveys. Optimize questions too.
Audience Needs or
Proof of Concept
Simple Site Intercept. Launch at arrival.
Page Improvements or
Problem Diagnosis
Page survey. Launch after delay (or scroll to bottom) or exit.
Find Technical Problems User Initiated Feedback. (Or Site Intercept)

DIY vs Advanced
DIY
1. Best for small websites & apps or
simple problems
2. User-initiated surveys (easy)
3. Initial on-page surveys
• Visitor needs
• Top improvements
4. Simple site intercept
• Audience profile & satisfaction
• Basic user feedback
Advanced / Use Experts
1. Best for larger website & getting more
actionable insight
2. Redesign: in-depth insight to guide
redesign & build case for the project
3. Optimization: diagnosing the top
problems
4. Dollarizing website impact
5. Starting a digital voice of customer
program
6. Integration w/ behavior and analysis

Getting Started: if DIY
Initial Tactics to Start
1. 1-3 question page survey
• Test different first questions
• Launch at high traffic page
2. 4-8 question arrival or exit site intercept
• Audience, purpose, satisfaction
• Test invitation verbiage
Checklist
 Simple invitation
 Multiple invitation/question tests
Questions
 Appeal to all visitors
 Use conditional questions
 <30% open ended
 Measure impact of displaying invite
 If you have the capability, pass some
answers to web analytics tool (analysis in
web analytics tools is difficult though)

Questions?
KnowClick.com
ben@knowclick.com
215.840.2703

Capture Real-Time User Feedback with Website Intercept Surveys

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Capture Real-Time User Feedback with Website Intercept Surveys

Similar to Capture Real-Time User Feedback with Website Intercept Surveys (20)

Recently uploaded

Recently uploaded (20)

Capture Real-Time User Feedback with Website Intercept Surveys