How to Run AB Tests that Get REAL Results

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

How to Run A/B Tests that Get REAL Results

Lars Lofgren
Marketing Analyst - July 2012 info@kissmetrics.com - Confi...

@larslofgren #KISSwebinar

We’ll cover...

1 The wrong way to A/B test

2 5 places to get ideas for testing

3 Get the most from your A/B te...

HOW MOST PEOPLE
A/B Test

A/B testing is totally rad! We should do it!

Everyone loves button colors

VS

Next up? Layouts!

What about photos?

VS

And conversions went up by 4%! Party time!

There’s a better way...

Testing random elements is very
inefficient. You will not find the big
...

You’re also not learning

Red vs green buttons doesn’t help you
understand your customer.

GET
Qualitative

Qualitative data definition

Data that isn’t a number.

Start with qualitative insights

Use other sources of data to get a
basic understanding of what your
...

Understand first, then do the A/B test

You’ll focus on the right tests and
increase your conversions a LOT
...

The A/B Testing Process

1 Get qualitative insights

2 Predict how to improve

3 Confirm prediction with an A/...

Why is this process better?

Helps you find the 40% increase in
conversions instead of only the 4%.

WHERE IS THE QUALITATIVE
Data?

Location #1

Surveys

Use one-question surveys on your site

Also try long surveys

Survey best practices

1 Keep it short

2 Only ask for answers that you’ll use

3 Start with open-ended questions

Location #2

Feedback

Do you have a feedback email?

Talk to your support team

They know exactly what problems
keep coming up.

Also talk to your sales team

They talk to customers every day.

Location #3

Reach Out

Nothing beats talking to customers personally

Get them on the phone and deeply
understand how they use your
...

If local, take them out to lunch or coffee.

This is not a pitch

Focus on asking good questions and
listening.

Location #4

User Activity

Look at activity of individual people

You’ll need customer analytics for
this.

KISSmetrics People report

When you have an insight...

Confirm by looking at other people.

Location #5

Usability Tests

What’s a usability test?

Have someone use your site or
product while you watch.

Grab someone off the street

Amazon’s Mechanical Turk - mturk.com

UserTesting.com

Places to Find Qualitative Insights

1 Surveys

2 Feedback

3 Reach Out

4 User Activity

5 Usability Te...

The A/B Testing Process

1 Get qualitative insights

2 Predict how to improve

3 Confirm prediction with an A/...

When should you test the small stuff?

After you’ve found the big wins and
you have a full-time growth team.

A/B TESTING
Tools

A/B testing tools only measure the next step

Great for testing copy, photos, and
layouts. Terrible for bigger te...

Tie A/B tests back to the customer life-cycle

Critical for measuring how your
changes impact customers over th...

How do we actually do this?

First, make sure you’re using customer analytics

Then use Visual Website Optimizer or Optimizely

OR

Now all your A/B testing data is in KISSmetrics

For a demo of KISSmetrics or to learn
how KISSmetircs can help grow your
business, contact Ben Sardella at:

bsa...

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

Human resonance for leaders pap 2015 by Bernhard K.F. Pelzer 2459 views
Why So Many Businesses are Hopeless... by Passle 5268 views
Email Optimization: A discussion ab... by MarketingSherpa 5022 views
How to Start Using Analytics Withou... by Kissmetrics on Sl... 3412 views
Using Single Keyword Ad Groups To D... by Sam Owen 4821 views
Brand Equity Measures by sat2coolguy 2156 views

View on Slideshare

1 of 50 Ad

View on Slideshare

1 of 50 Ad