How to create better A/B tests
based on user research
Karl Gilis
@agconsult

Some of our clients

Part I
A/B testing: the basics

Part II
Main problems with A/B testing

Don't test stupid things

Don't test every idea that pops into your head

Make sure you measure the right goals

Don't copy tests you've seen on other websites

Know why you're running an A/B test

Make sure you've got a solid hypothesis
This example is from Michael Aagaard (@contentverve)

Part III
How user research can help you to
create more successful A/B tests

Which methods can you use?
• Google Analytics
• User testing (Morae)
• Eyetracking (MyGaze eyetracker, Tobii)
• User sessions recording (Mouseflow, Inspectlet, Hotjar)
• Mouse heatmaps (Hotjar, CrazyEgg)
• Form analysis (Formisimo)
• Targeted mini surveys (Qualaroo, Hotjar,…)

User testing with eyetracking

+ 45,9% clicks on "Get a quote"
+ 15,6% clicks on "Book a test drive"

Put your calls-to-action where
people see them.
I know this doesn't sound very innovative. But it’s really important.

User session recording & mouse heatmaps

43,8% more clicks
Why does this always work?
A user who scrolls to the end of your
page is probably pretty interested in
what you have to offer.
You just have to reel him in at the end.

Repeat your CTA at the bottom of
your page.
Believe me. This works. Always.

I'm sorry, but this client doesn't allow me to show the real page and
his brand name. So I made a mock-up.

• 27,2% more clicks on
the red contact buttons.
• 46,0% more clicks on
the blue button.
12,2% more clicks on these boxes
6,6% more clicks below the page fold.

Be careful with big headers & big,
useless photos.
Focus on your message. That will result in more conversions.

Only 1 out 2 visitors started to scroll.
So we made some changes to the first screen. (We're not
allowed to show this in detail. Sorry.)
Those changes have to do with copy. Not so much with
design.

32,5% more people reach this point:
- Our version: 69,45%
- Original: 52,41%
44,1% more people reach this point
100% more visitors go to a product detail page
76,2% more requests for quote
57,8% more brochure downloads
These are the results of those changes…

Use tools to find where the problem
areas are.
And use your brain or user input to make them better.

Form analysis (& user session recording)

Formisimo revealed which fields caused most
problems

52,9% more requests for offer.

Make forms as easy as possible.
And use tools to reveal where the real problems are. Don't guess.

35,4% more forms submitted

Explain why you want certain
information.
Make sure you use positive words. You don't want to frighten the visitor.

Targeted surveys

What is the purpose of your visit to our website today?

+ 188% more clicks
+ 75% more sales from homepage visitors

Push product categories instead of
individual products.
This depends from site to site. But most of the time it works. Big time.
.

What is the purpose of your visit to our website today?

23 times more people use the dealer locator

Integrate first steps of interactions
in your pages.
Ton from @OnlineDialogue calls this the 'foot-between-the-door-trick'
.

What’s important for you when choosing a new car?
Besides the way it looks.

- 7,5% extra clicks to model pages
- Even a bigger increase in number
of requests for offer

Make it easy to choose/compare.
Make sure you use the main decision criteria and emotional triggers of
your users
.

What are the most important things when buying a
used car?

26% more reservations

Place what's important for your
visitors on the left side of your page.
This can sometimes make a difference. Sometimes it doesn't do sh*t.
.

What convinced you the most to decide to ask a
quote?
Was there something that made you hesitate?
If so, what?
77,1% more requests for quote

Know what your clients & visitors
want.
And get filthy rich 
.

Free bonus tip

Get my free ebook now
 10 conversion tips that work on every website
 15 cases from the real world
 6 best practices from leading websites
 www.agconsult.com/ebook.html
 www.slideshare.net/agconsult
Already more than 2.357 downloads

That's it!
Thx