This presentation highlights the key points of our competitor testing article, which can be found here: http://bit.ly/CompetitorTesting A lot of competitor tests come our way here at UserTesting’s Research Team. It’s great to learn how your site measures up in relation to your competitor’s site, but there are a couple of different methods of comparing sites that are appropriate for different situations. To get the feedback you need, you’ll have to decide which method to use and how to structure your study. But not to fear! We’re here to explain the differences between the two methods so you can determine which type of study is right for you.

1. Test Your Competitors
with These 2 Methods

2. It’s great to learn how your site measures
up in relation to your competitor’s
site, but there are a couple of different
methods of comparing sites that are
appropriate for different situations.
To get the feedback you need, you’ll have
to decide which method to use and how
to structure your study.
Introduction

3. But not to fear!
We’re here to explain the differences between the
two methods so you can determine which type of
study is right for you.
Introduction

4. Introduction
For starters, if you’re unfamiliar with competitor
testing, check out this article on how and why to
test your competitors’ sites.
Once you’ve got the hang of how competitor tests
work, keep reading!

5. Why are there two
different methods?

6. Here’s the problem:
If you want users to complete a full test on your
site and a full test on a competitor’s site, it’s going
to take twice as long.
Why Two?

7. Why Two?
And while you can always offer an extra incentive for
tests that take extra time, humans will be humans.
Test participants are going to get tired and frustrated.

8. We found out, after doing many of these
tests, that you don’t always have to send
users to multiple sites to get comparable
results among competitors.
Why Two?

9. What’s more, there are a few ways to go about
getting these competitor insights without wearing
users out, whether the user is viewing:
Multiple Sites
(Direct Comparison)
or Just One Site
(Objective Comparison)
Why Two?
vs.

10. Direct Comparison Testing

11. Direct Comparison
This type of testing helps when you want to identify
specific parts of your site that your competition
does better. It can be done within a fairly short
amount of time.
vs.

12. How to do it
You have a handful of sites you want to test:
Your site, and, let’s say, your top 3 competitors.
It might seem efficient to test all of them at once,
but that would be exhausting for users, and the
results likely wouldn’t be very helpful.
Direct Comparison

13. Instead, test specific parts of each site against each
other.
For example:
• Test your site’s checkout process to Competitor A’s
checkout process.
• Then, with a different batch of testers, do the same
thing with your site vs. Competitor B, and so on.
Direct Comparison

14. Advantages
When you want to implement something that your
competition does, but don’t know the best way to
do it, you can test one competitor against another
to see which one users prefer.
Direct Comparison

15. Direct Comparison
You can also ask users outright which site
they would choose to buy from. It can be
very helpful to hear a user explain their
choice, like in the following clip.

16. Objective Comparison Testing

17. Let’s say you want to gain an understanding of how
your site is doing compared to your competition
over time. This is where Objective Comparison
Testing comes into play.
Objective Comparison

18. How to do it
For an objective approach, you’ll create a standard
set of questions, and then you’ll run the same test
for each site separately. Then, you’ll do it again
whenever either site is updated.
You can use a tool like ChangeDetection or Website Alert
to get notifications about any updates on a site.)
Objective Comparison

19. Advantages
This method is great because you don’t need to
show multiple sites in one test. It won’t fatigue
users; in fact, they’ll never even know that the test
is part of a comparison study. To them, it’ll just be a
regular test of a regular site.
Objective Comparison

20. On the surface, you won’t get the immediate
feedback of, “I preferred the first site,” like you
would in a Direct Comparison Test.
You’ll have to watch every test video, analyze the
metrics, and draw the conclusions on your own.
Objective Comparison

21. You will gain an initial baseline for your site and your
competition, and then, over time, you’ll have a basis
for several different comparisons.
Objective Comparison

22. Your current site vs.
Your competitor’s current site
Which do users prefer and why?
Objective Comparison
vs.

23. Your competitor’s newest version vs.
Your competitor’s previous version
Did they make important improvements?
What can you learn from them?
Objective Comparison
vs.

24. Your newest version vs.
Your previous version.
Did your updates make a positive difference
in the user experience of your site?
Objective ComparisonObjective Comparison
vs.

25. Iterating is inherently apparent with Objective
Comparison Testing. When using this method, you’ll
need to run tests on each site whenever there is an
update. Simply testing each site once is not enough.
(This method is used to track progress over a
longer period of time, and is not meant to help fix
small issues that your site is facing today.)
Objective Comparison

26. Use competitor
insights to get a leg
up on the competition

27. Get a Leg Up
To recap, it’s important to do some kind of
competitor testing to make sure your site can stay,
well, competitive with other companies.
Here’s a quick overview of the two types of
competitor testing you can do:

28. Think about your goals to determine which method to use.
Get a Leg Up

29. To get started and see how UserTesting can help
your company, sign up for a free trial here.
Get a Leg Up

30. www.usertesting.com