This document discusses the importance of usability testing for websites. It notes that usability testing focuses on making sure a website is easy to use and understand for people of average or below average ability. The document then discusses different types of usability testing like A/B testing, multivariate testing, unmoderated remote user testing, and moderated user testing. It provides examples of each type of testing and highlights how tests can significantly increase key metrics like conversion rates and form submissions. The overall message is that usability testing is crucial for optimizing conversions and improving the user experience of a website.

1. LITTLE THINGS
MEAN A LOT
JUNE 3, 2014
THE IMPORTANCE OF USABILITY TESTING

2. ABOUT TOP FLOOR
TOP FLOOR MAINTAINS DEDICATED TEAMS:
!
• MARKETING STRATEGY
• DIGITAL DESIGN
• DIGITAL DEVELOPMENT
• INTERNET MARKETING
• SEARCH AND ANALYTICS
• CONVERSION OPTIMIZATION
• BRANDING
• INTEGRATED MARKETING SERVICES
14 YEARS OF DELIVERING DIGITAL MARKETING RESULTS

3. ABOUT ME
PHOTO
(329W X 357H)
• 20 YEARS DIGITAL MARKETING EXPERIENCE
• HUNDREDS OF WEBSITE DESIGNS
• MAJOR GLOBAL BRAND EXPERIENCE
• FEATURED NATIONALLY AS SPEAKER
• UX DESIGN AND OPTIMIZATION FOR LARGE,
GLOBAL BRANDS SUCH AT THE GAP, SONY,
BOMBAY COMPANY, FARMERS INSURANCE,
NORTHWESTERN MUTUAL, GE HEALTHCARE,
MOTOROLA, HONEYWELL AND MANY OTHERS.
CLAY KONNOR
DIRECTOR OF DIGITAL DESIGN

4. ABOUT YOU
• B2B?
• B2C?
• E-COMMERCE?
• ANY UX TESTING EXPERIENCE?
A SHOW OF HANDS PLEASE

5. USABILITY

6. “Usability • making sure that something works well: that a
person of average (or even below average) ability and
experience can use the thing – whether it’s a website,
a fighter jet, or a revolving door – for its intended
purpose without getting hopelessly frustrated”
!
- Steve Krug, Author
USABILITY

7. USABILITY

8. 22
35
26
17 18
29
22
24
1313
2120
2007 2010 2012
Ease of
conversion
Ease of use
(usability)
Security and
confidence
Confirmation
of action
TOP USER COMPLAINTS FOR WEBSITES
USER EXPERIENCE

9. SOMEBODY DOES SOMETHING YOU WANT THEM TO DO
Page Visits x Conversion Percentage = Results
10,000 x .03 = 300
WHAT IS A CONVERSION?

10. AVERAGE CONVERSION?
High Tech
Travel &
Hospitality
Retail
B2B
0 2 3 5 6
1.5
2.2
2.1
1.5
5.2
4.2
3.6
2.9
Best of Best The Masses

11. SEARCH PPC PR ADVERTISING
CONTENT MARKETING SOCIAL
ELEVATING CONVERSIONS

12. PUBLIC
RELATIONS
PAID SEARCH
MARKETING
AUTOMATION
BRAND
AWARENESS
GETTING MORE TRAFFIC
CONTENT
MARKETING
SEARCH
SOCIAL
OFFLINE
MARKETING

13. • Expensive
• When budget ends, efficacy can end
• Over-optimization leads to less qualified users
GETTING MORE TRAFFIC
THE DOWN SIDE

14. SEARCH PPC PR ADVERTISING
CONTENT MARKETING SOCIAL
ELEVATING CONVERSIONS

15. • The component most under your control
• Delivers the biggest bang for your buck
• Exponential effect on bottom line
• You get the benefit… forever
• Competitive advantage
• Customer loyalty goes to the best user experience
USABILITY OPTIMIZATION
WHY OPTIMIZE USABILITY?

16. BOTTOM LINE EFFECTS
Design Design 1 Design 2
Budget 1,000,000 1,000,000
Traffic 1,000,000 1,000,000
CPC $1 $1
Conversion Rate 3% 5%
Sales 30,000 50,000
Average Revenue Per
Sale
$100 $100
Total Web Sales
ELEVATING CONVERSIONS
$3,000,000 $5,000,000

17. • Companies that have engaged in conversion
enhancement initiatives have seen average increases of
25%-600% or more conversion
!
• The median conversion increase is 40-60%
!
- Jakob Nielsen’s report “ROI on Usability”
BOTTOM LINE EFFECTS
ELEVATING CONVERSIONS

18. ELEVATING CONVERSIONS
WHO SHOULD TEST
EVERYONE

19. AFTER LAUNCH
• UX IS DESIGN
• CHOOSE THE RIGHT PLATFORM
• UTILIZE UX PROFESSIONALS
• DESIGN FOR THE LESS INITIATED
• HIGH FIDELITY WIREFRAMES
• TEST CONCEPTS
• TEST CREATIVE
DEVELOPMENTDESIGN
ELEVATING CONVERSIONS
• PROTOTYPES
• ALPHA RELEASE
• TEST BEFORE LAUNCH
• YOU JUST BUILT THE ULTIMATE
TESTING PLATFORM
• HONE YOUR ANALYTICS
• NEVER STOP TESTING
WHEN TO TEST

20. • GOOGLE CONVERSION GOALS
• FUNNEL VISUALIZATION
• GOAL FLOW
• NON-GOOGLE ANALYTICS METHODS
ANALYTICS
ELEVATING CONVERSIONS
WHEN TO TEST

21. ELEVATING CONVERSIONS
WHICH PAGE DESIGN WORKED BETTER?
A B

22. • FORM A HYPOTHESIS BASED ON ANALYTICS, USER
OBSERVATION OR USER TESTING
• CREATE A TEST THAT FOCUSES ON THE DESIRED CONVERSION
• SELECT A TEST GROUP
• AS WIDE AS POSSIBLE
• CONSIDER GEOGRAPHIC TESTING
• CONDUCT TEST
• ANALYZE RESULTS
TESTING
TYPES OF TESTING

23. UNMODERATED REMOTE TESTING
• TESTING A SINGLE OPTION
AGAINST ANOTHER SINGLE
OPTION
• COMPARING TWO (OR MORE)
DISSIMILAR APPROACHES
MULTIVARIATE TESTINGA-B-N TESTING
TESTING
• TESTING A HOW COMBINATIONS
OF ITEMS WORK AGAINST EACH
OTHER
• COMPARING TWO SIMILAR
APPROACHES
• TYPICALLY VIDEO-BASED
• TASK-BASED TESTING
• USER GUIDED BY SYSTEM
TYPES OF TESTING
MODERATED TESTING
• TASK-BASED TESTING
• MOST POWERFUL
• MOST EXPENSIVE
• USER GUIDED BY SYSTEM AND
MODERATOR
• REAL-TIME ANALYSIS

24. TESTING
A-B-N TESTING
• OPTIMIZELY
• JAVASCRIPT
• MANY OTHERS
TOOLS
• VERY SIMPLE TO RUN
• INEXPENSIVE
• MULTIPLE TESTS ARE SIMPLE
BENEFITS
DISADVANTAGES
• WORKS BEST WITH LANDING
PAGES OR SIMPLE CONVERSION
PATHS
• DIFFICULT TO IMPLEMENT FOR A
SITE-WIDE TEST

25. TESTING
A-B-N TESTING EXAMPLE

26. TESTING
A-B-N TESTING EXAMPLE
• 6% increase in banner CTR
• 21.3% increase in visitors that reached the
shopping cart

27. TESTING
A-B-N TESTING EXAMPLE

28. TESTING
RESULT
422% increase in CTR

29. TESTING
A-B-N TESTING EXAMPLE

30. TESTING
RESULT
9% increase in RFQ

31. TESTING
MULTIVARIATE TESTING
• OPTIMIZELY
• VISUAL WEBSITE OPTIMIZER
• GOOGLE CONTENT EXPERIMENTS
• MANY OTHERS
TOOLS
• SIMPLE TO SET UP
• INEXPENSIVE
• FOLLOW-UP TESTS ARE SIMPLE
BENEFITS
DISADVANTAGES
• WORKS BEST WITH LANDING
PAGES OR SIMPLE CONVERSION
PATHS
• DIFFICULT TO IMPLEMENT FOR A
SITE-WIDE TEST

32. TESTING
MULTIVARIATE TESTING
IMAGE 1
HEADLINE ONE
IMAGE 1
HEADLINE TWO
IMAGE 2
HEADLINE ONE
IMAGE 2
HEADLINE TWO

33. familiar top nav
Telephone contact
& Request for demo
brief headline
simple
instructions
stock photo of
happy person
at computer
TESTING
MULTIVARIATE TESTING
Three column
layout

34. Removed all
nav
Replace entire
header of site
with just logo
Replace stock
photo with
actually
relevant image
!
Remove
alternate
contact info
More detailed
copy about
offer
Simplify layout
to two columns
and shade form
column
Big submit
button - only
button on page

35. Less copy
worked better
Visual parity to
online ad

36. TESTING
RESULT
300% increase in form submissions

37. TESTING
UNMODERATED REMOTE USER TESTING
!
• TRY MY UI
• OPEN HALLWAY
• USERLYTICS
• USERTESTING.COM
• USER ZOOM
• WHAT USERS DO
TOOLS
• ROBUST RESULTS
• GEOGRAPHIC VARIABLES
• RECRUITING OPTIONS
• WRITE YOUR OWN TESTS
• VIDEO TO REVIEW
BENEFITS
DISADVANTAGES
• LACK OF MODERATOR
• SUBJECT TO USABILITY OF ACTUAL
TEST DESIGN

38. TESTING
UNMODERATED REMOTE USER TESTING

39. TESTING
RESULT
38% increase in conversion

40. TESTING
MODERATED USER TESTING
!
• MORAE
TOOLS
• MOST ROBUST RESULTS
• REALTIME OBSERVATION
• RECRUITING OPTIONS
• WRITE YOUR OWN TESTS
• VIDEO TO REVIEW
• CUSTOM ANALYTICS
• CUSTOM PRESENTATIONS
BENEFITS
DISADVANTAGES
• HIGHEST COST
• REQUIRES GREATER EXPERTISE TO
DESIGN TESTS
• REQUIRES EXPERTISE TO
CONDUCT TESTS

41. TESTING
MODERATED USER TESTING
TESTING
STATION
REMOTE
OBSERVATION
RESULTS,
ANALYTICS &
PRESENTATION

42. TESTING
MODERATED USER TESTING

43. TESTING
MODERATED USER TESTING
1. Find 10x20 trade show display
!
Your company is attending a trade show. The show is high profile and demands an
exhibit that will stand out. You have been instructed to find a trade show display that
measures 10 x 20 feet. Your budget is $30,000 - $40,000. The display must fit
between other displays which is known as "inline".
Find a display or displays and request a quote for a similar display customized for Top
Floor.
2. Find a road case
!
In addition to the display Top Floor needs a case to transport the display to various
shows.
Find an appropriate case and request a quote.

44. TESTING
MODERATED USER TESTING

45. TESTING
MODERATED USER TESTING

46. TESTING
MODERATED USER TESTING
• Lack of AJAX loading when filters are engaged in Solutions make the page jump
to the top creating user frustration and confusion
• No indication that page is reloading when filter is engaged
• "Events" is in the wrong section - nobody found it where it is. Everyone looked
in “what we do”
• "Our Portfolio" should be renamed
• "Read More" button on product array lists should say “Details”
• Listing pages lack a direct RFQ button
• Lack of images in main area makes users think that the page hasn't loaded and
causes confusion
• Custom is two levels down. If indeed that is a primary offering, then Custom
should be a main navigation item or within solutions

47. TESTING
RESULT
22% increase in requests

48. TESTING
MODERATED USER TESTING

49. TESTING
RESULT
?

50. TESTING
COMBINED TESTING

51. TESTING
COMBINED TESTING

52. ELEVATING CONVERSIONS
WHICH PAGE DESIGN WORKED BETTER?
A B

53. TESTING
RESULT
310% increase in conversions

54. ELEVATING CONVERSIONS
WHO SHOULD TEST
EVERYONE

55. THANK YOU
@clayewi
@topfloortech

56. THANK YOU