Survey Tool and Internet Research Program

UCD Survey Tool & Internet
Research Program
Tom Callaghan

Copyright ©2006 CA. All rights reserved. All trademarks, trade names, services marks and logos referenced herein belong to their respective companies.
The Survey Tool:
WebSurveyor/Vovici
2

Survey of Tool Users -- Summary
- 32 of 99 users from across the company responded.
- Users find WebSurveyor (now Vovici) useful, support
adequate, and would recommend it to a colleague.
- Training and the intranet site could be improved.
- Users indicated they saved $98,000 and 1282 hours
using WebSurveyor.

Users saw WebSurveyor as a useful tool they
would recommend to others, but they wanted
to see additional training and additions to the
intranet site.
Indicate the degree with which you agree with the following statements regarding WebSurveyor.
(Percentages)
Strongly
Disagree
Disagree Neutral Agree
Strongly
Agree
WebSurveyor is a useful tool. 0.0% 0.0% 0.0% 43.8% 56.3%
Support for WebSurveyor is adequate. 0.0% 3.1% 12.5% 59.4% 25.0%
Training for WebSurveyor is adequate. 3.1% 6.3% 25.0% 46.9% 18.8%
The WebSurveyor intranet site is useful. 0.0% 0.0% 46.9% 31.3% 21.9%
I would recommend WebSurveyor to a colleague at CA. 0.0% 3.1% 3.1% 34.4% 59.4%

There was considerable return on
the $36,000 investment.
- 32 users saved $98,000 (~ $300,000 for 99 users)
- 32 users saved 1282 hours (~ 4000 hours for 99 users)
- Monetary savings ranged from $0 to $25,000
- A number of users said they found it hard to quantify
due to the lack of a comparable alternative, and
entered $0
- Time savings ranged from 0 to 120 hours.
*Statistics do not include past account owners and those sharing
accounts (~ 60 users), and of course, 67 account owners that did not
respond to the survey.

Internet UCD Survey Program
- Goal: Monitor the usability of site
- Strategy: Use online surveys to segment users
and collect feedback on user tasks.
- Timing: Site section importance survey (1) begun
Oct 5; redesign survey (6) in May

Internet Survey Program
9

Goals
- Impetus: Need for a subjective and “direct from the user”
compliment to the information in the site content ratings, web
metrics, search metrics, support calls, general feedback, and other
business owner customer input channels.
- Primary Goal: Provide a process for gathering end user
experiences regarding the usability of the site web presence in a
cost-efficient manner.
- Secondary Goals:
- Create a survey infrastructure for an Internet UCD Program.
- Generate a pool of profiled users to provide timely answers to
questions about site site design and performance.
- Create a community of site users by providing summary
information and insightful conclusions to survey results.

Strategy
Using a series of 5 (or 6) online surveys to segment users by 18 sections
of the site, then collect feedback on specific user tasks they indicate are
important and that they use frequently.
- Survey 1: Importance of site sections, and join web survey panel
- Intercept survey, respondents volunteer with email address
- Survey 2: Gather users tasks
- Email invite, opt out, track previous responses, more demographic info
- Survey 3: Rate task importance and frequency of use
- Email invite, track previous responses
- Survey 4: Rate usability of site in relation to specific task
- Email invite, track previous responses
- Survey 5: Repeat survey 3 when significant functionality is added to site
- First instance being the redesign
- Survey 6: Repeat survey 4 (1-2 months after launch)

Results: Survey 1
Site Section
Average
Importance
Rating
Site Section
Average
Importance
Rating
Support 4.1 News 2.9
Products 3.8 About site 2.8
Solutions 3.8 SMB 2.7
Trials and Downloads 3.7 Worldwide 2.6
Search 3.7 World 2.6
Contact 3.6 Events 2.6
Services 3.6 Partners 2.4
Home and Home Office 3.3 Careers 2.3
Education/Training 3.2 Investor Relations 2.1
Detail survey report: http://surveys.com/wswebtop.dll/WSPubReport?esid=1060&subaccountid=1
Survey example: https://surveys.com/wsb.dll/1/-Web-UCD_1_Example.htm
- Joined Internet UCD Survey Program: 1903 (1100 “Not sure”)
- 5000+ responses to date at 27% completion rate of survey
- Rating of site section importance

Results: Survey 2
- Gather User Tasks
- Users provided top 3 tasks for site sections only if they rated that section
important, or higher, in Survey 1
- Tasks were refined and compiled for each site section. They were shared
with Internet Marketing and section owners for review and additions.
- Example:
- User Profile expanded with additional questions
Detail Survey Report: http://surveys.com/wswebtop.dll/WSPubReport?esid=1098&subaccountid=1
Survey Example: https://surveys.com/wsb.dll/1/-UCD_2-Example.htm
Emails Sent: 770
Usable Completed Surveys: 53

Results: Survey 3
- Rate task importance and frequency of use
- Tasks from Survey 2 ranked by importance and frequency of use to
understand what is important to our users to help us better focus our efforts.
- Tasks were sorted by importance and frequency of use, and a “top-ten” list
of tasks for each of the 18 sections was created to represent that section.
Survey Example: https://surveys.com/wsb.dll/1/-UCD_3-BO-Tasks-Participants-EXAMPLE.htm
Emails Sent: 2015
Usable Completed Surveys: 88

Results: Survey 4
- Rate usability of site in relation to specific task
- Benchmark setting of site task usability
- Over 2500 users were invited to rate the usability of the top site section
tasks, and 258 acceptable responses were received. The percentage of
users that clicked on the survey link versus completing the survey
(49.9%) was nearly twice that of Surveys 1, 2, and 3.
Survey Example: https://surveys.com/wsb.dll/1/-UCD_4-Task-Usability_DUMMY.htm
231 users rated the site
usability for Support tasks.

Strategy: Survey 5
In Survey 5, we rank the tasks by importance and frequency again if we feel
these tasks have changed or need to be expanded.

Appendix

Results: Survey 2
Demographic feedback on our users also provide insight.

Results: Survey 3

Results: Survey 4

Survey Invite Email
site Business Owner,
You are being invited to complete this survey because as a business owner you have insight into the tasks your
web site visitors need to complete. The results of this survey will be used to improve site design and focus
resources.
The purpose of this survey is to ask you to rate the importance and frequency of use of tasks your users might
complete on site. The tasks were compiled from the responses to another survey, completed by site end
users. While not exhaustive, they represent some of the key tasks by which we can judge the usability of the
site.
You will be asked only to rate the sections for which you are responsible. Also, you will have the opportunity to
add up to two addition tasks for that section. There are eighteen (18) site sections, and a number of tasks in
each. It should take about 15 minutes to complete. Your ratings will refine the list presented in our next user
survey
In a follow up survey, users will rate each of these tasks according to how well the site supports them now. We
will then repeat the survey periodically, most notably after the April redesign.
Please click the following link to access the survey: |LINK1|
We ask that you complete this survey by January 12.
Thank you for your time and input,
33

Survey Tool and Internet Research Program

Recommended

Recommended

More Related Content

Similar to Survey Tool and Internet Research Program

Similar to Survey Tool and Internet Research Program (20)

More from Thomas Callaghan

More from Thomas Callaghan (9)

Survey Tool and Internet Research Program