Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
WHAT DO USERS REALLY THINK?
Surveying Users About
Your Help Content
Patty Gale
WHAT WILL YOU LEARN TODAY?
Why should you conduct a survey?
What can you expect to learn? not learn?
How do you design the...
WHY SHOULD YOU CONDUCT A SURVEY?
Where do they go for learning content?
Do they use your learning content?
What do they th...
WHAT CAN YOU EXPECT TO LEARN? NOT LEARN?
Learn
• What people think they do
• How they feel about it
• Feedback about their...
HOW DO YOU DESIGN THE SURVEY? TOOLS
WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 5
http://www.capterra.com/survey-softw...
HOW DO YOU DESIGN THE SURVEY?
Goals Questions Plan Peripherals Feedback Pilot
WHAT DO USERS REALLY THINK? #tkrytr #usersur...
IDENTIFY GOALS
What’s the point you’re trying to prove,
or the question you’re trying
to answer?
Why do you want
to survey...
DESIGN QUESTIONS: CHOOSE A DEMOGRAPHIC
Who will you send the survey to? How will you distribute it?
WHAT DO USERS REALLY T...
DESIGN QUESTIONS: BIAS
WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 9
DESIGN QUESTIONS: AVOID BIAS
We think our online help is really awesome.
How awesome do you think our online help is?
How ...
DESIGN QUESTIONS: SCREENING
Which product do you use the most?
Do you use the software at least weekly?
Which software ver...
DESIGN QUESTIONS: A GOAL FOR EACH
Every question must have a purpose.
If you are asking it just because you are curious ab...
DESIGN QUESTIONS: DEMOGRAPHICS
Include a few demographic questions that will allow you to slice and dice
the resulting dat...
HOW DO YOU DESIGN THE SURVEY?
Goals
Identify
goals for the
survey
Questions
Design questions
to meet goals
Plan
Make a dat...
MAKE A DATA ANALYSIS PLAN
• Before you send out the survey
• How will you analyze the
results from each question?
• How wi...
DATA ANALYSIS PLAN: EXAMPLE 1
Goal:
This question is intended to help us set priorities on
the types of content to develop...
DATA ANALYSIS PLAN: EXAMPLE 2
What can we do to improve your experience with XYZ learning content?
(open text response)
An...
HOW DO YOU DESIGN THE SURVEY?
Goals
Identify
goals for the
survey
Questions
Design questions
to meet goals
Plan
Make a
dat...
CREATE SURVEY PERIPHERALS
End of survey
WHAT DO USERS REALLY THINK? #tkrytr #usersurvey
Invitation Introduction
19
INTRODUCTION
• Message before the survey
questions
• Short and sweet
• What is it? (a survey)
• What is the reason for the...
INVITATION
• Short and sweet
• Repeats a lot of the
information in the survey intro
• Provides the survey link
• Signed by...
THANK YOU
• Message after survey questions
• Short and sweet
• Give them the chance to provide
final feedback
• Provide an...
HOW DO YOU DESIGN THE SURVEY?
Goals
Identify goals
for the survey
Questions
Design questions
to meet goals
Plan
Make a dat...
GET FEEDBACK
WHAT DO USERS REALLY THINK? #tkrytr #usersurvey
Share
• Show the completed survey (including peripherals) to ...
HOW DO YOU DESIGN THE SURVEY?
Goals
Identify goals
for the survey
Questions
Design questions
to meet goals
Plan
Make a dat...
PERFORM A PILOT TEST
WHAT DO USERS REALLY THINK? #tkrytr #usersurvey
Goal: Identify and correct issues with the survey bef...
WHAT GOTCHAS SHOULD YOU AVOID?
• Qualified respondents? >> screening questions
• Wording of questions
• Number of question...
WHERE DO YOU FIND USERS TO PARTICIPATE?
SurveyMonkey Audience
 “Access millions of highly qualified, on-demand survey tak...
HOW DO YOU CONDUCT THE SURVEY?
WHAT DO USERS REALLY THINK? #tkrytr #usersurvey
Plan the
timing
and
duration
Send
invitatio...
WHAT DO YOU DO WITH ALL THOSE RESULTS?
Quantitative results
• multiple choice, Y/N, rating scale
• use graphs & charts to ...
QUANTITATIVE: CHARTS AND GRAPHS
WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 31
QUALITATIVE: AFFINITY DIAGRAMMING
WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 32
QUALITATIVE: IMPORTANCE-DIFFICULTY MATRIX
WHAT DO USERS REALLY THINK? #tkrytr #usersurvey
Important
Difficult (time, cost,...
HOW DO YOU CREATE AN ACTION PLAN BASED
ON THE SURVEY FEEDBACK?
• Look for themes.
• Quantitative results: What did you lea...
HOW CAN YOU SHARE THE RESULTS?
…with stakeholders in a way that they can digest it?
• Boil it down. What is the main point...
HOW CAN YOU SHARE THE RESULTS?
WHAT DO USERS REALLY THINK? #tkrytr #usersurvey
Boil it down
to the main points
Provide sup...
NOW YOU KNOW WHAT USERS THINK
and you have a better understanding
of how to create learning content
that supports their ne...
RESOURCES
Basics of creating and running a useful online survey:
http://uxmastery.com/better-user-research-through-surveys...
ATTRIBUTIONS FOR IMAGES
From the Noun Project (CC-by-3.0)
• Show-and-Tell by Aha-Soft
• Moods: Excellent, Good, Regular, B...
WHAT DO USERS REALLY THINK? #tkrytr #usersurvey
Patty Gale
patricia.gale@autodesk.com
LinkedIn: patty-gale-9774461
Twitter...
SURVEY QUESTION:
SOURCES OF INFORMATION
When you need information about how to use XYZ,
how do you find the information yo...
QUESTION: WHEN DO YOU NEED LEARNING
CONTENT?
When do you use the learning content produced by XYZ Inc?
(Check all that app...
In the XYZ learning content, what types of information
would you like to see more of? (Choose up to 3.)
PRIORITIES FOR FUT...
Upcoming SlideShare
Loading in …5
×

What Do Users Really Think? Surveying Users About Your Help Content

744 views

Published on

You explore the product. You interview SMEs. You write until your fingers cramp. You polish. You publish. And…then what?
Do users use your beautiful documentation? Do they like it? Do they find it useful? How do you know? Ask them! Learn how to conduct a user survey to understand customer satisfaction with your learning content. Who should attend: Technical communicators of all stripes who want to understand what users think of their content, with the goal of improving the content, its findability, and/or usability.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

What Do Users Really Think? Surveying Users About Your Help Content

  1. 1. WHAT DO USERS REALLY THINK? Surveying Users About Your Help Content Patty Gale
  2. 2. WHAT WILL YOU LEARN TODAY? Why should you conduct a survey? What can you expect to learn? not learn? How do you design the survey? What gotchas should you avoid? Where do you find users to participate? How do you conduct the survey? What do you do with all those results? How do you create an action plan based on the survey feedback? How can you share the results with stakeholders in a way that they can digest it? WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 2
  3. 3. WHY SHOULD YOU CONDUCT A SURVEY? Where do they go for learning content? Do they use your learning content? What do they think about your learning content? Are they satisfied with your learning content? • CSAT = Customer Satisfaction Learn how you can better meet the needs of your users. WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 3
  4. 4. WHAT CAN YOU EXPECT TO LEARN? NOT LEARN? Learn • What people think they do • How they feel about it • Feedback about their most recent experiences Not learn • What people actually do • How they use it • Feedback about past experiences So why do it? Because you can still learn a lot from this information. WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 4
  5. 5. HOW DO YOU DESIGN THE SURVEY? TOOLS WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 5 http://www.capterra.com/survey-software/ • Easy to use • Free trial/free edition • Supports skip logic • Supports many types of questions • Survey management tools • Analysis tools/features
  6. 6. HOW DO YOU DESIGN THE SURVEY? Goals Questions Plan Peripherals Feedback Pilot WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 6
  7. 7. IDENTIFY GOALS What’s the point you’re trying to prove, or the question you’re trying to answer? Why do you want to survey users? What do you hope to learn? EXAMPLE: Measure usage and satisfaction with learning content Identify areas and opportunities for improvement Understand how we can better support the learning needs of our users WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 7
  8. 8. DESIGN QUESTIONS: CHOOSE A DEMOGRAPHIC Who will you send the survey to? How will you distribute it? WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 8
  9. 9. DESIGN QUESTIONS: BIAS WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 9
  10. 10. DESIGN QUESTIONS: AVOID BIAS We think our online help is really awesome. How awesome do you think our online help is? How would you rate your satisfaction with the XYZ online help?      Very Dissatisfied Neutral Satisfied Very Dissatisfied Satisfied https://www.surveymonkey.com/mp/survey-guidelines/ WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 10
  11. 11. DESIGN QUESTIONS: SCREENING Which product do you use the most? Do you use the software at least weekly? Which software version do you currently use? rest of survey A, B, C Yes 2015, 2014 No None of these A prior version A general question? exit survey WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 11
  12. 12. DESIGN QUESTIONS: A GOAL FOR EACH Every question must have a purpose. If you are asking it just because you are curious about the answer, eliminate it. WHAT DO USERS REALLY THINK? #tkrytr #usersurvey When you need information about how to use XYZ, how do you find the information you need? Goals: Understand where users instinctively go when they need help. Consider whether their favorite resources have characteristics or information types that we can emulate to improve the quality of our learning content. 12
  13. 13. DESIGN QUESTIONS: DEMOGRAPHICS Include a few demographic questions that will allow you to slice and dice the resulting data to compare responses from different groups of users. WHAT DO USERS REALLY THINK? #tkrytr #usersurvey Approximately how long have you used or worked with XYZ software? • Less than 6 months • 6 to 12 months • 1 to 3 years • 4 to 5 years • 6 to 10 years • More than 10 years When you need information about how to use XYZ, how do you find the information you need? • Search online • Visit expert blog • Use product help • Contact customer support • Ask co-worker • Other (please specify) 13
  14. 14. HOW DO YOU DESIGN THE SURVEY? Goals Identify goals for the survey Questions Design questions to meet goals Plan Make a data analysis plan Peripherals Feedback Pilot WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 14
  15. 15. MAKE A DATA ANALYSIS PLAN • Before you send out the survey • How will you analyze the results from each question? • How will you use the resulting information? WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 15
  16. 16. DATA ANALYSIS PLAN: EXAMPLE 1 Goal: This question is intended to help us set priorities on the types of content to develop going forward to meet our users' needs. Choices include the following: • Troubleshooting • In-depth information about how XYZ works • Step-by-step instructions • Video tutorials • Detailed workflows between multiple products • Examples • Other (please specify) In the XYZ learning content, what types of information would you like to see more of? (choose 3) Analysis: Determine the top 3 requested types of information. Do these reflect current initiatives? • If so, it's confirmation that we are moving in the right direction. Be sure to perform follow-up user testing and/or surveys to make sure we hit the mark. • If not, consider whether we should place an emphasis on these types of information for our sprint work. Examine responses for "Other." Are they requesting any types of information that we haven't considered? If so, investigate further. WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 16
  17. 17. DATA ANALYSIS PLAN: EXAMPLE 2 What can we do to improve your experience with XYZ learning content? (open text response) Analysis: Look at individual responses. What specific suggestions can we implement? Make user stories to add to a future sprint to make these improvements. Look for overall themes: • What do multiple respondents ask for? • What areas of learning content seem to cause the most angst? • What larger improvements can we make that might help to reduce pain/frustration for a large group of users? Identify these themes and make an action plan to implement a solution. WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 17
  18. 18. HOW DO YOU DESIGN THE SURVEY? Goals Identify goals for the survey Questions Design questions to meet goals Plan Make a data analysis plan Peripherals Feedback Pilot WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 18
  19. 19. CREATE SURVEY PERIPHERALS End of survey WHAT DO USERS REALLY THINK? #tkrytr #usersurvey Invitation Introduction 19
  20. 20. INTRODUCTION • Message before the survey questions • Short and sweet • What is it? (a survey) • What is the reason for the survey? • How long will it take? • Signed by: the name and title of the person sponsoring the survey. (a specific individual, not just a group or division) You are invited to participate in the XYZ Learning Content Survey. By participating, you are helping us to: • Understand your experiences with XYZ learning content and how you use it. • Get your feedback on how we can improve existing learning content. • Set priorities for future learning content development. The survey should take about 10-15 minutes of your time. If you cannot complete the survey in one session, you can stop at any time and your answers will be saved. To continue where you left off, click the link in the email invitation. Thank you for your feedback. Sincerely, Somebody Important Director, User Experience XYZ Corp. WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 20
  21. 21. INVITATION • Short and sweet • Repeats a lot of the information in the survey intro • Provides the survey link • Signed by: the name of the person who is sending out the invitation, and the group they are working with for the survey Dear <name>, As an XYZ user, what is helpful to you when using learning resources for XYZ? What learning content is most relevant to you? The XYZ Learning Content Team would like to hear from you. By participating, you are helping us to: • Understand your experiences with XYZ learning content and how you use it. • Get your feedback on how we can improve existing learning content. • Set priorities for future learning content development. The survey should take about 10-15 minutes of your time. If you cannot complete the survey in one session, you can stop at any time and your answers will be saved. Please click the link below to provide your feedback. Thank you. <Survey link here>WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 21
  22. 22. THANK YOU • Message after survey questions • Short and sweet • Give them the chance to provide final feedback • Provide an email in case they’d rather reach out in person or they think of something after submitting the survey If you have additional comments about the XYZ learning content or questions about this survey, enter them below, or send email to learning-content@xyz.com. Do you have additional feedback about XYZ learning content? [ Submit ] Thank you for participating in the survey. Your feedback is important to us. WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 22
  23. 23. HOW DO YOU DESIGN THE SURVEY? Goals Identify goals for the survey Questions Design questions to meet goals Plan Make a data analysis plan Peripherals Write the invitation, intro text, thank you text Feedback Show it to others for feedback Pilot WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 23
  24. 24. GET FEEDBACK WHAT DO USERS REALLY THINK? #tkrytr #usersurvey Share • Show the completed survey (including peripherals) to colleagues • Especially any with experience in user research and user experience Review • Ask them to read through it carefully and provide feedback • Any feedback is welcome: wording, questions, length, focus, etc Revise • Incorporate their feedback • Rinse and repeat 24
  25. 25. HOW DO YOU DESIGN THE SURVEY? Goals Identify goals for the survey Questions Design questions to meet goals Plan Make a data analysis plan Peripherals Write the invitation, intro text, thank you text Feedback Show it to others for feedback Pilot Send it to a small group of internal users to identify and fix issues WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 25
  26. 26. PERFORM A PILOT TEST WHAT DO USERS REALLY THINK? #tkrytr #usersurvey Goal: Identify and correct issues with the survey before it goes to customers. Invite 3-5 internal colleagues to take the survey. • Who? Folks who can represent the user to some extent. QA? UX? PM? • Send them the invitation with the link, and have them complete the survey. Ask them for direct feedback: • Are the invitation and introduction clear? • Are the questions clear? Did any cause confusion? • Is the survey too long? • Did they encounter any issues? Or think of questions as they took the survey? Revise the survey, and have a few more folks take it. Rinse and repeat. 26
  27. 27. WHAT GOTCHAS SHOULD YOU AVOID? • Qualified respondents? >> screening questions • Wording of questions • Number of questions (survey fatigue) • Order of questions • Timing of the survey (summer vacation, winter holidays) • Duration of survey (2-3 weeks) • If no NDAs, do not ask about or show content that is not yet released • If your organization conducts many surveys, avoid survey fatigue WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 27
  28. 28. WHERE DO YOU FIND USERS TO PARTICIPATE? SurveyMonkey Audience  “Access millions of highly qualified, on-demand survey takers. Choose from 30+ criteria to reach the people you need. Get results in 2 days and watch data appear in real time”  Better suited to consumer goods and services Talk to Product Management, Customer Support. Who is actively using the software? Marketing >> customer list? Get a list of beta testers. (may be under NDA) Goal: 100+ responses. How many invitations will it take to achieve that goal? WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 28
  29. 29. HOW DO YOU CONDUCT THE SURVEY? WHAT DO USERS REALLY THINK? #tkrytr #usersurvey Plan the timing and duration Send invitations Wait and watch Send reminders after one week Close the survey 29
  30. 30. WHAT DO YOU DO WITH ALL THOSE RESULTS? Quantitative results • multiple choice, Y/N, rating scale • use graphs & charts to easily visualize the results WHAT DO USERS REALLY THINK? #tkrytr #usersurvey Qualitative results • open-text response • analyze text 30
  31. 31. QUANTITATIVE: CHARTS AND GRAPHS WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 31
  32. 32. QUALITATIVE: AFFINITY DIAGRAMMING WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 32
  33. 33. QUALITATIVE: IMPORTANCE-DIFFICULTY MATRIX WHAT DO USERS REALLY THINK? #tkrytr #usersurvey Important Difficult (time, cost, culture) Not Important Easy Luxury Strategic Quick winsTargets Resolve differing opinions Prioritize ideas quickly Leads easily to an action plan 33
  34. 34. HOW DO YOU CREATE AN ACTION PLAN BASED ON THE SURVEY FEEDBACK? • Look for themes. • Quantitative results: What did you learn? • Qualitative results: What patterns emerge? • What do you have control over? • Make a plan to implement improvements/changes. • Get management buy-in. • Implement the plan. • What don’t you have control over? • Share your survey results with the responsible group. • Suggest potential changes they could make to improve the user’s learning experience. WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 34
  35. 35. HOW CAN YOU SHARE THE RESULTS? …with stakeholders in a way that they can digest it? • Boil it down. What is the main point that you want to share? • Provide supporting evidence: one or two charts, supporting quotes from open-text responses. • Summarize your action plan: what will you do in light of these results? • AVOID death-by-PowerPoint WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 35
  36. 36. HOW CAN YOU SHARE THE RESULTS? WHAT DO USERS REALLY THINK? #tkrytr #usersurvey Boil it down to the main points Provide supporting evidence Summarize your action plan 36
  37. 37. NOW YOU KNOW WHAT USERS THINK and you have a better understanding of how to create learning content that supports their needs WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 37
  38. 38. RESOURCES Basics of creating and running a useful online survey: http://uxmastery.com/better-user-research-through-surveys/ Designing questions: https://www.surveymonkey.com/mp/survey-guidelines/ Survey tools: http://www.capterra.com/survey-software/ Sharing research results: • https://medium.com/%40FBResearch/beyond-bullet-points-four-creative- ways-to-share-research-c10fa047f025#.sayukl640 • http://www.userfocus.co.uk/articles/dashboard.html • http://www.userexperience.co.nz/2015/12/visualising-design-research/ • http://annkemery.com/qual-dataviz/ WHAT DO USERS REALLY THINK? #tkrytr #usersurvey 38
  39. 39. ATTRIBUTIONS FOR IMAGES From the Noun Project (CC-by-3.0) • Show-and-Tell by Aha-Soft • Moods: Excellent, Good, Regular, Bad, Terrible by Rflor • Mom and Boy by Fx Pedron • Toy Robot by Oliviu Stoian • Map by Alen Krummenacher • Analysis, Error by Chameleon Design • Flow chart by Jhun Capaya • Invitation by Creative Stall • Add conversation, Successful conversation by Oliviu Stoian • Users by Wilson Joseph • Group by Felix Westphal • Demographics by Fiona OM WHAT DO USERS REALLY THINK? #tkrytr #usersurvey • Chart by Pham Thi Dieu Linh • Documents by Eugen Belyakoff • Boil by Antistatique • Bar Chart by Pham Thi Dieu Linh • Pages by Eugen Belyakoff • Reports by Joao Marcelo Ribiero • Calendar by Ed Harrison • Presentation by Garrett Knoll • Question by Fred Selbert Public domain: • Meeting (qualitative results) • Confused Other: • Affinity diagramming photos by Patty Gale • Quantitative charts by SurveyMonkey 39
  40. 40. WHAT DO USERS REALLY THINK? #tkrytr #usersurvey Patty Gale patricia.gale@autodesk.com LinkedIn: patty-gale-9774461 Twitter @tkrytr 40
  41. 41. SURVEY QUESTION: SOURCES OF INFORMATION When you need information about how to use XYZ, how do you find the information you need? (Choose up to 3 methods.)  I search online (Google, Bing, etc.).  I visit the blog/social media of my favorite XYZ expert.  I visit a XYZ user forum.  I access product help (for example, by pressing F1 or a Help button in XYZ).  I contact XYZ Customer Support.  I ask my co-worker for help or use an internal resource.  Other (please specify) 41
  42. 42. QUESTION: WHEN DO YOU NEED LEARNING CONTENT? When do you use the learning content produced by XYZ Inc? (Check all that apply.)  When I can't remember how to complete a task  When I don't know what something on the screen means  When my co-workers are not available to answer my questions  When I want to find a quicker way to do my work  When something goes wrong and I don't know how to fix it  When I need to research features so I can teach others  When I am teaching others  Other (please specify) 42
  43. 43. In the XYZ learning content, what types of information would you like to see more of? (Choose up to 3.) PRIORITIES FOR FUTURE LEARNING CONTENT  Troubleshooting  In-depth information about how XYZ works  Step-by-step instructions  Video tutorials  Detailed workflows between multiple products  Examples  Other (please specify) 43

×