Delivering Results: How Do You Report User Research Findings?

10,304 views

Published on

The long, textual written report is dead, isn’t it? So how do you deliver your findings to your clients? Is it PowerPoint? An e-mail? A spreadsheet? Post-it notes? And what do you include? Positive findings? Screenshots with callouts? Just issues? Or recommendations as well? Are they prioritized?

If you ask our panelists, some of us have developed templates that we use and modify for each research activity, and others change the deliverable based on the activity and client.

Jen McGinn, Principal Usability Engineer, Oracle

Eva Kaniasty, Founding Principal, RedPill UX

Dharmesh Mistry, Usability Specialist, Acquia

Kyle Soucy, Founding Principal, Usable Interface

Carolyn Snyder, Founding Principal, Snyder Consulting

Published in: Technology, Business
0 Comments
25 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
10,304
On SlideShare
0
From Embeds
0
Number of Embeds
71
Actions
Shares
0
Downloads
229
Comments
0
Likes
25
Embeds 0
No embeds

No notes for slide
  • Accessible, Transparent
  • No recommendations
  • Credibility and Turn around time
  • How cool is your audience?
  • Presenting at conferencesStory tracking – Drupal (D.O.), Acquia (JIRA)
  • Goal: Make the report as easy to read as possible!
  • Don’t demand changes. Use language like, “Consider changing…” or “If possible…”.
  • Some clients don’t need/want a formal report; they just need these notes.
  • Delivering Results: How Do You Report User Research Findings?

    1. 1. New Hampshire Usability Professional’s Association www.nhupa.orgJanuary 2012 MeetingDelivering Results: How Do You Report UserResearch Findings?Jen McGinnEva KaniastyDharmesh MistryKyle SoucyCarolyn Snyder
    2. 2. New Hampshire Usability Professional’s Association www.nhupa.orgIntroductionAbout NH UPAWe want to make New Hampshire a good place for user experienceprofessionals to work and liveJoin Us: www.nhupa.org or @nhupa or NH UPA on FacebookHelp Us: Volunteer or give us suggestionsAnnouncements• NH UPA Board of Directors: Debra Arneson, Megan Fields, Rick La Vache, Bob Thomas• Thanks to Mad*Pow• Jobs
    3. 3. New Hampshire Usability Professional’s Association www.nhupa.orgUpcoming NH UPA MeetingsTentative Date: Wednesday, February 15Tom Clancy, “Using Development Frameworks to Rapidly PrototypeApplications”• Have an idea for an upcoming meeting?• Want to practice an upcoming presentation or panel you’re giving?• Want to give a 10-minute talk?Contact Us: robertl.thomas@libertymutual.com or debra.arneson@tds.net
    4. 4. New Hampshire Usability Professional’s AssociationDelivering Results: How Do You Report UserResearch Findings?The long, textual written report is dead, isn’t it? So how do you deliveryour findings to your clients? Is it PowerPoint? An e-mail? A spreadsheet?Post-it notes? And what do you include? Positive findings? Screenshots withcallouts? Just issues? Or recommendations as well? Are they prioritized?
    5. 5. New Hampshire Usability Professional’s AssociationPanelistsIf you ask our panelists, some of us have developed templates that we useand modify for each research activity, and others change the deliverablebased on the activity and client. Each panelist will spend 5 minutesshowing you their typical deliverables, and then we’ll open the floor foraudience Q&A.•Jen McGinn, Principal Usability Engineer, Oracle•Eva Kaniasty, Founding Principal, RedPill UX•Dharmesh Mistry, Usability Specialist, Acquia•Kyle Soucy, Founding Principal, Usable Interface•Carolyn Snyder, Founding Principal, Snyder Consulting
    6. 6. New Hampshire Usability Professional’s AssociationJen McGinnPrincipal Usability Engineer, Oracle
    7. 7. New Hampshire Usability Professional’s AssociationOverview• I present my research results via slides in 60-minute meeting (generally remote, via web conference)• I’m going to spend 3-5 minutes walking you through the structure of one of my PowerPoint presentations• Then I’ll summarize the take-aways 
    8. 8. New Hampshire Usability Professional’s Association [Product ][Method] My name, title, and date2/6/2012
    9. 9. New Hampshire Usability Professional’s AssociationExecutive Summary• In [When?], the [what product?] was tested by [number and type of participants] in [method type] to evaluate the ease of use of several features including [features or use cases].• High level findings included [usually a total of 3 to 4 bullets]: • [ 1 - 2 biggest positive findings] • [ 1 - 2 biggest positive findings] • [ 2 or 3 biggest usability issues] • [ 2 or 3 biggest usability issues]• This presentation covers all of the findings and subsequent recommendations. 2/6/2012
    10. 10. New Hampshire Usability Professional’s AssociationAgenda• Goals• Tasks• Participants• Findings• Recommendations• Next Steps 2/6/2012
    11. 11. New Hampshire Usability Professional’s AssociationGoalsEvaluate the usability of the following features of the U-Haul.com website: – Are users confused about how to price a rental? A storage unit? – How do users react to the insurance options? Do they understand the coverage? – How do users feel about the presentation of items for purchase or for rent? – How effective is the shopping cart content? Are users confused by when they need to pay for items? – Do users value the star ratings? U-Haul brand? – How do users feel about the targeted FAQ and search result pages? – Does our online documentation help prevent calls to the service center? Can they determine how to reach out to the U-Haul vendor nearest them? 2/6/2012
    12. 12. New Hampshire Usability Professional’s AssociationTasks 1. Get the price of a 1-way move across country 2. Find a specific piece of information in the FAQ 3. Determine the size and cost of a storage unit needed to hold specific items 4. Find the phone number of a U-Haul location 5. Book the truck (and insurance), adding rental items and purchased items 6. Determine insurance coverage 7. Find the U-Haul location nearest you 2/6/2012
    13. 13. New Hampshire Usability Professional’s AssociationParticipantsParticipant Gender Age Occupation Web- ID savvy U1 Male 24 Missionary Average U2 Male 52 Small business manager Average U3 Female 62 62 Retired. Formerly Average television news producer, then licensed paralegal. U4 Female 36 Housewife Average U5 Male 31 Sales and marketing Average 2/6/2012
    14. 14. New Hampshire Usability Professional’s Association Findings 2/6/2012
    15. 15. New Hampshire Usability Professional’s AssociationChoosing a Truck 2 participants had this issue and did „x‟ to work around it Another issue One participant suggested this fix Jen McGinn5/25/2011
    16. 16. New Hampshire Usability Professional’s AssociationGoals and Questions Re-visited • [All the same as before] Are users confused about how to price a rental? A storage unit? • How do users react to the insurance options? Do they understand the coverage? • How do users feel about the presentation of items for purchase or for rent? • How effective is the shopping cart content? Are users confused by when they need to pay for items? • Do users value the star ratings? U-Haul brand? • How do users feel about the targeted FAQ and search result pages? • Does our online documentation help prevent calls to the service center? Can they determine how to reach out to the U-Haul vendor nearest them? 2/6/2012
    17. 17. New Hampshire Usability Professional’s AssociationPositive Findings [these always come first]• All participants easily found the links to the FAQs and had no trouble finding the answer to the license question under FAQs• All participants made use of the maps when comparing options.• All participants did scroll down to compare prices, locations and reviews• 4 participants valued the presence of the [higher] star ratings• 2 participants valued U-Haul location more than the off-brand vendors• 2 participants were pleased that the truck rental page "retained her information" -- the addresses and dates• 2 participants appreciated the visuals of the items inside the storage units and the graphic of the person shown in the small unit icon• 2 participants easily added the dolly, blankets and boxes during the truck rental task flow 2/6/2012
    18. 18. New Hampshire Usability Professional’s AssociationRecommendationsPriority Description Recommendation Location Participants dont understand what Re-format coverage and High the purchased insurance actually exclusions into bulleted Damage coverage covers lists; Dont use legal jargon Participants have a very hard time High Provide more user Self Storage location estimating the storage unit size that assistance details page would meet their needs Put the purchased items Up-sell process for items to rent or into another page in the Additional rental items, Medium purchase is confusing flow, and make it clearer Shopping cart that users can opt out. Participants are concerned that the Add a link to display the Select your preferred Medium site is incorrectly calculating the map, so they can check it in pickup location mileage and therefore overcharging place Participants were not sure what Display the distance "from" Select your preferred location the giant thumbtack/pin was the specified location, like Low pickup location, (address or zip code) or how far the Self-storage results Location away the locations were page 2/6/2012
    19. 19. New Hampshire Usability Professional’s AssociationNext Steps• Work with [which stakeholders or teams] to prioritize changes• Work with [stakeholders or teams] to design alternatives• Validate that the new designs address the issues with users 2/6/2012
    20. 20. New Hampshire Usability Professional’s AssociationSummary• Tell them what you’re going to tell them – Executive summary – Agenda – Goals/Questions• Tell them – Tasks & participants (sometimes methodology) – Animated slides for progressive disclosure – Screen shots annotated with findings• Tell them what you told them – Review goals of the research and the questions it was intended to answer – Positive findings (go slowly here) – Prioritized opportunities for improvement
    21. 21. New Hampshire Usability Professional’s AssociationEva KaniastyFounding Principal, RedPill UX
    22. 22. New Hampshire Usability Professional’s AssociationReport FormatsPPT: visually engaging but real-estate constrained(and will force you to be brief). Formatting can betime-consuming.MS Word/Narrative: more room for context; quick, butcan appear dry and boring.3rd Option: No report.
    23. 23. New Hampshire Usability Professional’s AssociationDeciding Factors• Time/Budget• (Mode of) Presentation of Results• Company Culture / Industry• Stakeholder Involvement• Deliverable Shelf Life
    24. 24. New Hampshire Usability Professional’s AssociationReporting Findings (1)
    25. 25. New Hampshire Usability Professional’s AssociationReporting Findings (2)
    26. 26. New Hampshire Usability Professional’s AssociationDharmesh MistryUsability Specialist, AcquiaContent Management System Products built on DrupalOpen Source Software Open Source/ ProprietaryCommunity Start-up
    27. 27. New Hampshire Usability Professional’s Association Stakeholder behaviorSupporting information Main Report Detailed Information Transparencyhttp://drupal.org/node/1289476 http://drupal.org/node/1399056 http://drupal.org/node/1399258
    28. 28. New Hampshire Usability Professional’s Association Recommendations Tracking Stakeholders comments Main Reporthttp://www.drupalusability.org/ http://drupal.org/node/1175694
    29. 29. New Hampshire Usability Professional’s Association Credibility Turn around time
    30. 30. New Hampshire Usability Professional’s Association Not Cool? Cool
    31. 31. New Hampshire Usability Professional’s AssociationStakeholders Thousands of Stakeholders 3-5Turn around time Weeks-Months Hours-day-weekTracking High, Extensive Moderate-Very High Twitter, Conferences, Front Email, Conference meetingsPresenting page on Drupal.orgProvide No, never! Sometimesrecommendations
    32. 32. New Hampshire Usability Professional’s AssociationKyle SoucyFounding Principal, Usable Interface
    33. 33. New Hampshire Usability Professional’s AssociationUsable InterfaceFormal Usability Testing/Research Report
    34. 34. New Hampshire Usability Professional’s AssociationExecutive Summary
    35. 35. New Hampshire Usability Professional’s AssociationFindings: Severity Ratings
    36. 36. New Hampshire Usability Professional’s Association FindingsMajor Usability Problem
    37. 37. New Hampshire Usability Professional’s AssociationFindings: Recommendations
    38. 38. New Hampshire Usability Professional’s AssociationPresenting Findings…Finding: All of the participants were disoriented after clicking the “View in 3D”button on the quote. Recommendations: • The navigation tabs should not disappear in the 3D view. • If the page needs to be reloaded than the orientation should remain the same and not take the user back to the top of the screen. • If possible, the “View in 3D” button should download the ActiveX plug-in.
    39. 39. New Hampshire Usability Professional’s AssociationHighlight Video
    40. 40. New Hampshire Usability Professional’s AssociationObserver Debrief Notes
    41. 41. New Hampshire Usability Professional’s AssociationCarolyn SnyderFounding Principal, Snyder Consulting• There is no one “best” format• Do what works for the client, culture, circumstances• Steal good ideas, drop losers
    42. 42. New Hampshire Usability Professional’s Association Text Report: “I’m not dead yet!” Finding Severity ratingExplanation ofissueSupportingobservations fromnotesRecommendations
    43. 43. New Hampshire Usability Professional’s AssociationPowerPoint with “report” in Notes Field
    44. 44. New Hampshire Usability Professional’s Association PowerPoint, Screen Shots with Callouts Interest in these Interest in these links Ambiguous Some people saw this as a bar graph, but others did not. OneImportant sentence person read only [first 4 words]buried in paragraph Why just show Most people read this text; [scenario]? Some everyone drilled into [noun] misinterpreted it asPeople liked “worst case scenario.”suggestions, butwanted concrete,prioritized advice.Order can implypriority. Amount isn‟t explicit. The userCan‟t explore must do the[action]. People math.knew it wasimportant. People seemed to People understood the understand the purpose stacked bar graphs, Not clear why it showed [2 variations of graph] 44
    45. 45. New Hampshire Usability Professional’s AssociationSometimes the best report is…
    46. 46. New Hampshire Usability Professional’s AssociationQuestions1. Do you change your delivery of usability results depending on your role as a internal/external consultant or as a company employee?2. How important are positive vs. negative findings?3. How have your reports changed over the years? Is there anything you do differently than when you first started writing them?4. How do you categorize the findings in your reports? For example, do you categorize them by the page/screen, by the step in a certain process (e.g. checkout process), or by the task?5. Lean UX is a trending topic. Have you had experience with Lean UX or Agile methods, and had to change the way you conduct research and deliver results?6. What guidelines do you follow when writing recommendations or proposed solutions to problems?7. Do you decide ahead of time how long a report should be and make an effort to keep it that length? If so, what dictates the length?8. If you think a report is too long and needs to be trimmed down, how do you decide what to cut out?9. What part of a report is the hardest for you to write?

    ×