Note-Taker's Perspective During Usability Testing (Kristen Davis & Dana Douglas)
Upcoming SlideShare
Loading in...5
×
 

Note-Taker's Perspective During Usability Testing (Kristen Davis & Dana Douglas)

on

  • 775 views

Given at UXPA-DC's User Focus Conference, Oct. 19, 2012

Given at UXPA-DC's User Focus Conference, Oct. 19, 2012

Statistics

Views

Total Views
775
Slideshare-icon Views on SlideShare
775
Embed Views
0

Actions

Likes
1
Downloads
10
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Copyright UserWorks, Inc. 2008
  • Dana: Let’s begin with a few misconceptions about note-taking. Preparation = getting familiar with the product being tested and understanding project and task goals Copyright UserWorks, Inc. 2008
  • Dana: So why is important to be a good note taker? [Recording] So you don’t need to re-watch the recordings All of these help the report basically write itself. Copyright UserWorks, Inc. 2008
  • Kristen: Let’s start with the first exercise. Copyright UserWorks, Inc. 2008
  • Kristen: Let’s say you are being pulled into a study at the last minute. I’m going to show you a video clip and I want you to take notes on what you observe. I’m not going to give you any background to the study shown in the clip or to the situation. I just want you to write down what you think is important to note. [Video Clip: NIHSeniorHealth.gov phase 1B P4 13:57-16:59] Copyright UserWorks, Inc. 2008
  • Kristen: Without having the appropriate context it can be difficult to take meaningful notes. So did you capture the key findings? Did you miss anything? It’s probably safe to say with a little more context you could improve your notes and therefore the report. Copyright UserWorks, Inc. 2008
  • Kristen: So let’s try this activity again, this time with some background information on the study. Copyright UserWorks, Inc. 2008
  • Kristen: This is the same study, but a different information retrieval task… For this exercise let’s focus on the goals related to pagination. So either if the participant noticed and used the pagination and if he was able to successfully locate the information that he was asked to find, appearing on the 2 nd page. Copyright UserWorks, Inc. 2008
  • Kristen: [Video Clip: SeniorHealth.gov 1B P1 30:07-32:29] Copyright UserWorks, Inc. 2008
  • Kristen: Now, that you had come context your note-taking should improve. Copyright UserWorks, Inc. 2008
  • Kristen: Copyright UserWorks, Inc. 2008
  • Dana: Now, let’s move on to the third exercise. This time, we’ll be focusing on metrics. It’s important for all members of the research team to come to a consensus and define all metrics prior to data collection. Copyright UserWorks, Inc. 2008
  • Dana: Copyright UserWorks, Inc. 2008
  • Dana: Again his task is to change the high alter. Copyright UserWorks, Inc. 2008
  • Dana: By a show of hands how many people rated it a … We scored this participant task with a 2 (minor difficulty) because the participant tried two different menu items before reaching the correct item, and as defined in the rating scale, that would be a 2. By a show of hands, how many also gave this participant a 2… But, as you can see, that task would have received various ratings had this scale not been established before data collection began. Copyright UserWorks, Inc. 2008
  • Dana: Though we didn’t include it in this study’s rating scale, your scale may include assistance or help from the moderator, so you would want to clearly define that as well. Copyright UserWorks, Inc. 2008
  • Kristen: Obviously, there are a lot of other factors that go into efficient and effective note-taking, but unfortunately we were unable to cover all of them in detail, but…. Copyright UserWorks, Inc. 2008
  • Kristen: These are the overall keys to success [read bullets]…. Thoroughly familiar with the tool: You can either use some type of data collection tool such as Morae or Silverback or you could use some type of Word processor or spreadsheet, or even take notes on paper using a checklist. Add to the last bullet: Participant says one thing and does another Participant thinks they have been successful when they really haven’t Participant doesn’t want to look bad, so bluffs Participant initially struggles, then succeeds, and speaks highly of the product Participant blames self and rates product highly despite disastrous task performance Participant is overly chatty and goes off on tangents Copyright UserWorks, Inc. 2008
  • Kristen Copyright UserWorks, Inc. 2008

Note-Taker's Perspective During Usability Testing (Kristen Davis & Dana Douglas) Note-Taker's Perspective During Usability Testing (Kristen Davis & Dana Douglas) Presentation Transcript

  • The Note-Takers Perspective During Usability Testing:Recognizing Whats Impor tant, Whats Not User Focus 2012 October 19, 2012 www.userworks.com (301) 431-0500 Kristen Davis & Dana Douglas
  • Note-Taking Misconceptions No preparation needed Take the notes, then figure out how to analyze and summarize the data Anyone can do it Attention to detail, fast typing are the key skills A one-size-fits-all approach will work for all practitioners and for all products under evaluationThe Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 2
  • Why is Ef fective Note-TakingImpor tant? To improve the data analysis process by: • Capturing the appropriate data the first time • Making analysis more efficient • Increasing accuracy • Identifying trends easilyThe Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 3
  • Exercise 1: Taking Notes
  • Exercise Record your observations as you watch this video clipThe Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 5
  • How Did You Do? Did you capture the key findings? Did you miss anything?The Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 6
  • Exercise 2: Know the Study
  • Practice Exercise Product: NIHSeniorHealth.gov website Participant: Seniors who look for health information online Task: Locate a specific piece of information Study Goals: • What navigation path do participants use when looking for specific pieces of information? • Do par ticipants notice and use the pagination? • Does the visual design treatment used in the left hand navigation clearly indicate the user’s location within a specific health topic? Task Goals: • Are par ticipants able to locate information on the 2 nd page? • Are the left hand navigation labels clear?The Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 8
  • Exercise Record your observations while watching this video clipThe Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 9
  • How Did You Do? Did you capture the key findings? Did you miss anything? Did you note: • If the participant noticed and used the pagination? • If the participant was able to locate information on the 2nd page?The Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 10
  • Lessons Learned: Understand theGoals Goals should be set prior to data collection in order to:• Record appropriate data• Target the type of data that will be useful when reporting the findingsThe Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 11
  • Exercise 3: Know theMetrics
  • Practice Exercise Product: Blood glucose meter Participant: Person with diabetes who uses a meter Task: Change default settings in the meter Rating scale for scoring task completion: • Completed with ease: Participant easily completed the task • Completed with minor dif ficulty: Participant somewhat struggled to complete the task (e.g., attempted more than two paths) • Completed with major dif ficulty: Participant significantly struggled to complete the task (e.g., attempted numerous paths and/or used customer service) • Failed to complete: Participant was unable to complete the taskThe Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 13
  • Exercise Record your observations while watching this video clipThe Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 14
  • How Did You Do? What score did you give the participant? • Completed with ease: Participant easily completed the task • Completed with minor dif ficulty: Participant somewhat struggled to complete the task (e.g., attempted more than two paths) • Completed with major dif ficulty: Participant significantly struggled to complete the task (e.g., attempted numerous paths and/or used customer service) • Failed to complete: Participant was unable to complete the taskThe Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 15
  • Lessons Learned: Define Metrics Defining the task completion rating scale prior to data collection allows:• Data to become less subjective• Note-taker to record the scores in a systematic and consistent way Some additional metrics:• Clickpath• Time on task• Shorthand codes  Example: Number of times a participant clicked the paginationThe Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 16
  • Summar y
  • Keys to Success Both moderator and note-taker need to know the product and the issues of interest Be thoroughly familiar with whatever tool you are using Plan for categorization and data analysis before data collection Don’t try to capture comments verbatim; interpret what is noteworthy as you goThe Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 18
  • Questions/Comments? Kristen Davis Dana Douglas UserWorks, Inc. UserWorks, Inc. User Experience Specialist/ User Experience Specialist Note-Taker Extraordinaire kdavis@userworks.com ddouglas@userworks.com www.userworks.com (301) 431-0500The Note-Takers Perspective During Usability TestingUser Focus 2012Kristen Davis and Dana Douglas- UserWorks, Inc. 19