GENERAL CONSTRUCTION1. Goals2. Prepare- Recruit (Moderated)- Conduct (Moderated)- Set up (Unmoderated)3. Watch the videos & analyze4. Report ﬁndings
PREP STEP 1: GOALSTalk to stakeholders:“What are the business goals of the study?”“What do we need to learn?”“What are our primary tasks & personas?”“When do we need it?”“What’s our budget for usability testing and ﬁxingproblems?”
PREP STEP 2: WHAT TO TEST? Information architecture Prototype Staging site Production siteYou can test at any point during the projectlifecycle
PREP STEP 3: PARTICIPANTSWho to test?New users vs. current usersPrimary roles & personasHow many to test?5 participants per study is ideal for uncoveringthe major usability problems while remainingquick and cost effective.15+ participants for a card sort / IA study
PREP STEP 4: CHOOSING THE METHOD Variety of webappsUsertesting.com, Verifyapp, Treejack, etc
PREP STEP 5: STUDY OUTLINE Starting URL Scenario Tasks (include login details) Post-session questions Choose your participants http://bit.ly/views-study
PREP STEP 5a: WRITING TASKS Do not use words that are in the UI Phrase things in the way people think Write in plain languagePut tasks in order; dependencies between the tasks? Include assists Ask them to rate their experience
PREP STEP 5: TEST SITE ENVIRONMENT How will they access the site? Content and user account setup is needed? Do you need to set up separate sites for each? Or re-set between sessions? Tip: Use can use Drush to disable and uninstall modules! Or, if you have a dev, stage and prod environment, overwrite the database on the testing environment
Starting URL: drupal.org Scenario: You have built sites in otherCMS, have heard of Drupal, and would like to know more. You work for a non-proﬁt that is looking to replace it’s site with an open source solution.
Task 1: Explore for a couple minutes and tell me what you have learned about Drupal.Task 2: You want to ﬁnd Drupal experts for hire. Show me how you would do this.Task 3: If you didn’t ﬁnd it already, Marketplacelists Drupal companies. Now, you want to learnmore about the Drupal community. What would you do? Task 4: Let’s say you want to post a supportquestion in the forums. Tell me what steps you would take.
PREP STEP 8: OBSERVATION FORMPriority/SeverityIssue summaryLink to screenshotTask nameParticipants who had the issue (frequency)
STEP 5: WATCH THE VIDEOSWhat to note?Things that went wellThings that did not go wellCapture quotes and annotate screenshots or getvideo clips
STEP 5: ANALYSIS, cont.Categorize things that did not go well into bugsand usability issues Severity: Low/Medium/ High (or 1-5) Frequency: Number of occurrences Scope: Local/ Global
STEP 6: THE EXECUTIVE SUMMARYOverall: There were no Critical problemsuncovered. Overall, the response was positiveand participants could ﬁnd their way towardsachieving the tasks quickly.High severity ﬁndings• Bulleted listMedium severity ﬁndings• Bulleted listThe detailed ﬁndings can be found at [URL].
Tools for testing livesites- Usertesting.com- Userlytics.com- Loop11Tools for testingscreenshots- Five Second Test- Invision app- VerifyappOther- Treejack (IA studies)- Webform.com (Diary studies)