The document discusses usability testing and its key aspects. It defines usability testing as employing techniques to collect empirical data by observing representative end users perform realistic tasks on a product. There are different types of usability tests - exploratory, assessment, and validation - conducted at various stages of product development to evaluate usability. An effective moderator is important for a successful test, requiring skills like rapport building and objectivity. Tests involve representative tasks, questionnaires, and debriefing to understand user difficulties and identify areas for improvement.

1. Usability Testing

2. What makes something usable?

3.  In large part, what makes something usable is the absence of
frustration in using it.
 Usefulness concerns the degree to which a product enables a
user to achieve his or her goals, and is an assessment of the
user's willingness to use the product at all.
 Five Characteristics:
 Error Tolerant
 Efficiency
 Effectiveness
 Easy to Learn
 Engaging

4. What is usability testing?

5.  Employs techniques to collect empirical data while
observing representative end users using the product
to perform realistic tasks.
 Basic Elements of Usability Testing are research
questions or test objectives rather than hypotheses.
Use of a representative sample of end users which
may or may not be randomly chosen.

6.  Basic principles of user-centered design:
 Early focus on users and their tasks
 Evaluation and measurement of product usage
 Iterated design

7. When should you test?

8.  Some type of usability testing fits into every phase of
a development lifecycle. The type of testing is
distinguished by:
 Research questions asked
 State of the completeness of the product
 Time available for implementing solutions to problems
revealed in testing.
 Three main tests:
 Exploratory (or formative)
 Assessment (or summative)
 Validation (or verification)

9. What types of tests?

10. Exploratory or Formative
Typical user-oriented questions that an exploratory
study would attempt to answer might include the
following:
 What do users conceive and think about using the
product?
 Does the product's basic functionality have value to
the user?
 How easily and successfully can users navigate?
 What type of prerequisite information does a person
need to use the product?

11. Assessment or Summative
 The assessment test is probably the most typical type
of usability test conducted. Of all the tests, it is
probably the simplest and most straightforward for
the novice usability professional to design and
conduct.

12. Validation or Verification Test
 Usually conducted late in the development cycle
 Intended to measure usability of a product against
established benchmarks
 Confirm that problems discovered earlier have been
remedied and that new ones have not been
introduced.

13. Testing Environment

17. Skills for the moderator

18.  He or she has the potential to make or break the test.
An ineffective moderator can seriously negate test
results and even waste much of the preliminary
preparation work.
 This is even true for experienced test moderators
who conduct the test from an external control room.
In fact, asking someone to test his or her own
product is like asking parents to objectively evaluate
the abilities of their child. It is an impossible
endeavor.

19.  quick learner
 instant rapport with participants
 excellent memory
 good listener
 comfortable with ambiguity
 flexibility
 empathic “People Person”
 “Big Picture” thinker
 good organizer and coordinator
 capability to play apprentice to the master
Moderator should consist of:

20.  Not Relate Well to Each Participant
 Jump to Conclusions
 Too Involved with the Act of Data Collection
 Acting Too Knowledgeable
 Too Rigid with the Test Plan
 Leading Rather Than Enabling
Not to do as Moderator

21. What are the parts of Usability testing?

22.  Orientation script
 Background questionnaire
 Data collection instruments
 Nondisclosure agreement and recording consent
form
 Pre-test questionnaire
 Task scenarios
 Post-test questionnaire
 Debriefing topics guide

23.  Communicate Research Questions
 Be clear on Goals for testing
 Summarize Participant Characteristics
 Describe the Method
 List the Tasks

24.  Ways to Prioritize Tasks:
 Prioritize by frequency
 Prioritize by criticality
 Prioritize by vulnerability
 Prioritize by readiness
 Due to time constraints, very rarely do you actually
test the full range of tasks that comprise an entire
interface, documentation, or both together.

25. Questionnaires

26. How to ask questions for questionnaires:
Open-ended
 Provide enough parameters
 For example “What is your current occupation?”
 What are you asking? What do you want to know?
 Might be better to break out into several questions:
 What is your job title?
 Describe your job responsibilities
 Who do you work for?
Close-ended
 Likert scales
 Strong disagree to strongly agree
 Very unfavorable to very favorable
 Poor to excellent
 Low priority to high priority

27. Principles of Writing Questions
 Choose simple words
 Chose simple combinations
 ‘Subnational region’ becomes ‘area of the country’
 ‘Post-school extracurricular activity’ becomes ‘what you do after school’
 Chose as few words as possible to pose the question
 Use complete sentences to ask questions
 Avoid vague quantifiers when more precise estimates can be
obtained
 Avoid specificity that exceeds respondents potential to have
ready-made answers
 For example: Most people could not come up with a precise amount of
books they have read over the last year
 NEVER make people predict their behavior

28.  Distinguish undecided from neutral by placing it
(undecided) on the end of the scale
 Use equal numbers of positive and negative
categories for scalar questions
 Avoid bias from unequal comparisons
 “Which of the following do you feel is most responsible for
recent outbreaks of violence in America’s schools?”
 Irresponsible parents
 School policies
 Television programs

29. Conducting the test session

30. Steve Krug’s Demo
http://www.youtube.com/watch?v=QckIzHC99Xc

31. Debriefing

32. Debriefing refers to exploring and reviewing the participant's
actions during the performance portion of a usability test.
 Never make participants feel at all defensive about their actions or their opinions.
 While questioning a participant, do not react to the participant's answers one way or
another.
 Gather your thoughts while the participant fills out any post-test questionnaires.
 Begin by letting the participant say whatever is on his or her mind.
 Focus on understanding problems and difficulties, not on problem solving.
 Finish your entire line of questioning before opening up the floor to discussion by
observers.

33. Reports and Analysis

34. 1. Compile and summarize data.
2. Analyze data.
3. Develop findings and recommendations.
4. Produce the final report.
The report should discuss the design implications for
the rest of the team. What were the findings, how the
findings resulted in recommendations, and how those
recommendations should be implemented in the
design.

36. Majority of information from this book:
*These slides were created for educational purpose and not commercial.

37. Further References
 http://www.wqusability.com/articles/more-than-
ease-of-use.html
 http://www.steptwo.com.au/papers/kmc_whatisusa
bility