1. Today’s plan
1. Tasks left to do in the course?
2. Things you need to know in order to accomplish tasks?
3. Usability reporting - assignment instructions
4. Create usability testing teams & plans
5. NO class next week - email me if you need me
6. Dec. 4 - Final presentations: UCD (10 min) + Usability (10 min)

2. Usability Reporting

3. Report Structure
1. Executive summary
5. Results
2. Product description
a. Performance metrics
3. Testing goals
b. Qualitative results
4. Methods
c. Usability issues &
Recommendations
a. Participants
b. Data collection
procedures
c. Tasks

4. 1
Performance
Task success Time on task
2
Issue Severity
Behavioral
Obs:Verbal bhvr
4
Efficiency
Issue metrics
Identify Issue
3
Errors
Obs: Nonverbal bhvr
Self-reported
Ease
Satisfaction
Clarity
Comprehension
etc.

5. By task
1. Results
a. Performance metrics
b. Qualitative results
c. Usability issues &
Recommendations
Cross-task
comparison
What goes first?

6. HOW do you present results?
visually
graphs & charts -what
kinds?
tables
enable easy
comparisons

7. Assuming 5 users & the following measures:
1. Demographics, background
2. Task completion
3. Time on task
4. Expected difficulty
5. Actual difficulty
6. Observations
7. Overall satisfaction

8. 1. Demographics, background
3
2.25
1.5
0.75
0
Male
Female
Male
Female

9. M
30
Graduate
(G1)
Senior
Electrical
and
Computer
Engineering

10. Participants Info.
4 Grad students (male)
1 Faculty member (female)
Who
How often
Age
Mechanical Eng.
Material Eng.
Electrical & Computer Eng.
Once a week (average)
27.2 years old (average)
Range: 20~40

11. Participants Info. (Cont’d)
Total length of usage
Time spent per Log-in
5
4
4
2.5
Hour(s)
Year(s)
3
2
1
3
2
1.7
1
0
P1
P2
P3
P4
Participant #
P5
AVE
0
P1
P2
P3
P4
Participant #
P5
AVE

12. 1. Demographics, background
Which presentation mode
did you like best?
Show & Tell:
Recognition rather than Recall

13. 1
Participants Info.
4
Participants Info. (Cont’d)
Total length of usage
4 Grad students (male)
Who
How often
Age
5
4
Mechanical Eng.
Material Eng.
Electrical & Computer Eng.
Time spent per Log-in
3
Once a week (average)
27.2 years old (average)
Range: 20~40
4
2.5
Hour(s)
1 Faculty member (female)
Year(s)
3
2
2
1
3
2
1.7
1
0
P1
P2
P3
P4
Participant #
P5
AVE
0
P1
P2
P3
P4
Participant #
P5
AVE

14. 1. Demographics, background
Which presentation mode
did you like best?
Because of small number of
users, present both
averages and individual
data.

15. 2. Task completion
5 users -- 5 tasks
•
•
•
All 5 users completed tasks 1 and 2
3 users completed task 3; 1 partial
completion, 1 incomplete
3 users completed task 4; 2 partial
completions
• task 5: 2 users completed, 3 partial
completions

17. 3. Time on task
5 users -- 5 tasks
T1
T2
T3
T4
U1
U2
U3
U4
U5
Average per user
OR
Average per task?
T5

18. Which task was more difficult to complete?
Mean Time-on-Task (95%CI)
270!
240!
210!
203!
Time(sec)
180!
150!
120!
90!
66!
75!
60!
30!
49!
25!
0!
Task#1!
Task#2!
Task#3!
Task#4!
Task#5!

19. Is this graph useful? Why?

20. 4. Expected difficulty and 5. Actual difficulty
• Per task?
• Average across tasks?

21. 4. Expected difficulty and 5. Actual difficulty

22. 6. Observations
•
What do you think would be important to
report?
• How?

23. 7. Overall satisfaction

24. Overall “barometer”
# of Participants
0
Search
Feature
1
1
2
3
1
Ease of
Navigation
4
3
4
1
Terminology
Clarity
3
2
Organization
Clarity
3
2
Satisfied
5
Neutral
Disatisfied

25. Meaningful, Easy to read & Understand
1. Demographics, background
2. Task completion
3. Time on task
4. Expected difficulty
5. Actual difficulty
6. Observations
7. Overall satisfaction