Today’s plan
1. Tasks left to do in the course?
2. Things you need to know in order to accomplish tasks?
3. Usability repo...
Usability Reporting
Report Structure
1. Executive summary

5. Results

2. Product description

a. Performance metrics

3. Testing goals

b. Qu...
1

Performance

Task success Time on task
2

Issue Severity

Behavioral
Obs:Verbal bhvr

4

Efficiency

Issue metrics
Ident...
By task

1. Results
a. Performance metrics
b. Qualitative results
c. Usability issues &
Recommendations

Cross-task
compar...
HOW do you present results?

visually
graphs & charts -what
kinds?
tables

enable easy
comparisons
Assuming 5 users & the following measures:

1. Demographics, background
2. Task completion
3. Time on task
4. Expected diffi...
1. Demographics, background

3
2.25
1.5
0.75
0
Male
Female

Male

Female
M
30

Graduate	
  (G1)

Senior
Electrical	
  and	
  Computer	
  Engineering
Participants Info.
4 Grad students (male)

1 Faculty member (female)
Who

How often
Age

Mechanical Eng.
Material Eng.
Ele...
Participants Info. (Cont’d)
Total length of usage

Time spent per Log-in
5

4

4

2.5
Hour(s)

Year(s)

3

2

1

3

2

1.7...
1. Demographics, background

Which presentation mode
did you like best?

Show & Tell:

Recognition rather than Recall
1

Participants Info.

4

Participants Info. (Cont’d)
Total length of usage

4 Grad students (male)
Who

How often
Age

5
...
1. Demographics, background

Which presentation mode
did you like best?
Because of small number of
users, present both
ave...
2. Task completion

5 users -- 5 tasks

•
•
•

All 5 users completed tasks 1 and 2
3 users completed task 3; 1 partial
com...
3. Time on task

5 users -- 5 tasks
T1

T2

T3

T4

U1
U2
U3
U4
U5

Average per user
OR
Average per task?

T5
Which task was more difficult to complete?
Mean Time-on-Task (95%CI)
270!
240!
210!

203!

Time(sec)

180!
150!
120!
90!
66!...
Is this graph useful? Why?
4. Expected difficulty and 5. Actual difficulty

• Per task?
• Average across tasks?
4. Expected difficulty and 5. Actual difficulty
6. Observations

•

What do you think would be important to
report?

• How?
7. Overall satisfaction
Overall “barometer”
# of Participants

0

Search
Feature

1

1

2

3

1

Ease of
Navigation

4

3

4

1

Terminology
Clari...
Meaningful, Easy to read & Understand

1. Demographics, background
2. Task completion
3. Time on task
4. Expected difficulty...
Usability reporting
Upcoming SlideShare
Loading in...5
×

Usability reporting

1,130

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,130
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Usability reporting

  1. 1. Today’s plan 1. Tasks left to do in the course? 2. Things you need to know in order to accomplish tasks? 3. Usability reporting - assignment instructions 4. Create usability testing teams & plans 5. NO class next week - email me if you need me 6. Dec. 4 - Final presentations: UCD (10 min) + Usability (10 min)
  2. 2. Usability Reporting
  3. 3. Report Structure 1. Executive summary 5. Results 2. Product description a. Performance metrics 3. Testing goals b. Qualitative results 4. Methods c. Usability issues & Recommendations a. Participants b. Data collection procedures c. Tasks
  4. 4. 1 Performance Task success Time on task 2 Issue Severity Behavioral Obs:Verbal bhvr 4 Efficiency Issue metrics Identify Issue 3 Errors Obs: Nonverbal bhvr Self-reported Ease Satisfaction Clarity Comprehension etc.
  5. 5. By task 1. Results a. Performance metrics b. Qualitative results c. Usability issues & Recommendations Cross-task comparison What goes first?
  6. 6. HOW do you present results? visually graphs & charts -what kinds? tables enable easy comparisons
  7. 7. Assuming 5 users & the following measures: 1. Demographics, background 2. Task completion 3. Time on task 4. Expected difficulty 5. Actual difficulty 6. Observations 7. Overall satisfaction
  8. 8. 1. Demographics, background 3 2.25 1.5 0.75 0 Male Female Male Female
  9. 9. M 30 Graduate  (G1) Senior Electrical  and  Computer  Engineering
  10. 10. Participants Info. 4 Grad students (male) 1 Faculty member (female) Who How often Age Mechanical Eng. Material Eng. Electrical & Computer Eng. Once a week (average) 27.2 years old (average) Range: 20~40
  11. 11. Participants Info. (Cont’d) Total length of usage Time spent per Log-in 5 4 4 2.5 Hour(s) Year(s) 3 2 1 3 2 1.7 1 0 P1 P2 P3 P4 Participant # P5 AVE 0 P1 P2 P3 P4 Participant # P5 AVE
  12. 12. 1. Demographics, background Which presentation mode did you like best? Show & Tell:
 Recognition rather than Recall
  13. 13. 1 Participants Info. 4 Participants Info. (Cont’d) Total length of usage 4 Grad students (male) Who How often Age 5 4 Mechanical Eng. Material Eng. Electrical & Computer Eng. Time spent per Log-in 3 Once a week (average) 27.2 years old (average) Range: 20~40 4 2.5 Hour(s) 1 Faculty member (female) Year(s) 3 2 2 1 3 2 1.7 1 0 P1 P2 P3 P4 Participant # P5 AVE 0 P1 P2 P3 P4 Participant # P5 AVE
  14. 14. 1. Demographics, background Which presentation mode did you like best? Because of small number of users, present both averages and individual data.
  15. 15. 2. Task completion 5 users -- 5 tasks • • • All 5 users completed tasks 1 and 2 3 users completed task 3; 1 partial completion, 1 incomplete 3 users completed task 4; 2 partial completions • task 5: 2 users completed, 3 partial completions
  16. 16. 3. Time on task 5 users -- 5 tasks T1 T2 T3 T4 U1 U2 U3 U4 U5 Average per user OR Average per task? T5
  17. 17. Which task was more difficult to complete? Mean Time-on-Task (95%CI) 270! 240! 210! 203! Time(sec) 180! 150! 120! 90! 66! 75! 60! 30! 49! 25! 0! Task#1! Task#2! Task#3! Task#4! Task#5!
  18. 18. Is this graph useful? Why?
  19. 19. 4. Expected difficulty and 5. Actual difficulty • Per task? • Average across tasks?
  20. 20. 4. Expected difficulty and 5. Actual difficulty
  21. 21. 6. Observations • What do you think would be important to report? • How?
  22. 22. 7. Overall satisfaction
  23. 23. Overall “barometer” # of Participants 0 Search Feature 1 1 2 3 1 Ease of Navigation 4 3 4 1 Terminology Clarity 3 2 Organization Clarity 3 2 Satisfied 5 Neutral Disatisfied
  24. 24. Meaningful, Easy to read & Understand 1. Demographics, background 2. Task completion 3. Time on task 4. Expected difficulty 5. Actual difficulty 6. Observations 7. Overall satisfaction

×