Presentation for the Penn State Web Conference on June 11, 2012.

1. THESE KIDS TODAY
Usability Testing with Current
and Prospective Students
Lori Packer
Penn State Web
June 2012

2. WRITING DOWN
EVERYTHING I SAY?
STOP!
http://gim.ie/L6jV

3. Let me tell
you a story...
...about wine.

4. Let’s play
“Find the wine”
MSN
circa 1998

5. USABILITY TESTING
“Evil Empire” $tyle

6. USABILITY TESTING
“Evil Empire” $tyle
Two-way mirrored glass
Eye-tracking cameras
Multiple monitors
Representative audience samples
Report = Statistical analysis

7. IT DOESN’T HAVE TO BE
THIS WAY

8. KEEP IT
SIMPLE
Steve Krug provides a
“common man” approach
to Web usability testing.

9. KEEP IT
SIMPLE
Steve Krug provides a
“common man” approach
to Web usability testing.

10. USABILITY TESTING
“Don’t Make Me Think” style

11. USABILITY TESTING
“Don’t Make Me Think” style
Your office, your computer, and a microphone
Free screen-capture software
Whoever you can get ahold of
Donuts and iTunes gift cards for participants
Report = three paragraph email

12. USABILITY TESTING
“Evil Empire” $tyle
Two-way mirrored glass
Eye-tracking cameras
Multiple monitors
Representative audience samples
Report = Statistical analysis

13. USABILITY TESTING
“Evil Empire” $tyle
Two-way mirrored glass
Eye-tracking cameras
Multiple monitors
Representative audience samples
Report = Statistical analysis

14. WHAT IS USABILITY TESTING?
The act of observing a real human being attempt
to use something for its intended purpose.

15. WHAT IS USABILITY TESTING?
The act of observing a real human being attempt
to use something for its intended purpose.
Can apply to coffee makers or websites.

16. WHAT IS USABILITY TESTING?
The act of observing a real human being attempt
to use something for its intended purpose.
Can apply to coffee makers or websites.
“It only takes five users to uncover 80 percent
of high-level usability problems.” -- Jakob Nielsen

17. WHAT ISN’T USABILITY TESTING?
A usability test in NOT a focus group.
You are not asking people what they LIKE or WANT.
You are observing what they can DO and how you
can IMPROVE how they do it.

18. WHAT ISN’T USABILITY TESTING?
A usability test in NOT a focus group.
“How would you describe this shade of blue?
Sophisticated? Somber? Peppy?”
You are not asking people what they LIKE or WANT.
You are observing what they can DO and how you
can IMPROVE how they do it.

19. WHAT ISN’T USABILITY TESTING?
A usability test in NOT a focus group.
“How would you describe this shade of blue?
Sophisticated? Somber? Peppy?”
NO.
You are not asking people what they LIKE or WANT.
You are observing what they can DO and how you
can IMPROVE how they do it.

20. SO HOW DO WE DO THIS
THING?

21. HOW DO I DO IT?
• At least once a year, conduct test of homepage, second-level pages
• Recruit 3-5 volunteers from each audience group: new students,
prospective students, parents
• Bring them to my office, with a computer set up with microphone,
screen capture
• Ask them to “think aloud” while using the site as they normally would
first, then through a series of tasks
• Each session takes 30-40 mins
• Report results to Web team

22. RECRUIT PARTICIPANTS
• 3-5 people from each group
• Forparents and new students:
walk around with a clipboard
during orientation expo
• For
current students: listing in
weekly e-newsletter
• For prospective students: listing
in faculty/staff e-newsletter for
high-school age kids

23. WHY NEW STUDENTS?
Do usability tests
during the first
two days on
campus.
Students learn the
“lingo” so fast --
Registrar, LAS, ITS
Center.
What do they do
before they know
what they’re doing?

24. SET UP “USABILITY LAB”

25. SET UP “USABILITY LAB”

26. SET UP “USABILITY LAB”
Screen recording software
• CamStudio: free, records to .AVI format only
• Debut Video Capture: $39.95, records to multiple
formats
• Camtasia: $299, speech-to-text transcriptions
• Morae: $1,495, statistics, picture-in-picture
Microphone
• Cheap USB microphone: $25
Other observers
• Invite other members of team to session
• For large teams, use Skpe or GoToMeeting to share
screen to a conference room

27. STICK TO THE SCRIPT
Hello, _________,
My name is Lori and I’ll be conducting this session with you
today testing the University homepage...
... I want to make clear right away that we are testing the site,
not you ...
... As we go along, I am going to ask you to think out loud:
basically say whatever it is that is going through your mind as
you use the site ...
... I’m using screen recorder software and this microphone to
record your actions ...

28. TIME NEEDED
• Introductionand background
questions: 5 mins
• Homepage initial reactions
(don’t click on anything): 5 mins
• “Free-range” clicking: 5 mins
• Specific tasks: 15 mins
• Final thoughts: 5 mins
• TOTAL: 30-40 mins

29. HOW ‘BOUT SOME
EXAMPLES?

30. HOMEPAGE USABILITY TESTS
WINTER 2011

31. HOMEPAGE USABILITY TESTS
WINTER 2011
• 14 current students (broke my own rule right away!)

32. HOMEPAGE USABILITY TESTS
WINTER 2011
• 14 current students (broke my own rule right away!)
• Asked to use the page on their own -- thinking out loud as
they go -- then asked a series of tasks like:
• Find the next holiday or break
• Log on to Blackboard
• How late is the library open tomorrow?
• What’s your professor’s e-mail address?
• What do you do if you need to apply for summer housing?

33. HOMEPAGE USABILITY TESTS
WINTER 2011

34. HOMEPAGE USABILITY TESTS
WINTER 2011
•4 prospective students

35. HOMEPAGE USABILITY TESTS
WINTER 2011
•4 prospective students
• Use the page on their own, then tasks like:
• Do you have an idea of what you might like to study in college?
• Can you major in psychology here?
• What are the dorms like?
• How much is tuition?
• What is the deadline for applying?

36. HOMEPAGE USABILITY TESTS
WINTER 2011
•4 prospective students
• Use the page on their own, then tasks like:
• Do you have an idea of what you might like to study in college?
• Can you major in psychology here?
• What are the dorms like?
• How much is tuition? Watch
• What is the deadline for applying?
word choice!

37. SOME EXAMPLE FINDINGS

38. SOME EXAMPLE FINDINGS
• Currentstudents rely much more on searching than
prospective students.

39. SOME EXAMPLE FINDINGS
• Currentstudents rely much more on searching than
prospective students.
• Prospectivestudents didn’t notice Majors page; current
students didn’t browse Student Services page.

40. SOME EXAMPLE FINDINGS
• Currentstudents rely much more on searching than
prospective students.
• Prospectivestudents didn’t notice Majors page; current
students didn’t browse Student Services page.
• Three of the current students and one of the prospective
students did not scroll down on the homepage; the rest
scrolled pretty much immediately.

41. SOME EXAMPLE FINDINGS
• Currentstudents rely much more on searching than
prospective students.
• Prospectivestudents didn’t notice Majors page; current
students didn’t browse Student Services page.
• Three of the current students and one of the prospective
students did not scroll down on the homepage; the rest
scrolled pretty much immediately.
• One person hated the homepage photos. :-(

42. EXAMPLE
“Do you know what you want
to study in college?”

43. EXAMPLE
“What do you need to do if
you want to spend a semester
overseas?”

44. EXAMPLE
“How would you find one of
your professor’s email
addresses?”

45. EXAMPLE
“Where would you go to find
information about internships
you might qualify for?”

46. SOME SAMPLE CHANGES

47. SOME SAMPLE CHANGES
• Re-arrange Student Services to be subject-based not school-based.

48. SOME SAMPLE CHANGES
• Re-arrange Student Services to be subject-based not school-based.
• Add Academic Services section to Academics page.

49. SOME SAMPLE CHANGES
• Re-arrange Student Services to be subject-based not school-based.
• Add Academic Services section to Academics page.
• Made Majors, Degrees and Programs a top heading.

50. SOME SAMPLE CHANGES
• Re-arrange Student Services to be subject-based not school-based.
• Add Academic Services section to Academics page.
• Made Majors, Degrees and Programs a top heading.
• Add “SEARCH ROCHESTER.EDU” to search box so users don’t confuse
it with phone/email directory.

51. SOME SAMPLE CHANGES
• Re-arrange Student Services to be subject-based not school-based.
• Add Academic Services section to Academics page.
• Made Majors, Degrees and Programs a top heading.
• Add “SEARCH ROCHESTER.EDU” to search box so users don’t confuse
it with phone/email directory.
• Add link to Blackboard to the top of the template.

52. SOME SAMPLE CHANGES
• Re-arrange Student Services to be subject-based not school-based.
• Add Academic Services section to Academics page.
• Made Majors, Degrees and Programs a top heading.
• Add “SEARCH ROCHESTER.EDU” to search box so users don’t confuse
it with phone/email directory.
• Add link to Blackboard to the top of the template.
• Add Career Services to Working page.

53. SOME SAMPLE CHANGES
• Re-arrange Student Services to be subject-based not school-based.
• Add Academic Services section to Academics page.
• Made Majors, Degrees and Programs a top heading.
• Add “SEARCH ROCHESTER.EDU” to search box so users don’t confuse
it with phone/email directory.
• Add link to Blackboard to the top of the template.
• Add Career Services to Working page.
• Crop homepage photos narrower whenever possible.

59. OK, WHY DO WE DO THIS
THING?

60. BECAUSE I SAID SO.

61. BECAUSE I SAID SO.
Because after only one session, you will learn at
least two things you can do to improve how users
experience your site.
Because no website ever got WORSE because you
spent 40 minutes watching someone use it.
I promise.

62. QUESTIONS?