Are learners getting lost while navigating your online course or LMS? Learn how to test and spot the most common usability issues in eLearning.

1. Do-It-Yourself
Usability Testing for eLearning
Maria Ginsbourg
Instructional Designer
Maria.Ginsbourg@Gmail.com

2. Agenda
Definitions
Examples
Usability Testing Overview
User Testing
Heuristic Evaluation

3. “Making sure that something works well: that
a person of average (or even below average)
ability and and experience can use the thing −
for it’s intended purpose without getting
hopelessly frustrated.”
Steve Krug, author of Don’t Make Me Think
What does usability mean?

4. What does usability issue mean?
An aspect of the
system which
makes it difficult for
the user to achieve
their goals.

5. Examples of usability issues
in online courses and
Learning Management Systems

6. Inconsistent navigation
Submit Quiz
Are you sure you want to submit the
quiz?
.
Cancel Submit
Submit Quiz
Are you sure you want to submit the
quiz?
Submit Cancel
People are on auto-pilot; they will click the button to the right without
reading what it says. Designing navigation that follows conventions and
best practices reduces extraneous processing, so learners can focus on
what matters most: mastering the subject matter.

7. Ignoring visual standards/conventions
This feedback slide contains checkmarks, Xs, and arrows.
The arrows do not provide additional information.

8. System oriented language (not English)
These two
tabs seem to
be saying the
same thing.
What does
‘Commons’
mean?

9. Ambiguous Icons
Is that a
back
button?
Can I click
on the
compass?
When the meaning of an icon is ambiguous, the learner can get
distracted, confused and disoriented. Add text labels to improve clarity.

10. Complex
To reduce cognitive overload, eliminate redundancy. Each additional
link depletes users’ attention because it competes with all others.
See: https://www.nngroup.com/articles/duplicate-links/

11. Usability Testing Overview

12. Comparison of methods
Legend
5 = very useful
4 = somewhat useful
Heuristic evaluation and user testing are the most popular and useful
usability methods, according to one study.
See: https://www.nngroup.com/articles/technology-transfer-of-
heuristic-evaluation/

13. 2-5 evaluators
separately review
the course/LMS and
judge against best
practices.
Heuristic Evaluation User- Testing
An evaluator
observes 3-5 people
use the course/LMS
and note what
people say, do and
facial expressions.

14. Best practices for usability testing
• Start earlier than you are comfortable
• Get 3-5 test users/evaluators
• Test, tweak, test (it’s an iterative process)

15. Benefits: ROI
Graph shows return on investment for usability testing for web projects.
See: https://www.nngroup.com/articles/usability-roi-declining-but-still-
strong/

16. User Testing
A test facilitator sits with a user and observes
the user complete tasks.

17. What’s the process?
Write tasks
Recruit
participants
Conduct testPrioritize
Tweak

18. Plan it
Download PDF: https://www.userfocus.co.uk/pdf/usabilitydashboard.pdf

19. How to write a task?
Task Goal:
Look up grades.
Poor task (do not provide step-by-step instructions)
You want to see the results of your midterm exam.
1. Go to the websiteS
2. Sign in
3. Tell me where you would click to get your grade.
Better task (do provide a specific goal)
Look up the results of your midterm exams.
View a list of tasks for testing an online course & an LMS
https://christytucker.wordpress.com/2008/02/06/lms-usability-testing/

20. Who do you test with?
Time
Representative Users
Internal Staff
(different division)
Budget
Friends, family
neighbors
Internal staff
(Your division)
Actual Users
Actual and representative users can help you check if the industry
jargon makes sense, if the learning activities are useful and contextually-
relevant. Internal staff can help you find obvious usability issues.

21. Test day:
“Everything that can go wrong will.”
Murphy's Law
Usability testing is the one day where you want
everything that can go wrong, to go wrong. It’s
better that usability issues occur during testing,
than with actual learners.

22. Observing users
Body
• Fidgeting
• Rubbing head/eyes/neck
• Leaning in
Eyes
• Skimming
• Squinting
• Looking around page, confused
Hand
• Hovering to see if element is interactive
• Hesitating before clicking
• Clicking randomly
• Struggling with scrolling or dragging

23. What should we fix first?
Users can’t do task
Users are annoyed
Affect few Affect Many
“Focus ruthlessly on only the most serious problems.”
~Steve Krug, Rocket Surgery Made Easy

24. How should we fix things?
“Tweak, don’t redesign”
“When fixing problems, try
to do the least you can do.”
~Steve Krug, Rocket Surgery Made Easy

25. 2-5 evaluators separately
review the course/LMS
against best practices.
Then evaluators meet and
discuss findings.
Heuristic Evaluation

26. Nielsen’s Heuristics
https://chatbotsmagazine.com/usability-heuristics-for-bots-7075132d2c92

27. Heuristic Evaluation Form
Slide # Screenshot Heuristic Notes
2.1 1, 5 Buttons need hover and
visited states.

28. 1. Visibility of system status
Your Essay is Approved!
Grade
Good example
Learner: I know what’s going on.

29. 2. Match between system and the real world
Learner: I don’t understand these terms.
Poor example

30. 3. User control and freedom
Locked course navigation
Learner: I can’t access the info I need, when I need it for my job!
https://blogs.articulate.com/rapid-elearning/heres-why-unlocking-your-course-navigation-will-create-better-learning/

31. 4. Consistency and standards
Submit Quiz
Are you sure you want to submit the
quiz?
.
Cancel Submit
Submit Quiz
Are you sure you want to submit the
quiz?
Submit Cancel
Poor example
Learner: I meant to click submit. Why did I hit cancel?

32. 5. Error Prevention
Learner: Did I already visit this one? I can’t tell.
Poor example

33. 6. Recognition rather than recall
Good example
Learner: This helps me remember.

34. 7. Flexibility and efficiency of use
Good example
Novice user: That was easy!
Advanced user: That was fast!

35. 8. Aesthetic and minimalist design
Poor example
Novice: It’s overwhelming. I’m lost and confused.

36. 9. Help users with errors
Submit Quiz
Are you sure you want to submit the quiz?
You didn’t answer question #7.
Go BackSubmit
Good example
Learner: Phew! What a relief!

37. 10. Help and documentation
Good example
Learner: Cool, I get it!

38. www.linkedin.com/mariaginsbourg
Maria.Ginsbourg@Gmail.com

39. Great Resources
• https://www.nngroup.com/
• https://usability.gov