Want to make your course website more usable? Follow these principles!

1. The reMoodle Project:
Web Usability Meets Course Website Usability
The reMoodle Project. Authored by Ann Fandrey and Emily Stull Richardson. Presented to University of Minnesota eLearning Network June 17,
2015, by Ann Fandrey with Esther Maruani. Ann is an academic technologist in College of Liberal Arts, and Esther is a multimedia specialist for
CLA’s Department of Psychology.

2. Today’s talk
Ice cream by Jardson Almeida via The Noun Project
We’re obviously going to be talking within the context of Moodle and the way Moodle gets used in undergraduate CLA courses. But my hope is
that this information can be more broadly generalizable. In particular, it will be interesting for any of you who are participating in the Canvas
pilot to think about how you would apply these principles as you’re designing courses for your specific audiences and contexts.

3. In CLA, most instructors and grad TAs are in charge of their own sites, with very little help or guidance from support staff or instructional design
professionals. From department to department and even within departments, Moodle sites are utilized in a variety of ways, resulting in
inconsistent experiences from a student perspective.

4. Prior to working on the reMoodle project, I had done a lot of prior work thinking about content strategy and information architecture in relation
to course website design, specifically, ways that the wisdom of those fields could be applied to the work that we do in instructional design.
Annette McNamara and I did a talk on content strategy and instructional design at the eLearning Summit last year, and we published an article
of the same topic in Learning Solutions Magazine.
Click
image
to go to
article

5. The striking theme that emerged for me from that work, was this idea of user goals and how fundamentally, websites in the “real world” get
constructed differently than they do for online instruction. In real-world web design, web projects involve a team of people sitting and thinking
really deeply about user goals, and then they create an information architecture around those goals.
the
user

6. Too often we choose a
design because it’s doable,
not because it’s the best
we could do.
Jared Spool
Instructors have a lot more to think about: they’re planning assignments and evaluations, selecting texts and supplemental material to reinforce
the content of the subject matter, and planning for face-to-face instruction. So really the design of the course website becomes a much more
secondary concern. In situations like this, when we then try to translate our planning into a course website using unfamiliar technology, we do
whatever we can figure out how to do, rather than what is best.

7. In these situations we are most susceptible to the default suggestions within a given system. This is precisely the reason that PowerPoint is
blamed for the ubiquitousness of dull, outline-based slide decks: because the habit of vague topic header + boring bullet points is both
suggested and reinforced by the software interface and default slide layout.

8. In Moodle the interface is mostly vertical, and the prominent call to action in a blank site is “add activity or resource”.

9. Within that call to action, we are offered some 30 activities and resources, 20 side blocks with additional functionalities, and 5 course format
options. At no point does the interface ask “What is your overall course structure?”, “What are your goals with this course?” or “How would you
like your students to experience and navigate this course website?” Web usability would ask questions like these, because the answers have a
direct impact on how best to design the layout and navigation of the course website.

10. We Moodle users are at the mercy of what the interface provides us in terms of functionality, and our design decisions follow, especially if the
user is an instructor who doesn’t have the time to really learn the tools. They will accomplish their instructional goals whatever way they can
figure out on the fly, and their decisions setting up the course website have a direct impact on the student experience through an entire
semester. It is useful for us as instructional designers to understand the system as a fundamental issue for why course websites get set up like
they do, so we can help instructors understand how their design decisions impact the student experience in the course site. It’s that old mantra:
if you can articulate the problem, you can articulate the solution.
the
system

11. The system naturally encourages sites like this, which are very linear, very text-based, sort of abandoned and lifeless. In the course of deciding
to do the reMoodle project, Emily and I looked at this problem of interface and user, and we asked the question: how can we make Moodle a
better experience for both students and instructors?

12. We reviewed existing sets of guidelines for online course site design, including Quality Matters, which is probably the most well known online
course design rubric. We found that most did not address layout and navigation at all. None provided a visual, positive example. And all lacked
specificity--as in, telling an instructor exactly what to do to achieve a certain end. We wondered what would happen if we were to move away
from this ubiquitous general advice and to instead provide instructors with actual prescriptive, specific, descriptive parameters for how to
construct a successful course website.

13. Our solution became the reMoodle project (to give a clever name to a project that would follow a before-and-after home remodel metaphor.

14. We went into the project with some ambitious assumptions: that 1) it is possible to describe a one-size-fits-all design for the layout and
navigation of a Moodle site that fits all types of CLA courses, 2) if you give an instructor a positive example of a strong use of Moodle, your good
example will be instructive in and of itself, supporting higher quality course websites in the future, and 3) that limiting infinite choice by
providing prescriptive advice should actually help improve the final product because it would result in a more focused, less arbitrary design.
Lastly, 4) we assumed that our prescription would actually liberate faculty from having to expend energy on some of these more formulatic,
pedestrian decisions, freeing them to focus on teaching.
We made
some assumptions.

15. framework faculty
enrichment
service
The resulting project was one part instructional design framework, one part service, and one part faculty enrichment program, with a larger
objective of doing some outreach and getting to meet and develop relationships with faculty.

16. 1. Initial meeting
2. TEL shop performs the
work
3. Debrief, train. Then,
instructor maintains.
serviceThe reMoodle
The basic service model was to meet with the instructor to find out their attitudes and comfort level with technology and with Moodle, how
they use their Moodle site, etc. Then, the CLA TEL shop does the reMoodle itself, iterating via email with the instructor as we go along. Then we
meet with them to give 1-1 training support, introduce them to the instructor support site, and show them how to maintain the site (and also to
ask them questions that would help us improve the service).

17. framework
Our -development
process
Soft
launch
Vet
Pilotesearch Survey
We used a standard iterative design process during development of the framework.

18. 1. Prepare for class
2. Grades
3. Due dates
Top 3 reasons students come
to Moodle:
Our first question was, can we know what students’ goals are when they visit a Moodle site? To answer this question we did an informal
qualitative analysis of the free-text results of the 2013 Make Moodle Better survey, and we found that the top 3 things students are going to the
Moodle site for are to find out what they need to prepare for class, to check their grades, and to look up due dates. The top fourth thing they
were going to the Moodle site for was to complete assignments and activities. We used those user goals to design our information architecture.

19. Shopping cart by Ulrich Pohld from the Noun Project
= Everything in 1 place
= Minimal interaction costs
= Reduced clutter
= Maximum scannability
We also looked at attributes of “real world” websites to help inform the rest of the design. Inspired by real-world websites, we tried to minimize
the number of clicks it takes for students to get what they need for class by putting everything in one spot; minimize the amount of visual noise
and clutter that some sites seem to gather over time; and optimize the pages’ scannability so students can quickly orient themselves and find
what they’re looking for. “Interaction costs” is a web usability term that refers to the user’s real or perceived effort at achieving their goals, and
of course it was our goal to minimize or even eradicate the interaction cost.

20. We realized that, in a nutshell, what we were really talking about was web usability. Once we identified that connection, we looked to web
usability research and recommendations, chiefly the work of the Nielsen Norman group, WebAIM, and Web Content Accessibility Guidelines.
Usability is taking a look at how well a system helps the user achieve their goals, so it is essential to know what those goals are. User experience
is about helping them achieve their goals in a pleasurable way. Both sit on a foundation of simple functionality, e.g., can students log in? When
they press the Submit button, does it upload their assignment? And etc.
Usability is about helping them
achieve their goals.

21. The result was a 30-page framework document with 61 specific recommendations at the site, page and document level of materials. We opened
up the document for comment and discussion by diverse experts from around the U: graphic design, web design, Disability Resources Center,
the usability team, Moodle support, undergraduate students, librarians, and academic technologists. We refined and reiterated based on that
discussion.
Click
image
to go to
doc

22. Accessibility
Adaptability
Efficiency
Findability
Learnability
Predictability
Simplicity
Utility
Legibility
Readability
Satisfaction
Vitality
Maintainability
We mapped the 61 specific recommendations to 13 system quality attributes. “Quality attributes” is a systems design term that refers to
benchmarks used to measure the degree of system functionality along certain characteristics. The 6 attributes at bottom left are grouped
together because they are, in most cases, so closely associated. The numbers indicate how many recommendations mapped to each attribute.
43
7
6
7
8
8
16

23. We thought that we would also be able to use these quality attributes as heuristics to organize the faculty enrichment part of the project but
found that attempting to communicate course website usability principles in this way led to the same type of ineffective approach that we
hadn’t liked about Quality Matters and others: in the absence of specific examples, general guidance is often meaningless to the unskilled
practitioner, leading to misapplication. However, the quality attributes were useful as we wrote evaluation questions for the student and
Accessibility
Adaptability
Efficiency
Findability
Learnability
Predictability
Simplicity
Utility
Legibility
Readability
Satisfaction
Vitality
Maintainability

24. What does it look like?

25. This British History course shows what a reMoodled site looks like that uses months-based organizational labels. With a reMoodle, the
homepage becomes welcoming and digestible (rather than overwhelming and inconsistent). Unused side blocks and topic sections at the
bottom of the page are deleted. Information is grouped under easily scannable headers that are student centric. Everything required for
one class period is located on one page, and content pages utilize a consistent template to ensure learnability and predictability.
Self
enrollment
key: ID

26. This screenshot shows a templated content page. Some things that aren’t as readily apparent just from a cursory look are:
● all text is formatted with semantic HTML markup for scannability and accessibility
● language is unified, i.e., the way an activity is referred to in the syllabus is exactly the way it’s labeled in the site
● readings are linked to permalinks through Wilson Course Reserves
● the Course Files are labeled and arranged in an instructor-centric way (by topic) to ensure site maintainability (by contrast, student-
facing areas are labeled in a student-centric way)
1 content page per
F2F class session
Activities and docs
are linked in the page
Semantic markup
Consistent template
across content pages

27. Following a period of research and writing during July and August of 2014, we did a pilot of 9 course websites in order to test the framework and
see if it “worked” for the types of sites we were seeing in CLA. Those sites had no active students, but we did gather instructor feedback during
this time. We did a soft launch using 7 sites during Spring AY15 and were able to collect both student and instructor feedback, continuing our
iterative development of the framework. Currently we offer reMoodle as an established service with a standard workflow in our shop. We’re
still learning new things, the more sites we work on and the more instructors we meet.
Pilot Service →Soft Launch
● 9 sites
● No students
● Sep - Dec 2014
● 7 sites
● Live sites
● Jan - May 2015
● 3 completed sites for
Summer AY15
● 6 sites in progress for Fall
AY16
Where
we are
now

28. You are here
We ran an evaluation on Psychology 1001, the highest enrollment course in CLA. PSY 1001 routinely has about 1000 students: 600-700
classroom students who use Moodle as a companion site to their fully face-to-face experience, ~150 students who self select as partially online
students, watching recorded lectures and attending class discussion sections with a TA, ~150 students in a completely online section, and ~200
high school students participating in the College in the Schools program.

29. Students had been working in the original version of the course website all during Spring 2015. In their last month of the course, they were
offered extra credit to complete a survey comparing the original course they were used to with a reMoodled version of the course that was to
be used for the Summer 2015 version of the course. For the survey, students were asked a combination of scenarios-based questions (e.g., “find
chapter 3 quiz”) and opinion questions.

30. This is the reMoodled version of the site, referenced in the following data slides as “Summer 2015”. Students who completed the survey had not
previously encountered the reMoodled site prior to taking the survey.

31. What did students say?

32. Head by Federico Panzano
(Summer) is very crisp and well
organized compared to the last
site. The Spring site was a jumbled
mess at the top of the page and it
was very confusing to work with.

33. Head by Federico Panzano
Summer is a lot more
pleasing due to the
simplicity and organization.

34. Head by Federico Panzano
I think the Summer 2015 site is a
lot clearer and organized. I found
the Spring 2015 was hard at times
to find what I was looking for but I
did not have a problem with that
on the Summer one.

35. Head by Federico Panzano
I wish this layout was put into place
the semester I took this course! This
new layout is very easy. I remember
looking for online assignments for my
discussion group but I couldn't find it
a few times.

36. Head by Federico Panzano
I wish this layout was put into place
the semester I took this course! This
new layout is very easy. I remember
looking for online assignments for my
discussion group but I couldn't find it
a few times.
I think the Summer 2015 site is a
lot clearer and organized. I found
the Spring 2015 was hard at times
to find what I was looking for but I
did not have a problem with that
on the Summer one.
Summer is a lot more
pleasing due to the
simplicity and organization.
(Summer) is very crisp and well
organized compared to the last
site. The Spring site was a jumbled
mess at the top of the page and it
was very confusing to work with.
The Summer 2015 layout is
better in nearly every way.

37. N = 606

38. N = 606

39. N = 606

40. N = 606

41. N = 606

42. N = 606

43. Students most valued
simplicity,
utility, and
findability.
The top 3 quality attributes that students said they most valued were simplicity, utility and findability, that is, how easy the system was to use,
how easy it was to find things, and how well it helped them accomplish their goals. This finding is interesting, though only marginally useful in
making additional improvements because a lot of the reMoodle changes are not necessarily apparent from the student perspective. It is well
known in usability testing that users are not able to tell you what they want. Still, we were pleased to confirm that the reMoodle framework and
design strongly supports all three of these values.

44. What did instructors say?

45. I’ll definitely use this.
Head by Federico Panzano

46. I can see how students
like your way better.
Head by Federico Panzano

47. Head by Federico Panzano
You made it very easy
for me.

48. I get fewer student
questions. It’s replacing
the need for the
administrative PowerPoints
I usually have to give.
Head by Federico Panzano

49. I wish (this method)
would have been here
in the first place when I
first started using
Moodle.
Head by Federico Panzano

50. What else did instructors
say?
Instructors also offered some valid critique along with their laud and praise...

51. It’s harder to learn
than the old way.
Head by Federico Panzano

52. Coming in, it was
intimidating (to figure
out how to update),
especially in a time
crunch.
Head by Federico Panzano

53. I’d like your design
better if the topic
pages could expand
without having to click
into them.
Head by Federico Panzano

54. We have tried a number of methods for disseminating the framework--or at least the essence of the framework--to instructors. First, the 1-on-1
debrief/training meetings immediately following a reMoodle. We also set up an instructor support site with static images-based tutorials
embedded in topic pages. We tried to model the look and feel of a reMoodled site here as well.
Self
enrollment
key: ID
faculty
enrichment

55. 6 skills are required to
maintain a reMoodled site.
1. Edit/format topic pages using semantic HTML markup
2. Locate and copy permalinks from Wilson Course
Reserves
3. Add hyperlinks to topic pages
4. Add images and media to topic pages
5. Upload to/delete from course files
6. Orphaning versus hiding activities faculty
enrichment
The instructor support site teaches these six skills are required to maintain a reMoodled site and should be demonstrated or taught at the post-
reMoodle debrief meeting. These skills should be seen as in addition to other basic skills required to have a basic understanding of the Moodle
interface, as this list does not include setting up activities or adding additional resources.

56. 12 skills are required to create a
reMoodle-style site from scratch
1. Create a course banner
2. Define an information architecture
3. Delete unused topic sections
4. Turn on completion tracking
5. Add/delete side blocks
6. Create topic pages
7. Edit/format topic pages using
semantic HTML markup
8. Locate and copy permalinks to
articles created by Wilson course
reserves
9. Add hyperlinks to topic pages
10. Add images and media to topic pages
11. Upload to/delete from course files
12. Orphaning versus hiding activities
However, twelve skills are required to be able to create a new reMoodle site from scratch. The boldface type shows the additional skills required
as contrasted with the list to simply maintain a site shown in the previous slide. These skills also are included on the instructor support site.
faculty
enrichment

57. I also tried to distill the messages down to 6 heuristics and associated recommendations, which I published as a series of 2 posts on the
Techniques In Learning & Teaching blog. I also have recently shared out the framework document itself and had an instructor say she liked that
best of all, better than the TILT posts. She said she likes the framework document better because it’s less open for interpretation, she can just
use it like a checklist and it tells her exactly what to do.
Click to
view
article
Click to
view
article
faculty
enrichment

58. You are here

59. 1. Accept the things we cannot
change.
Link Followed link
We’re all at the mercy of the system and developers for improving some aspects of usability. One major limitation of Moodle is that followed
hyperlinks don’t change color as they do in most websites in the real world. Another is the absence of a search bar that can search just within a
specific Moodle course. Currently it’s possible to search forums only, but search should be possible across all activities, assignments and
resources located in a course website.
Magnifying glass by Hafizh from the Noun Project

60. 2. There’s no perfect solution
Flexible Sections format is probably the most efficient way to organize blended and fully online courses, over Topic format that uses a series of
content pages for companion sites. However, Flexible Sections format has its own issues, among them that it currently provides no visual cue to
students that content exists in collapsed sections. But we have seen throughout this presentation the value of keeping the Moodle homepage
simple, spare and scannable, so it doesn’t work to keep all the sections expanded. There are tradeoffs in terms of navigability and user
experience for any solution we choose, and we just need to iterate with our student groups what works best for them.
Current flexible
sections

61. Analytics Usability
Maintain-
ability
3. Your solution may be a
mixed bag.
Additionally, we need to work with instructors to find the solutions that works best for them while still prioritizing user experience. Like the “no
perfect solution” of the previous slide, it also is difficult to reconcile analytics and maintainability of the site with all aspects of usability.

62. Analytics Usability
Maintain-
ability
Good news in terms of maintainability: in our testing, hand-coded links in a content page DO somehow magically update when copying from
ay14 to ay15 and when copying from an ay15 to another ay15 site. We tested Assignment, External Link, Lesson, Quiz, Course files, Folder, Page,
Survey, Book, and Forum. Bad news in terms of maintainability, instructors prefer to do in-place file uploads rather than going through Course
Files, which can easily disrupt the simplicity of the design. There is a learning curve!

63. Analytics Usability
Maintain-
ability
Bad news: You lose some click data when you place all external links into a content page rather than using Add Activity or Resource > Add URL.
It’s still possible to capture this data by using the Activity and orphaning it, then hand-coding in the topic page, but then you are back to an issue
of maintainability and whether such effort makes sense for your instructor’s site.

64. Analytics Usability
Maintain-
ability
In sum, it is clear that students prefer a clean, scannable, navigable design that places their needs first. However, the reality is that instructors
are well intentioned and extremely pressed for time. Due to this tension, it looks as if pure-form reMoodled sites may be most practical for large
enrollment courses and those that can be maintained by IDs rather than instructors. Time will tell whether the method takes off and whether
we can improve our faculty enrichment efforts around the project. I’ve settled on a mantra of “progress, not perfection”. In CLA we have been
pleased with the reMoodle service largely because it has given us a chance to build new relationships with instructors, and strong relationships
are crucial for any academic technology collaboration.

65. Thank you
Ice cream by Jardson Almeida via The Noun Project
Thank you for your time and attention. Please contact me, Ann Fandrey, if you have questions or would like to work together on course website
usability initiative. fandrey@umn.edu