The document summarizes research and design work done to improve collaboration tools in the Canvas learning management system. It describes weaknesses identified in Canvas' existing discussion forums and group features. It then outlines the conception, research, design process and prototype testing done to develop new discussion, file sharing and project management tools to better support collaboration. User feedback from students and instructors was incorporated into iterative design improvements.

1. Collaboration in CanvasPatricia Gates, Yanwen Guo, Amy Maxwell, & Elena Winzeler

2. Agenda
● Introducing Canvas
● Conception and Research Phase
● Design Process
● High-fidelity Prototype Demonstration
● Design Cases & User Feedback
● Conclusion
● Q&A
2

3. Collaboration in Higher Education
● Web 2.0 tools enable students to connect virtually and
collaborate
○ Threaded discussion boards, Google Docs, chat,
video conferencing, task managers, etc.
● Learning Management Systems (LMS) allow instructors
to organize class materials and assignments
○ Canvas (used by UT), Blackboard, Desire2Learn, etc.
3

4. Weaknesses of Canvas: Groups
4
Lacks tools
students need
Confusingly
similar to
course pages

5. Weaknesses of Canvas: Discussions
5
Threaded
discussions
difficult to sift
through
Hides
connections
between ideas

6. Conception and Research Phase
6
Product Research
● learning management systems
● threaded discussion forums
● resource sharing and collaboration tools
User Research
● Instructor Interview
● Student Survey
● Instructor Survey
Goals

7. Product Research
What improves
participation outcomes?
● autonomy - quantity and
quality of participation
● connectedness - quality of
contributions and pro-social
behaviors
● meaning - outcomes in
participation, quality, and
student experience
7
Learning management systems
Popular collaboration tools

8. Student Survey
o discussion forum not satisfying
o commonly use outside tools for
group-work
● Most requested features
o tools for synchronous
collaboration
o co-authoring of documents
o file sharing
o tool tips
● Experiences with Canvas and collaboration
8

9. Instructor Interview and Survey
● Experiences with Canvas
○ discussion forum organization not easy to follow
○ existing groups feature inadequate
● Most Requested Features
○ file sharing by students
○ analytics of discussion board activity
● Top Priorities
○ greater ability to monitor activity and respond to students’ needs
○ meaningful discussions and student experiences
9

10. Project Conception
● Redesign of the current collaboration tools offered by
Canvas
● Our redesign will better address the needs of students,
professors, and teaching assistants
○ Richer interaction among students
○ More convenient to use than current collaboration tools
10

11. Design Requirements & Features
11
better organized discussions
conversation themes and threads easily visible
project management made easier
file sharing by students facilitated
topic-based discussion board
thematic labels - #tag
text-focused reply - “quote”
comprehensive group
management + workflow tool
analytics of discussion board activity
monitor group project process
individual and class activity
reporting
group monitoring tools
file sharing integrated into
groups pages

12. Design Process
12
paper-based
student view-
list features for
students
paper-based
solve the
problems got
from user test
teacher view-
design features
for teachers
fixed the
problem found
in user test 3
5 students
2 TAs
Low-fi 1 User Test 1 Low-fi 2 High-fi 1 High-fi 2User Test 2 User Test 3
6 students
2 TAs
2 Instructors
fixed problems
in low-fi 2
make a more
interactive
demo with
axure
make two
videos
7 students
1 Instructor

13. High-fidelity Prototype Demonstration
13
Discussion Forum Groups Pages

14. Design Cases: Editor
14
Streamlined
publishing process
Word limit
unpopular
Chunking
process a
hassle
Low-fi 1 High-fi 1

15. Design Cases: Workflow Tool
15
Not helpful for task
management
Low-fi 1 Low-fi 2
High-fi 2 High-fi 1
More details about tasks
Stage and task
status shown

16. Design Cases: Popular Tags
16
Low-fi 1 Low-fi 2
confusing
High-fi 2
positive
feedback

17. Student Feedback
17
“I think tags would be useful for me. I always want to find people
who wrote about similar topics, and tags makes that easy to do.”
“I like the tag. It just gives a new way to read posts and it’s more
convenient.”
“I think I will use quote to reply others, but I don’t think I will use tags as
much. I am too lazy to do more than is required by the teacher.”
“I like the workflow, because it help me keep track of what I have
to do in the whole project process. But I also hope the instructor
will create the stages and tasks so I won’t have to.”

18. Instructor Feedback
18
“Please develop this so I can use it next
semester!”

19. Concluding Thoughts
● Challenges
o product involves extended, complex interactions
with “expert” users
o multiple stakeholders with different needs
● Takeaways
o investment in product and user research pays off
o core design principles provide guidance at every
step of the process
19

20. 20
Questions?

21. 21
Thank you!

22. 22
Appendix

23. Background
● Collaborative learning has been proven more effective
than independent and competitive learning (Johnson,
Johnson, & Smith, 1998; Qin, Johnson & Johnson, 1995; Slavin, 1991).
● Constructivism-based pedagogies center the locus
for learning in the individual. Knowledge is generated by
the learner through experience and interaction with the
content and with others, not transmitted by the teacher.
● Knowledge-building communities have as their
primary goal knowledge creation.
23

24. Problems in Current System
● The current educational model - even in higher ed. - places a heavy
responsibility on teachers to be the gatekeepers of knowledge, while
students wait to passively receive enlightenment.
● Videos of cats go viral, while rarely do students engage with material
related to their classes outside of assigned readings. Even more rarely do
their share what they find with their peers.
● Learning management systems already exist as a platform for student-
professor and student-student communication. However, these interactions
are limited and often depend on the professor’s initiative in integrating the
tool into the course.
24

25. Low-Fidelity Prototype 1
(Student View only)
● Discussion Boards
o meaning-based organization → topical Boards
o user-generated semantic links → Tags
o reply to statements rather than people → Quoting
● Groups
o extended, text-based conversations → Discussions
o sharing documents → Files
o time-sensitive messages → Announcements
o project management → Workflow

26. User Testing Round 1
● Subjects
o 5 students
o 2 TAs
● Protocol
o Introduction
o Scenario-based tasks
o Targeted questions at key decision points
o Open-ended questions at the end

27. User Testing Round 1 Results
● Discussion Board
o ‘Themes’ unclear
o ‘New Post’ button difficult to find
● Editor
o ‘#’ symbol desired
o word limit unpopular
o more space desired
● Groups
o Workflow page unclear
o management of Files and Google Docs unclear
o Conversations tool desired to be accessible at all times

28. Low-Fidelity Prototype 2
(Student View)
● Discussion Board
o ‘Themes’ changed to ‘Popular Tags’ and made only visible on
Discussions homepage
o ‘New Post’ button moved
● Editor
o ‘#’ symbol added
● Groups
o Single group homepage modified to increase clarity
o Workflow and Files pages modified
o Conversations tool made accessible on all pages

29. Low-Fidelity Prototype 2
(Teacher View)
● Discussion Board
o Student Activity page → data on individual activity, including boards
visited, posts read, and posts authored
o Class Activity page → network visualization of class activity according
to posts read and responses (Quoting)
● Groups
o Group homepage → data on activity within the group over time
displayed in a line graph (activity based on the number of posts to the
group discussion board and number of visits to the group page
o Individual Group homepages → data on activity within the group over
time will be displayed with a larger version of the line graph, along with
a breakdown of activity (overall activity and activity over the previous
week)

30. User Testing Round 2
● Subjects
o 6 students and 1 TA so far
o 2 students, 1 TA, and 2 teachers scheduled for this week
● Protocol
o Students
 Tasks targeted to unresolved questions and changes made
o Teachers/TAs
 Student View
 Teacher View