Clas reporting nov2015

1. S
Usage Reporting in CLAS
Release Notes Nov, 2015
Generate deep student behaviour reports
per video or per course
can be anonymized for research purpose
Project Owner: Dr. Fred Cutler
Designer & architect: Thomas Dang
Arts ISIT, University of British
Columbia

2. Why?
S Monitor or evaluate the activities of students with regards to audio/video
S Evaluate the effectiveness of media content design and teaching method based on
deep data
S Make anonymous data available for learning analytics research
S All reports are formatted to be easily consumable by numerical data analysis products such
as Tableau and unstructured text analysis products such as In-Spire
S How is CLAS different from other video platforms? 35+ data types are collected.
To ensure that these data types are meaningful, our development has been
continuously guided by:
S Requests from instructors using CLAS over three years and on multiple different
educational scenarios
S Consultation with UBC Arts ISIT Learning Analytics office (special thanks to Dr. Leah
Macfadyen).

3. Types of report generated
S Comprehensive report for each student on every video per course, generated
nightly, downloadable at will. The report can be aggregated by video and
then by student, or aggregated by student over all the videos in a course
S Comprehensive report for a single video, generated at will, taking a few
seconds
S Optional anonymization: remove course name, student names, and video
title, generate anonymized video IDs and student IDs, and scan the full text
of all posts and replace student names with @userNumber
S Full text of all posts (annotations and comments) are included in all reports

4. Instructors can
download csv
reports for their
courses via the
“usage data”
side menu
button on the
video view page
This button is
simplified to
“download
annotation” in
the student
interface

5. Data dictionary
Introduction
The remaining slides are a list of all the data types (columns)
provided in the comprehensive reports.
Some of these columns are only available when the data is
aggregated per course, and some are only available when
aggregated per video.
Request for comment: if you believe that a certain data type
not yet presented here would be valuable to collect, feel free to
contact arts.helpdesk@ubc.ca!

6. Data dictionary: identification columns
Originating Host: Only available in anonymized reports, for aggregating CLAS data from multiple installations.
Instance Name: Only available in anonymized reports. As each CLAS installation has a multi-tenant architecture, the
instance_name column can be used to distinguish between these virtual instances (tenants).
Course ID: a unique number that denotes a particular course, in a particular section (taught by a particular instructor), in a
particular year and season. For example, MATH 100 201 2014W may be course ID 23.
Course Name: alpha-numeric course name without the academic year and season suffix. This column contains a
randomized code when the report is anonymized.
Academic year: which academic year and season (W for normal term, S for summer term) was this course offered? Using
the course name (or anonymized course name) and year, you can view the progression of the same course over time.
User ID: a numeric user ID. This number is positive for users with an account in CLAS, but can be negative. Negative,
random userID’s are attached to posts made by anonymous visitors if the video is shared publicly. Each specific negative
userID denotes a single session by a particular IP, making it possible to distinguish between multiple anonymous users.
Name: user’s full name, alpha-numeric. This column is removed when the report is anonymized.
Video id: alpha-numeric video ID, only available if the report is aggregated by video. If the report is anonymized, this will
be a randomized code that cannot be used to search for the video inside CLAS
Video title: alpha-numeric video title or video file name, only available if the report is aggregated by video. This column is
removed in anonymized reports.

7. Data dictionary: Columns describing the inherent
characteristics of the course, video, or user
Subject: 3, 4 letter subject code extracted from the course name, such as MATH or POLI (political science). “N/A” if
cannot be extracted
Year Lv: an estimate of the year level of the course, based on the course number. This is usually but not always accurate
and is also dependent on how each university name their courses. It will be “N/A” if the year level cannot be extracted
from the course code or if there is no course code (i.e. a video collection with a bespoke name).
Is annotating available: is the annotation feature available for this course, “YES” or “NO”?
Is commenting available: is general commenting available for this course, “YES” or “NO”? If both annotating and
commenting are disabled, then this course utilizes CLAS simply as a video library and player.
Role: role of this user in this course; can be “student” or “instructor/TA.”
User’s own video: Is this video about this user, "YES” or “NO”? This is meaningful for courses where the videos are
students’ assignments. Some instructors have students filming and uploading the videos themselves, making the statistics
collection simple, but it can be logistically more convenient to film at a central location and have the videos uploaded by
instructors/TAs, a heuristics is used to identify if this is “about” a student. A video is deemed “about” a student if either
that video is viewable only in a private group (handin-box) belonging to this student, or the full name of this student
appears in the title or description fields.
Shared to: Who was the video shared too, aka. Who can view this video? Can be "everyone (in this course)", "multi-user
group”, "a single student”, or “the general public”

8. Data dictionary
Usage Metrics Columns (1/3)
Num logins: number of times this student has logged in during the entire duration of this course. Note that if a student
uses CLAS for more than one courses in the same time period, then the login number is shared between the two courses.
Total posts authored: total number of public posts by this user in this particular video (posts can be annotations or
comments). This value is “N/A” if BOTH annotations and comments are turned off for this course, and CLAS is used
only as a video player/library.
Annotations authored: total annotations. Annotations are text notes, links, or video clips attached to a specific point on
the video timeline. Annotations can be replied to, creating a thread at a specific point on the video. This value is “N/A” if
annotations are turned off for this course, so that students can only makes general comments.
Aggregated annot content: Text content of all annotations by this user in this video. Note the time stamps in [ ] in front of
each annotation. This value is “N/A” if annotations are turned off.
Comments authored: comments are posts that are not anchored onto the timeline, but just refer to the video as a whole.
This value is “N/A” if comments are turned off for this course.
Aggregated comment content: all comment content by this user in this video, concatenated. This value is “N/A” if
comments are turned off.

9. Data dictionary
Usage Metrics Columns (2/3)
Total public posts length: word count of all public posts (annotations & comments) that this user makes in this video.
Note that the word count does not include the timestamps in the brackets. “N/A” if both annotations and comments are
turned off.
Avg public post length: average word count of all public posts. “N/A” if both annotations and comments are turned off.
Peer referrals: number of time this user, in this video, refer to another user either by Replying to a post, or writing @user.
“N/A” if both annotations and comments are turned off.
Private notes authored: number of private notes (visible ONLY to the author or another user which the author referred
to). Full text of private notes are not provided in the dataset. “N/A” if both annotations and comments are turned off.
Total private notes length: word count of all private notes by this user, in this particular video in this course. “N/A” if
both annotations and comments are turned off.
Avg private note length: average word count of private notes. “N/A” if both annotations and comments are turned off.
Num posts read: Number of posts, both annotations and comments, in this video that this user has read. “N/A” if both
annotations and comments are turned off.
An annotation is considered read when its hover preview or editing panel is displayed for at least 3 seconds. A
comment is considered read when the user scrolls over it slowly. Fast scrolling or mouse moving pass a comment will
not mark that comment as read.

10. Data dictionary
Usage Metrics Columns (3/3)
Video views from start: view events triggered by pressing PLAY at 0
Video seeks from middle: view events triggered by seeking back and forth in the middle of a video
Total video views: total number of view events: a sum of "video views from start" and "video seeks from middle”
Average duration per view: how many seconds on average does this user let this video play whenever he/she presses
PLAY or SEEK
Total video duration: the total length of this particular video, in seconds
Percentage coverage: percentage of this particular video has the student viewed? (in the “aggregate by video” report only)
Average coverage: percentage view coverage on average across all the videos in this course (for the “aggregate by course”
report only)
Num videos viewed: number of videos this user viewed
Num videos can view: number of videos in this course that this user CAN view. For courses where the videos are lectures
and other resources shared to the whole class, this number will be the same as “Total videos in course”, but some course
uses videos for group or personal activities, and it would be more fair to compare the number of videos a student viewed
with what they actually could view
Total videos in course: number of videos in this course in total

11. Caveats
S Some instructors deleted all the videos from their course at the end of each term for
confidentiality, or cleaned out all the annotations and moved the same videos to a new term.
This will make retroactive report generation impossible.
S Support team should no longer use the “clean annotations” feature in the admin tool to reuse
videos for each new term, unless absolutely must.
S Any videos that were uploaded to CLAS can be duplicated with the duplication tool in the
video management page. Only external URL-imports like Youtube, Mediasite, Dropbox,
etc. videos cannot be duplicated (yet).
S Even better, the “archive old course and recreate for new term” tool will automatically
duplicate all non-private videos into the new term in one click. This is the recommended
method of life cycle management for CLAS courses.