This document discusses benchmarking eLearning at The University of Adelaide. It aims to identify benchmarks and methods to measure performance metrics related to the usage of the university's learning management system, MyUni. This will help inform strategic planning and allow the university to report on improvements to relevant committees. The document discusses collecting usage data from MyUni and a media platform called MyMedia. It also provides recommendations from a previous benchmarking report, such as developing guidelines for minimum MyUni usage and selecting a benchmarking partner to compare performance against peer institutions.

1. Dayle Soong, eLearning Adviser
Benchmarking eLearning

2. Benchmarking eLearning
This project will identify benchmarks and methods to measure
performance in regard to eLearning at The University of Adelaide in order
to inform planning and to report on change and improvement in the use
of MyUni to the LTSG (Learning Technologies Strategy Group), LTOG
(Learning Technologies Operations Group) and LTT (Learning
Technologies Team).
Usage
Policy Strategy
Engagement
Support

3. In the beginning…
Sub Cruce Lumen: light under the
Cross. Against the vast Australian skies
the University of Adelaide shines as a
beacon bright as the Southern Cross: it
illuminates new knowledge and is a
lodestar of enlightened learning for
South Australia and the world.
“
”

4. Internationally higher education is now
being reshaped by globalisation and the
digital revolution… (p5)

5. We will reach out to a still broader student body
• providing flexible learning and e-learning to
meet new student needs (p6)

6. students need confidence their learning will be better
supported by digital learning resources. Where high-quality
content can be effectively delivered online with
demonstrable pedagogical integrity it will be, to free staff
time for small-group discovery where the focus can shift to
learning and problem solving.

7. the University will treble its
expenditure on IT learning support and
e-learning,

8. ˃ Online learning is available in 100% of
undergraduate programs
˃ MyUni is actively used in 80% or all courses in
all faculties
˃ All lectures are recorded and available for
download
˃ We should provide an online delivery option for
all student-focused administrative and
transactional services, including learning
administration services, such as the online
submission of assignments. (from SeE)
Operational Plan

9. How will we know if we are realising the
Beacon?
What do we want to measure?
What needs to be done to improve?

11. Defined set of benchmarks and measures
Usage
• Content
• Communication and collaboration
• Assessment
• Grade and monitor progress
User Engagement
• Distinct users
• Frequency
• Duration
Support
• Requests
• Incidents
• Resolution
Strategy and
Policy
• ACODE
benchmark 1

12. Getting the data
SELECT *
FROM [Users]
WHERE [Clue] > 0
No records found.

13. What data are we getting?
MyMedia Data Description Example Record
Column A MyUni Course ID associated with MyMedia sessions 3410_PSYCHOL_1000
Column B MyUni Course Description Psychology IA
Column C Count of the available MyMedia Sessions (individual
lecture recordings and any uploaded media items)
associated with the Course ID
25
MyUni Data Description Example Record
Course ID MyUni Course ID is a combination of the Term
Code, Subject Area Code and Catalogue Number
3410_PSYCHOL_1000
Course Name The Course Name Psychology IA
CourseDesc The course description. Useful to identify the 'child'
courses in the COMBINED courses
This course, together with PSYCHOL 1001,
provides an introduction to the basic concepts
and core…
Discipline The Subject Area PSYCHOL
Unit N.A. can be looked up from Peoplesoft data
School N.A. can be looked up from Peoplesoft data
Faculty N.A. can be looked up from Peoplesoft data
Institution The University of Adelaide uofa
PS Year Year the course ran measured by years after 1980
(for example, 34 refers to courses that are ran in
year 2014)
34
Approx Level Year level of the course within its program. Derived
from first digit of Catalogue Number.
1

14. Class Size Number of students enrolled in the course 687
Available Whether the course has been made available to students (Y = Yes, N = No) Y
AllContent Count Count of content items contained in the course i.e. uploaded files or text/media items entered directly into
MyUni
230
AvailableContent Count Count of content items which are available/visible to students 219
Pdf Count Count of all PDF files which are available/visible to students 93
PowerPoint Count Count of all PowerPoint files (.ppt/.pptx) which are available/visible to students 5
Word Doc Count Count of all Word files (.doc/.docx) which are available/visible to students 1
Image Count Count of all image files (added from Build Course -> Image) which are available/visible to
students. Does not include count of images which are added as 'File' or 'Item'.
0
Audio Count Count of all audio files (added from Build Course -> Audio) which are available/visible to
students. Does not include count of audio files which are added as 'File' or 'Item'.
0
Video Count Count of all video files (added from Build Course -> Video) which are available/visible to
students. Does not include count of video files which are added as 'File' or 'Item'.
0
ExtLink Count Count of all web links (added from Build Course -> Web Link) which are available/visible to
students.
2
SCORM Count of SCORM items (learning objects produced outside of MyUni that are scorm compliant
e.g. an Articulate presentation including a quiz that will interface with GradeCenter)
2
MyMedia PlaylistCount Count of links to MyMedia playlist feeds (one feed is likely to contain many individual session
recordings, but individual sessions are not counted here)
3
MyMedia SessionCount Count of links to MyMedia sessions (added with Embed code) - this is NOT a count of the
number of sessions recorded and available for this course
14
TLE Links to Online Content items managed by the DRMC (copyright journal articles and book
chapters) Note: TLE (The Learning Edge) is now called Equella
0
Cal Count Semester Count of Calendar entries which has a start date within the current semester 0
Task Count Semester Count of Task entries which has a due date within the current semester 0

15. Avail Announcement
Count
Count of Announcement which has a start date within the current semester 28
avail forum count Count of Discussion forums which are available/visible to students 0
Forum Post Count Count of posts in the available discussion forums (does NOT include Group discussion boards) 0
Forum Auth Count Count of distinct/unique authors in the posts (does NOT include Group discussion boards) 0
Blog Count Count of Blogs which are available/visible to students. Includes both Individual and Group blogs. 0
Blog Post Count Count of posts in the available blogs 0
Journal Count Count of Journals which are available/visible to students. Includes both Individual and Group
journals.
0
Journal Post Count Count of posts in the available journals 0
Wiki Count Count of Wikis which are available/visible to students. Includes both Course and Group Wiki. 0
Collaborate Count Number of Blackboard Collaborate session that have been deployed in a course and
availalble/visible to students
0
Group Count Count of Groups 5
Group Gt2 Count Count of Groups which has at more than 2 students. 5
Voice Podcast Count Count of Voice Podcaster which are available/visible to students 0
Voice Board Count Count of Voice Board which are available/visible to students 0
Voice Pres Count Count of Voice Presentation which are available/visible to students 0
Adaptive Rel Rule CountCount of Adaptive Release Rules 0
Assessment Count Count of MyUni Tests, Surveys and Mobile Compatible Test which are deployed in a course 6
Test Count Count of MyUni Tests which are deployed in a course 6

16. Survey Count Count of Surveys which are deployed in a course 0
Mob Assign Count Count of Mobile Compatible Tests which are deployed in a course 0
MapleTa Assign Count Count of Maple T.A. Assignments which are deployed in a course 0
MyUni Assign Count Count of MyUni Assignments which are deployed in a course 6
Icc Assign Count Count of ICC Printed Assignments which are deployed in a course 0
TurnItIn Assign Count Count of Turnitin Assignments which are deployed in a course 0
Grade Centre Column
Count
Count of Grade Centre Columns 35
Grade Centre Student
Visible Count
Count of Grade Centre Columns which are visible to students 35
Early Warning Rule
Count
Count of Early Warning System rules which are available (4 are available and active by default) 4
Rubric Count Count of MyUni Rubrics (Does NOT include Turnitin Rubrics) 0
Access Count Sem Count of number of ‘hits’ or times that a page within a course was accessed over the semester 230098
User Count Sem Count of active users who have accessed the course over the semester. Includes students,
academic and professional staff.
775
sum of all assignment
types
Sum of eSubmission assignments (sum of ICC, Turnitin and MyUni Assignments) 6
sum of all test types Sum of MyUni tests and Maple TA tests 6
hits per user Access Count Sem / User Count Sem 296.9006452
MyMedia Available
Session Count
Count of the available MyMedia Sessions (individual lecture recordings and any uploaded media items)
associated with the Course ID
25

17. Current Results

23. Why?
• Some areas have other LMSs
• Some areas run courses with very low
numbers of students
• Some areas have course structures that do
not align with PeopleSoft course codes
• Some areas create courses for administration
purposes
• Some areas have practical courses
• Some areas teach in spaces without AV
• Some areas ARE SPECIAL

25. Recommendations from 2013 baseline report
1. Guidelines for minimum usage of MyUni are developed
and proactively disseminated to and applied by staff.
2. Administration processes that create MyUni courses
should be examined in order to improve the accuracy of
reporting.
3. Specific strategies should be employed in 2014 to
increase utilisation of MyMedia lecture recordings and
eSubmission of assignments to meet targets set by the
Beacon of Enlightenment and Student e-Experience
Strategy.
4. Consideration should be given to increasing support for
browsers and devices that are most commonly used by
staff and students.
5. The new Closure Code Matrix (refer to Appendix C)
should be implemented in LANDesk and utilised by the
new MyUni Support Team to provide better information
on support issues.
6. Policies are developed that specify the use of
technologies to support learning and teaching covering
all aspects and stakeholder perspectives.
7. Consideration should be given to undertaking a
comprehensive ACODE or eLearning Maturity Model
assessment in the future.
8. Combined or merged courses in MyUni should have a
clear identifier that enables the source courses and year
levels to be automatically determined.
9. Reporting on eLearning performance should be sought
from areas of the University who are using alternative
tools and systems.
10. The Benchmarking eLearning project should be re-established
in 2014 with the following objectives:
– Automation of eLearning reporting mechanisms
with the implementation of an integrated business
intelligence reporting application;
– Selection of a suitable benchmarking partner in
order to compare our performance with our peers.

26. eMM/ACODE
• eLearning Maturity Model
• Assess the institutions eLearning capability maturity using
the eMM framework
• Compare with other Australian Universities
• Provide a visual representation of strengths and gaps
• Identify strategies for improvement that will have the most
impact

27. The future… automated and systematic

28. Data Warehouse
MyUni and
MyMedia