An early warning system was proposed at the University of Cape Town to help identify students at risk of failing. The system would need to record student performance data, flag students who meet at-risk criteria, communicate with struggling students, and generate reports. While the university's Vula (Sakai) LMS could help by tracking grades and enabling communication, it lacked features for automated risk assessment and pushing data to the student information system. The university plans to utilize the student information system for the early warning system in the first phase, while improving integration between Vula and the student information system and developing new features in Vula to better support an early warning application.

1. Early Warning System for Identifying students at risk of failing Roger Brown Center for Educational Technology University of Cape Town roger.brown@uct.ac.za

2. Overview PART 1 Why we started looking at EWS The functional requirements of the system Institutional fit Matching Vula (Sakai) affordances to the functional requirements Developments that would allow Sakai to act as an EWS (well some of them anyway!) How UCT is moving forward PART 2 - Discussion Activity and course grade Your ideas 12th Sakai Conference – Los Angeles, California – June 14-16 2

3. Part1: Introduction In 2009 Senate Re-admission Review Committee recommended that greater attention needed to be given to Faculty EWS. The SRRC report recommended that: the SRRC monitors data from the Faculty EWS with the aim of assessing and reporting on the impact mid-year exclusions have on throughput rates, and Investigate the EWS issue to assess the most effective approach to adopt a single system thereby providing consistency across faculties.

4. EWS – Functional requirements The ability to record a standard number of “in course”results/performance/grade The ability to create a current class list of all valid registered students in a course, populate this list with “in course results”, and load these to the students’ PeopleSoft record. Specification of an “at risk” threshold value for these results Recording of comments against a student Generation of communication to identified students Retentions of a permanent record of these communications Specified reporting of student performance For a student within a course across all courses For a specified cohort of students Access and authorisations to entry grades, viewing of grades and running of reports, course conveners and/or mentors and/or other “intervention” managers

5. EWS – Institutional criteria Technical Implementation Technical Integration Usability and User Support Requirements Technical Support Security and Authorisations Student Access Overall Reporting Capacity Cost (of licensing, implementation and support) Vendor and Product Sustainability

6. EWS Functional requirements and Vula (Sakai 2.7) - Integration PeopleSoft HEDA - Higher Education Data Analyzer Demographic and K12 data (currently used by IPD) IDvault

7. Vula: Affordances vs Requirements Groups: Easily created and populated Configurable Roles: Access and authority Gradebook

9. Familiar

11. Assessing students at risk? Back to My Workspace Out of Gradebook

12. Vula: Limitations (currently) “At Risk” assessment would need to be done outside Vula Not all lecturers/course convenors use Vula Vula is not the authoritative source of marks Vula does not “push” data to PS Staffing

13. Vula: R&D for EWS application Automated Gradebook export to ETL platform or preferably an internal logic Internal or external algorithm development for risk analysis Auto grouping based on risk analysis Reporting communications, display, etc Predictive logic based on previous student performance

14. How UCT is moving forward The task team recommended in phase 1 EWS to utilise the functionality of PeopleSoft (some developments required) Improve the integration of Sakai and PS gradebook export to PS (it’s easier to get grades into GB than into PS ) Samigo& Asn => GB => PS 12th Sakai Conference – Los Angeles, California – June 14-16 13

15. Part 2: Vula: Final grade vs all events (PSY1001W)

16. Your Ideas 12th Sakai Conference – Los Angeles, California – June 14-16 15 Predictive modelling GB in students’ “My workspace” ? Using ETL Communicating “failure” Learning analytics A “read only” SU role in Sakai Data mining – automating and exposing More than grades only? – is attendance a predictor?