Smudie Project Report 3The Academic ViewpointIntroductionThis report covers the outcomes of the third phase of interviews carried out as part of the JISCSwansea Metropolitan University Data Integration Exercise (Smudie) Project. The focus was primarilyon student information management by Academic Staff at the University.Three members of staff from each of the four Faculties were interviewed and the sample includedProgramme Directors, Year Tutors and Module Tutors. Further interviews were also carried out withIS staff, ADQs and Faculty Secretaries, as well as with staff in the Finance Department and theStudents Union.The main message that came from the discussions was that information management in theFaculties was focussed on day to day course management issues and only periodically on summativeassessment and statistics reporting. Communications with the students and their effective supportwas central to course team activity and for this reason local information recording systems and theMoodle online learning environment tended to play a much more prominent daily operational rolethan the institutional student records system.A further factor that emerged was the variability in delivery between the curriculum areas and theeffect that this had on student information management. It could also be seen that differences inreporting practice reflected differences in local management organisation within the Faculties. Thisranged from student information management systems organised on a school-wide basis to systemsdevised and operated by individual academics. The formal process of reporting student academicprogress, however, was periodically brought together through the entry of student assessmentoutcomes using the QLS V4 system and was reported and reviewed through the Faculty qualitysystems.Attendance monitoring was also a major component of Faculty student information managementactivity. As detailed in other reports1, this is an area where attention is needed in terms ofprocedural and operational consistency across the University. It also has implications for theinstitutional reporting requirements and the way the University is viewed by external stakeholders,particularly UKBA, SLC and the way the institutional profile is reported by HESA.This report summarises the systems in place in the Faculties for the purpose of aiding discussionsabout potential improvements. The fact that there are multiple systems within the institution isneither unusual for a large organisation nor, necessarily, ineffective. What may be undesirable is theexistence of local information management systems that are unknown generally and hence out ofinstitutional management control. The report concludes with some recommendations in this regard.1 https://www.dropbox.com/s/cwzx991qap5s23z/Student%20Attendance%20Monitoring.docx
An Overview of the SMU Student Information Management SystemThe schematic diagram below is a development of the comprehensive picture produced by JohnMillward2. It shows the range of inputs and outputs to the student information management system,both internally and externally. On the right of the diagram is a representation of all the internalinteractions and on the left are the external links. The overall message is that it is a complex systeminvolving many stakeholders.Important for the issues raised in this report are the components at the top and the bottom of thediagram. Firstly, academic staff have three different user interfaces with the student records system,which is reported as problematic. Secondly, the system communicates with other informationmanagement systems used by the University but, perhaps more importantly, does not communicatewith a range of local databases, spreadsheets and records that are also key components of theinformation management processes.2 https://www.dropbox.com/s/06uzb4tvrmna7y7/MIS%20Schematic.docx
Data entry into the system occurs through different interfaces and with different timing patternslargely aligned to the standard academic year. The timing patterns can be broadly described as: 1. Annual. Eg: UCAS applications, student enrolment, student registration, exam boards, HESA returns; 2. Periodic. Eg: student assessment recording, student attendance reporting; 3. Ongoing. Eg: student attendance monitoring, student progression monitoring, student support records.Much of the core student data is captured through the application and enrolment processes at thestart of the year using customised system interfaces/processes. This is largely managed centrally inliaison with the Faculties. Periodic and ongoing student data is managed by programme teams in theFaculties and is recorded and stored locally in paper form and, typically, using individual or sharedspreadsheets.Processes and timings vary, but assessment results will be entered at appropriate points to thecentral student record system through the V4 interface. Mid-session and end of session Facultyquality procedures ensure the consistency and accuracy of this process and, as the primaryperformance indicator for the institution derives from the outcome, the quality system that supportsit is taken very seriously.Attendance monitoring does not have the same level of quality control or consistency. Externalreporting obligations vary from the regular and detailed information about overseas studentsrequired by UKBA to the more top level annual reporting to HESA. As a consequence, thoseprogrammes with a significant proportion of overseas students have more rigorous monitoring andreporting systems than those that do not.Attendance monitoring is also used to inform the student support systems in the Faculties. Poorattendance triggers action to support students who have problems and, if not successful, leads tothe formal processes for withdrawal, transfer or suspension of studies. There is inconsistency acrossthe institution in the formality of this process and hence in the way data is captured. It should benoted however, that the interviews demonstrated that all academic staff are focussed on effectivestudent support and always seek to find the best solution when problems arise. All that differs arethe methods employed to achieve that outcome. The sharing and documenting of good practicewould be of general benefit.Information Systems in Academic Delivery and AssessmentThere are four main information systems used by academic staff at SMU as part of programmedelivery and assessment. These are: 1. The Faculty Query System (FQS): the original internally developed student record system at SMU, still used extensively by staff; 2. QLe: one of two user interfaces to the Agresso Student Management system introduced in 2010 and intended to replace FQS. This interface is read-only and provides access to student records; 3. QLS V4: the second interface to the Agresso Student Management system that academic staff use to populate the student records with assessment data;
4. Moodle: the Online Learning Environment adopted by SMU in 2011 as a replacement for the previous Blackboard system. Moodle is populated by core student data from QLe and provides teaching resources and communications functionality to support course delivery.A fifth systems component is planned for online assessment submission. The Turnitin system willreplace the current requirement for students to hand in assignments in the school office and toreceive a receipt.Summary of usage:The Faculty Query System: FQS remains in use by many academic staff to access student and cohortinformation. It was originally expected to be replaced once the Agresso QLS system was in place, butusability issues with the new system has led to its retention and updating.Key messages: FQS is used by many staff as their primary source of student information, particularly at the beginning of the year as it provides convenient access to core data including student photographs, and matrices of cohort information for checking correct registration on modules and similar details that are reported not easy to extract from QLe. FQS is populated by QLe when the students self-enrol and is used by many staff as their source of cohort information when creating class registers. Certain recent updates of FQS have not been universally welcomed. The two most quoted have been, firstly, the change from a name search to a student ID search process which staff find inconvenient because they are familiar with names but not with IDs. The second issue is with the decision to identify part-time students through the use of ‘year 9’ in the year of study data field. This means that their records do not include their actual year of study. This raises the issue of how system upgrades are agreed and implemented. The IS staff are seeking to optimise the system for the users and respond to requests for improvements. However, changes for one group of users may impact in an unforeseen way on the use by others. A review of change management processes may be needed here. Newer staff who joined SMU after the launch of QLe report using only the new system and not FQS.QLe: This read only interface provides intranet access to the QLS student records system foracademic staff. This includes application information from UCAS, student self-enrolment informationand other data needed by academics and support staff preparing for new cohort delivery.Key messages: There remains a view that FQS is a more user friendly interface to student cohort information than QLe, particularly when preparing registers and other administrative systems for course delivery. This is ironic as QLe populates FQS which effectively re-presents the same data. There also appears to be widespread confusion amongst academic staff why there are two different interfaces to QLS and that they need to be signed on to separately. The role of V4
in entering assessment data and preparing for exam boards is understood, but the purpose of a different read only interface to the same system is not. A common feature of academic programme management is that student information is managed locally most of the time and feeds into QLS only periodically. While FQS provides the day to day access to records needed by academics there is no incentive to adopt another system that is felt to provide a poorer service.QLS V4: This is the main interface used by academic staff to enter student assessment outcomes.Data is entered at the conclusion of assessment periods which are typically twice per academic yearbut vary between curriculum areas. Academic staff enter the data individually, with help from theFaculty MIO or with help from the Faculty office.Key messages: Because the system is only used by academic staff once or twice a year, they are unfamiliar with the menus and methods which make the process inefficient and frustrating (particularly part-time staff). Help from the MIOs and the Faculty office staff is often necessary and appreciated. Staff, particularly part-time staff, are not all aware of the training or support available from IS or the MIOs. Assistance is often sought from immediate colleagues. The interface is reported to be non-intuitive, but this may be due to infrequent usage. A common comment about usability, however, was that assessment data entry was frustratingly slow with a wait before the next entry could be made. This appeared to depend on the volume of system traffic at the time. Typically, academic staff keep their own spreadsheets of student assessment results as they can’t view a summary for the whole cohort from QLS. This was reported to be particularly the case when students were studying on a portfolio of modules drawn from a number of different programmes. In one curriculum area the tutor spreadsheet data was reported to be used for the external examiner pack rather than a QLS report. Multiple grade averaging and applying resit mark limits were identified as examples of procedural issues that needed to be resolved. There was a bottleneck leading up to exam boards when last minute assessment outcomes needed to be entered onto the system. This appeared to be more a management than a systems issue.Moodle: the open source Moodle VLE was launched as the institutional online learning environmentin 2011, replacing the commercial Blackboard application. Adoption by academic staff has beenpositive, not the least because of the support provided by the IS team and the e-Learning supportofficer. Moodle is populated with student information from QLS when the students have self-enrolled.Key messages: Moodle is used to communicate with students for personal and course information. Email & forums (the latter to communicate with the whole group) are used and it encourages the students to use the university email system. For a number of courses it is the main method of out of class communications and is very good for peripatetic and part-time students.
Moodle is used as teaching resource repository and to send materials to students. Student work can be submitted through Moodle. The intention is to use Turnitin for the future submission of all assignments and have receipts issued online, thus replacing the current Faculty office submission process. Moodle is also used for course announcements and students can use it to check on changes in daily course activity such a staff absence or room change for teaching sessions. It provides a calendar for course planning and can also contribute to the social aspect of cohort activity through appropriate communications. The availability of teaching resources online adds significantly to the choices available to students when they engage with their learning. Although it is not designed to replace formal taught classes, it can provide an effective backup if necessary. It is also very efficient for students during revision with all relevant resources assembled in the same place.Information Systems in Attendance Monitoring and ReportingThree reports have been produced concerning student attendance monitoring and reporting atSMU3,4,5. The overall conclusions were that student attendance monitoring was currently aninstitutionally unmanaged process and that locally managed systems were in place at Faculty,School, Programme and tutor levels. This applied at all stages in the process from the creation ofattendance registers, through the completion of attendance recording, the storing and archiving ofattendance records and the reporting of attendance to external bodies.Typically the system uses paper based recording in the classroom and usually, but not always, bystudents individually signing next to their names on a circulated list. Other methods are also used,including the tutor completing a traditional roll-call or simply noting who is present and completingthe form independently. These latter methods are only practicable for small class sizes.The system employed also depends on the curriculum area, the session format and the attendancepattern. Under any circumstances where the students in a cohort are not carrying out the sameactivity in the same place at the same time, appropriately customised recording procedures need tobe devised.The actual registers used by tutors to record attendance are created at various levels by programmedirectors, year tutors or individual module tutors depending on the local management arrangementsin place. Whoever creates the register will typically interrogate either FQS or QLe for details of theenrolled students as indicated in the diagram:3https://www.dropbox.com/s/hleeeijkgon3k3t/A%20Brief%20Report%20on%20Student%20Attendance%20Monitoring.docx4 https://www.dropbox.com/s/cwzx991qap5s23z/Student%20Attendance%20Monitoring.docx5 https://www.dropbox.com/s/zc7xufftogffquy/Update%20on%20the%20Student%20Non-attendance%20Review.docx
These paper records are then typically, but not always, transcribed onto local spreadsheets whichare themselves sometimes transcribed onto a master programme or school spreadsheet. Where thisdoes happen the spreadsheets are available as shared documents on the staff intranet and aredesigned to flag up problems which can then be acted upon.In addition to these largely similar attendance monitoring practices, there are examples of individualvariations where, for example, there are no paper records and the attendance is entered directlyonto spreadsheets in the classroom or where absence rather than attendance is recorded.
All programme teams use the attendance monitoring information, together with academic progressinformation, in their support activities for students. They follow the institutional proceduresactivated by successive non-attendance that, if not resolved, lead to the formal processes ofwithdrawal, transfer or suspension of studies.An important factor for the university, when instigating change to create a solution for problemsstudents have with their study programmes, is to demonstrate that: 1. The needs and aspirations of the students are paramount and that the institution will always look for the best solution for the student; 2. The first choice solution will always be to guide the students to more suitable programme arrangements before the less desirable options of suspension of studies or withdrawal.In that way the institution is demonstrating its commitment to finding pathways to student successand value for money both as a public sector body and, increasingly, as a student funded institution.This is important for the institutional public profile, future marketing and recruitment.
The need to establish a consistent and accurate system of attendance monitoring, particularly in thisera of increased student fees and institutional service provision accountability, is recognised by SMUmanagement. Trials have been carried out in the use of student proximity card attendance recordingand the use of CELCAT timetabling software. The conclusions from that trial showed significantpotential benefits, but that there are logistical, cost and procedural issues to be dealt with if aninstitution-wide implementation is to be successful.There is a genuine concern expressed by academic staff that a technology based solution wouldimpact on the need for flexible and adaptable curriculum delivery. This is particularly relevant asblended learning support systems evolve and student choice in engagement develops. A repeatedcomment was that they feared that an imposed IT solution might compromise academic quality andinnovation.ConclusionsThe main conclusion that can be drawn from this study is that the management of studentinformation at the university operates at different levels within the faculty structures and withdifferent levels of management planning and control.This is entirely expected as academic staff with different levels of management responsibility havedifferent roles to play in the information management process. Primary data is gathered by the module tutor about student progress and attainment and this is used for subject level student feedback and support. Local records are kept of attendance, assessment outcomes and other relevant information such as mitigating circumstances; Programme Directors and Year Tutors access summative data, typically in spreadsheet form, drawn from individual module records. Often, but not always, these are available to the course team through shared spreadsheets on the staff intranet. This facilitates team discussions about individual student progress and decisions about any actions to be taken; ADQs, MIOs and Faculty Secretaries are all involved with the programme teams ensuring that assessment outcomes are entered onto the student records system and that the information is accurate and complete. It is at this level that consistent quality procedures are assured across the university in preparation for Exam Boards.There are different categories of student data recorded that are used for different purposes bydifferent staff in the institution. These include: Core Student Data: largely drawn from the student application and enrolment process. This triggers student registration, the generation of student cards, email and Moodle accounts and the population of FQS and QLS. It is also used by Finance to set up SLC and fees arrangements, The Exams Officer and Student Support Services to make provision for special needs, the International Office to liaise with UKBA etc. Student Academic Data: used by module tutors to provide academic support; by programme teams to report on student attainment; by the quality system to judge programme and faculty academic performance and to plan quality improvement; to report externally on institutional performance.
Student Attendance Data: used by module and year tutors to monitor day to day progress and allow early identification of individual problems and take action; by the international office to report to UKBA; by finance to report to SLC; by information services to report to HESA.It is clear that such a multi-faceted, multi-purpose, multi-stakeholder system is unlikely to beadequately supported by a one size fits all management information solution. The way the system isprofiled here suggests a modular approach with shared data fields, which may well be possible usingthe Agresso QLS package with appropriate add-ons such as CelCat for timetabling and attendancemonitoring, Moodle for learner support etc.However, that is not the way it is working at the moment. It is also not obvious that the flexibilityand adaptability needed for curriculum variety and innovation would be possible using these tools. Itmay be, but we don’t really know.It is proposed, therefore, that the final phase of the Smudie project should carry out the task thatwas always in the plan: to create a ‘to be’ model for student information management. It issuggested that this should be strategic and process based rather than technology specific.The objective would be to address the inconsistencies and variable management practices identifiedin this study, whilst also specifying a system that the stakeholders at each level agree meets theirneeds. Such a specification could be applicable in the design of whatever technology system ispreferred by the newly merged institution.Tony TooleAugust 2012