Jisc smudie project report 2


Published on

JISC project

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Jisc smudie project report 2

  1. 1. Smudie Project ReportSynthesis of MessagesThis report covers the outcomes of the second phase of interviews carried out as part of the JISCSmudie Project. The focus was on the Faculty Management Information Officers and on the AssistantDeans with responsibility for Quality in the Faculties. Further interviews were carried out with IS andRegistry staff, as well as with two of the Faculty Secretaries, with the Examinations and AssessmentManager and with the Head of Library and Learning Services.1. Management Information OfficersThe role of the MIO is to oversee the completion and accuracy of student records managed by theFaculties. They are central to the adoption of QLS V4 by the Faculties and the direct entry of studentassessment information by academic staff. In principle they have no direct input of primaryinformation, but check that the students are registered on the correct modules and that assessmentoutcomes are entered and up to date. Having said that, where changes in marks, module transfersand student withdrawals are to be made, only the MIOs are authorised to make those changes.Periodic checks are made (twice or three times per year) of validity of module registration,assessment outcome entries, student transfers and withdrawals. These are done through the FQSsystem which generates programme matrices and are carried out directly with the year tutors.The management of the system is geared towards the exam boards at the end of the year when theQLS generates comprehensive spreadsheets of cohort outcomes. The outputs from the systeminclude comment sheets that record the exam board actions.Issues: Academic staff are not all engaging with the QLS V4 records system for a number of reasons: o there is reluctance/resistance to change to a new system; o there remains a proportion of staff who lack confidence/competence in the use of technology; o the task is seen as administrative as opposed to academic and regarded as a waste of their time/expertise; o aspects of the system user interface are regarded as non-intuitive/tedious; o certain course management information cannot easily be extracted from the system and staff keep their own local records; o Part-time staff do not use the system regularly and forget processes and procedures and need to be re-trained. The MIO management of the student record, where modules and grades need to be edited, is non-user friendly, time consuming and frustrating; Where resit marks are to be entered or students are transferring from one set of modules to another, this can be only be done by individually deleting the previous records; The system times out when not in continuous use and this can be very frustrating; The different ways of reporting marks and grades need reconciling.
  2. 2. Overall comment about MIOs: the system, as a bridge between registry, quality and the faculties isseen by all stakeholders as a successful arrangement that adds significant confidence to the validityof quality processes, whilst assisting academic staff in their reporting duties.The regular MIO group meetings are very beneficial for those lesser experienced than the others.The group shares experiences and identifies the differences in procedures applied within differentfaculties.A key activity for MIOs is working with academic staff to ensure that the QLS V4 student informationis complete and accurate. This includes checking that students are enrolled on the correct modules,that their assessment records are up to date and that staff have sufficient familiarity with the systemto reliably input data.2. Assistant Deans (Quality)The ADQs are responsible for all aspects of academic quality management in the Faculties. Thisincludes arrangements for new course proposals and validations, exam arrangements, exam boards,external examiners and the management of all associated information. It also includes responsibilityfor managing relationships with external agencies including QAA, Estyn, HEFCW and the University ofWales. The ADQs chair their Faculty Academic Quality and Standards Committee, attend the otherFAQSCs, and report to the main AQSC.Overall the perception is that the online student information management system is a significantimprovement and that its effectiveness will improve as academic staff become more familiar withits use. There is also a general agreement that the role of the MIOs in the system is central to theeffective maintenance of data accuracy and consistency.However, the discussions with the ADQs revealed a number of data management issues that need tobe considered. These issues relate to four different aspects of the system: 1. The ease of use of the online student records system; 2. The staff development needs of academic staff responsible for managing student information on the system; 3. The assumptions in the system structure based on a standard academic year and student cohort; 4. The fact that, currently, there are four different components of the student information management system which have yet to be fully integrated: a. QLS V4 b. FQS c. ACMS d. Student Attendance RecordingADQs do not input data to the student information system in that role, only as teachers in theirsubject specialisms. They do, however, retrieve data from the system for exam boards, annualcourse monitoring and similar quality assurance activities. The specific issues identified were: The identification of all part time students as year 9 on FQS – this prevents tracking of part time student progress;
  3. 3. The Agresso system is set up for a standard academic year which means that any course that does not follow that pattern requires students to re-enrol if their course straddles the normal entry and exit points. This is reported as particularly frustrating for overseas students; Students on courses that share modules with other courses are difficult to track. Currently, it is difficult to obtain an individual student record if they are in multiple groups; Attendance recording is manually input and very time consuming. The CelCat application was recommended by one ADQ and it is known that options are currently being discussed; The FQS system, developed by SMU data services and in use for over 10 years, is regarded as much easier to use than QLS and has been retained as a result. The matrix printouts are used for periodic checking of student module choice and accuracy of data; The course coding system is regarded as logical, but a barrier if academic staff (particularly part-time staff) do not know the codes. This could be linked to the mode of attendance issue, by including this information in the course code.An interesting issue was raised regarding disability access to the Agresso system: that the on-screencolours used were problematic for users with impaired vision and that the company should bealerted to the issue.This series of discussions was the first with academic staff at the University and demonstrated thecontext difference between them and non-academic support staff. Quite understandably, academicstaff are focussed on the support of learners and the quality of the learning experience in theirparticular curriculum area. They recognise the need for the accurate recording of studentinformation and their role and responsibilities in that process. However it is seen as a ‘necessarychore’ and separate to their primary role of teaching. For that reason they are looking for a systemthat is simple, intuitive and time efficient.3. Information SystemsEach member of the IS staff has a particular role in the development and maintenance of thestudent information management system. The roles are based on experience, technical expertiseand institutional need in terms of, for example, database support, systems development, usersupport etc. This is not at all unusual in organisations, but there is a risk of critical capacity loss ifindividuals are the sole repository of expertise in a particular systems function. This is no doubtbeing managed through appropriate duplication/documentation and, where appropriate, successionplanning.The IS team works directly with academic, registry and other staff with responsibility for using thestudent information management systems. This ranges from basic systems training and user supportfor activities such as student enrolment; through to workbench development to improve the userinterfaces with the QLS application. There are several areas where further improvements in usability,efficiency and effectiveness of the systems have been identified. All of these are known by the ISteam and the staff are working towards solutions within the constraints of team capacity.This includes working with Unit 4 to develop and improve the Agresso QLS and CMS applications.There are several aspects of the systems that have potential for improvement, particularly withregard to intuitive and efficient usability. The company is working with SMU, as well as other
  4. 4. institutional users, on a programme of continuous improvement; but this does give the impressionthat the application is in permanent beta mode and that the institutions are de facto test beds. Nodoubt this symbiotic relationship is reflected in the license fee negotiations.The overall strategy for information systems management centres on user responsibility. Wheredata is required by the system from a user then it is the user that enters the data. The students self-enrol, the academic staff enter the assessment results, the programme leader completes the CMScourse specification. The fact that the system is entirely online makes the process particularlyefficient and accessible. The effect is to eliminate the inefficiencies and cost of duplicated datarecording systems. The risk is that, with so many users having an input to the system, data reliabilitywill be compromised.Such a risk is addressed with appropriate checks and balances. The MIO arrangements are central tothis, particularly when it comes to ensuring that students are enrolled on the correct modules andthat assessment results have been entered by staff. The on-demand support from the IS team for allsystem users is also key to successful operation. This support is available to academic, registry andother staff as required and also to students, particularly when they self-enrol.The original Faculty Query System (FQS) that was used before the adoption of the Agresso QLS, isexpected to be abandoned once the new system is fully functional and stable. This has not happenedbecause there is certain FQS functionality that is valued by staff and cannot yet be delivered by QLS.An example is the cohort matrices provided by FQS that are used periodically to check that studentsare enrolled on the correct modules and that all information is complete. There are also varyingdegrees of use by staff of local record keeping, both paper based and digital, mainly for personalassurance that a backup exists should there be a systems problem or a query about data validity.Such local records are also used for QLS held student data that is regarded as difficult to access.The main issues and areas for development currently being addressed by the IS team include: Workbench development for the retrieval of information from QLS including the use of the Crystal reporting system. Current workbenches include interaction with UCAS, SLC, HESA and UKBA; Development and testing of the Agresso Curriculum Mangement System (ACMS) for curriculum design and validation. The definitive documents delivered would interface with QLS; Planning for an improved student attendance monitoring and recording system, possibly using the CelCat application. The most efficient and reliable method of recording attendance data is the present focus. Planning for the online enrolment of part-time and international studentsOther tasks involving student information management include the provision of exam registers, thesetting up of assessment parameters and the merging of student information when duplicaterecords have been entered on the system.
  5. 5. 4. Faculty SecretariesThe Faculty secretaries manage the Faculty office(s) and typically have administrative assistance.They do not have a direct data input role to the student record system but do retrieve informationfrom it in their daily activities.Interaction with students, and hence involvement in the management of student informationincludes: Receiving assignments submitted by students, issuing of receipts, logging of submissions (typically on a local spreadsheet record) and the distribution of assignments to the appropriate tutors; Receiving student application forms from Registry, distribution in paper form to the programme leaders and emailing students to confirm receipt of their application; Management of the Faculty student file system with paper records for each student that contain all relevant documents including letters, medical records, disciplinary records, withdrawal, transfer and suspension of studies forms etc; Sending of referral letters to students when (1) the need is indicated by the QLS V4 system and (2) after checking with the course tutor. Record of letters sent are typically kept on local spreadsheet; Assistance in student attendance monitoring, typically by receiving monitoring forms from lecturers and entering on a master attendance register. Registers are to be handed in to Faculty Secretaries at the end of the academic year and kept for a minimum of two years; Assistance in Exam Board preparation in collaboration with the Faculty MIO. This includes the preparation of results to be presented to the Board and the recording of meeting notes (one Faculty secretary also provides meeting minutes); Responding to student enquiries about such things as open days and UCAS applications, and the provision of general advice and guidance;The general issues relating to the student information management system identified by the Facultysecretaries are similar to those raised by the other stakeholders: The tedious multiple window operation of the V4 system when accessing data, and the fact that it times out when left for a few minutes; Changes in FQS (which is generally regarded as a useful application) that have altered its functionality. These have included the misleading use of the ‘year 9’ identifier for part-time courses and the fact that it no longer remembers where the user was when they last accessed the system;Suggestions that would improve the way Faculty secretaries manage student information includedthe establishment of a centralised student attendance monitoring/recording system and the auto-population of student addresses on letters to be sent (though I suspect this can already be done).5. Library and Examination Management
  6. 6. The library system at SMU uses the Capita/Talis application and student core data is included whenthe students enrol and receive a library account. The system records all details of library borrowingsand usage and communicates with students by email about overdue loans and reservations.The Examinations and Assessment Manager accesses the student information system to identifystudents with special needs in order to make provision for those needs. A local spreadsheet iscreated which contains all the information needed to cater for the special needs students currentlyenrolled (around 160 students in 2011-2012). The Examinations Manager liaises with studentservices who determine the kind of support needed and appropriate provision is made. All studentswith special needs are then informed of the arrangements for their particular requirements.The Examinations and Assessment Manager coordinates exam timetabling and organises thetransfer of exam board outcomes to the University of Wales for the issuing of awards.6. Summary, Conclusions and PlanningThis phase of the project involved evaluating the roles of staff who have a significant impact on thestudent information management system at SMU, but who do not have primary responsibility forentering core student data. The roles of the MIOs, the ADQs and the IS staff are mainly to ensurethat the recording system is fit for purpose; that the data being entered is accurate and complete;that access to information meets the needs of the university systems; and that staff and studentswho are responsible for entering primary data are adequately supported.A key message from this exercise, which supports the conclusion drawn in phase one, is that thestrategy of developing an online student information management system where primary data isentered by the academic staff and students who have ownership of that data, is entirely logical andachievable when supported in this way.Having said that, the system is still in development both in terms of its implementation at SMU andas a fully functional application from Unit4. This report has identified a number of areas forimprovement that would make the system more usable, efficient and effective.It should be said that the SMU IS team are aware of all the issues and are working with staff at theUniversity and at Unit4 to create solutions. However, it is hoped that the report will help inprioritising the actions to be taken in moving to fully functional and integrated student informationsystem.The main issues for consideration include: The usability of the QLS V4 application. There has been a consistent message from all users that navigation and data editing in the system is frustratingly inefficient. All users have also commented negatively on the time-out feature. Recommendation: discussions should continue with Unit4 about system usability with specific reference to these user comments. A more detailed analysis of the editing functionality should be carried out and improvement proposals documented. A time-out option that allows a direct return to the previous window would seem an obvious development.
  7. 7. The continued use of the FQS application. The stated reason for this is that it is perceived to provide a service that QLS V4 does not; particularly the periodic production of cohort matrices for quality checking. Recommendation: a detailed analysis be carried out of the ability for QLS V4 to deliver the same services as FQS (which I expect has already been carried out). Based on the outcomes, one of two options can be considered; (1) the QLS V4 functionality be adopted/developed to deliver the same services, or (2) FQS be developed with an interface to QLS (workbench?) so that it becomes an integrated sub-system using the same data. A further recommendation is that key system users are included in the development planning communications loop so they have an opportunity to comment on the impact of any changes on their interaction with the system. The alignment of the system with the standard academic year. If SMU plans to become an agile responsive HE institution with flexible attendance patterns, then the student information management system needs to reflect that. The present system is reported to be aligned to the traditional academic year for full-time students. Recommendation: the analysis of QLS V4 suggested above should include a consideration of its capacity to support a flexible non-traditional curriculum structure. It seems unlikely that a major vendor would not be able to deliver such flexibility. The student attendance monitoring system. It is known that this has been a matter of debate for some time. Different Faculties (and even programmes) manage it in different ways and a common system that is time efficient would clearly be of benefit. Recommendation: That the discussions already underway come to a speedy conclusion and that they include consultation with the lecturers and students who will be responsible for its implementation.Phase three of the Smudie project will continue with the final stakeholder consultations: courseadmissions tutors, lecturers and the students themselves. Where necessary, repeat discussions withpreviously interviewed stakeholders will be undertaken to clarify aspects of the informationgathered.At the end of the phase the project will have been in operation for 6 months and will be deliveringan ‘as is’ model of the student information management system, together with a consolidated set ofdevelopment recommendations based on this information.This draft report will be presented at the Smudie project steering group meeting at SwanseaMetropolitan University on Friday 30th March 2012.Tony TooleMarch 2012