Good Practice in ValidationProcesses: Is going online the way forwardMatthew Watson Cathy ShawQuality Officer (Validation & Review) Assistant Registrar (Quality Management)Quality and Academic Partnerships Academic Standards & Quality UnitUniversity of Northampton University of Wolverhampton
Background to idea for this session National conference on ‘Good Practice in Validations’ held at the University of Northampton in May 2011 The introduction of QAA Institutional Review and the QAA UK Quality Code The funding regime – a much less benign environment!
Outline of sessionDefinition of validation for the purposes of the session(QAA definition & requirements of the UK Quality Code)1. Validation processes at UoN2. Validation processes at UoW3. Debate: Pros & Cons of each methodology against 6 key principles4. Conclusions – Is going on-line the way forward?
Validation Processes at the University of NorthamptonThe more traditional style validation
Key Elements to validation process (UoN) All validations, changes of approval and Periodic Subject Reviews are managed centrally by Quality & Academic Partnerships (QAP). Validations are based on the element of risk – the higher the perceived risk the more ‘complete’ the validation: i.e. a new programme with more than 50% new modules is considered ‘high risk’ so there is a full validation with externals in attendance. All Programme Specifications, Module Specifications and Award Maps are produced, maintained and provided by a central ‘Curriculum Team’ which is part of QAP. Prior to the ‘real’ validation event, documentation is signed-off by the proposing School at an ‘Internal School Meeting’. The process is based on the end point meeting giving approval (or not) Validation reports are audited by central committees.
DocumentationAll validation Panels are provided with: Curriculum documents: Programme Specifications, Module Specifications and Award Maps Rationale document from the Programme Team outlining the ‘whys, whens and hows’ of the programme Draft Student Handbook The CVs of the Programme Team Memorandum of Co-operation (if collaborative) Supporting Statement from Library & Learning Services Indicative reading lists Panel Briefing Note from the Officer (usually from QAP)
Roles within Curriculum DesignProgramme Teams normally work with the following during the curriculumdesign process: Academic Advisor (one in each School): an academic with expertise in the University Modular Framework Curriculum Team representative: will produce, process and provide the Programme Specifications, Module Specifications and Award Maps from the Academic Database System (ADS) Deputy Dean in the School (normally has the remit for Quality Assurance) Officer from QAP: will provide procedural advice External examiners (in some cases)
Timelines (a typical example)For a normal high risk validation event (involving a full Panel): September: Development Approval Form approved by University Executive Team and goes onto validation schedule October: QAP organises an initial information meeting with the Programme Team October to December: Programme Team working on validation documentation January: Internal School Meeting – documentation signed-off Early February: Final documentation sent out to validation Panel Early March: Validation event held Late April: Response to conditions submitted May: Panel signs off response to conditionsBut it can be done a lot quicker (or slower) than this! The School decides…
Validation Processes at theUniversity of Wolverhampton The Virtual Validation
Key Elements to validation process (UoW) Validation embedded within curriculum development No end point validation meeting and introduce a process for validation through development stages External Advisers to work with Teams in development Involvement of other University Departments at the earliest stages in curriculum development Only key documents required – Module and Course Specifications – available only on-line / electronically All documents presented to be written for students not for validation so information can be reused Validation / approval granted when all members of the validation panel are clear that all requirements / actions have been fulfilled Validation reports are audited by central committees
Roles within Curriculum Design External Adviser Central Registry ASQ Officer Services School Proposing Institute forManagement Team Curriculum Team Learning(Associate Dean of Enhancement (ILE) School) Learning Validation Chair & Information Panel members Services (LIS)
Documentation Tools used for viewing on-line documents and discussion Submission of drafts on e:Vision and WOLF Modules on e:Vision, other documents on WOLF All documentation is written for a student audience, not for validation. CVs required if new area for the University If collaborative provision include CVs of staff based with collaborative partner plus MoC and Operations Manual Academic Planning documentation can be made available for reference, this includes market demand data, provision of resources etc.
Timelines (a typical example) University level academic planning approval On approval: ASQ Officer set up IPM with Chair and Developmental Lead Deadlines set for: Receipt of nomination for External Adviser Date for visit from External Adviser (recommend 1st draft received) Process points for receipt of draft documents Process points for feedback from Validation Panel Discuss a possible completion date for full approvalBut it can be done quickly (or slowly) - the School decides…
Accountability Who has overall accountability for a validation process: the Panel Chair, the QA Officer, external or internal Panel members, the Programme/Course Team? In an online validation approach will the roles, responsibilities and levels of accountability change? How are issues resolved (does it differ between online and ‘traditional’ approaches?) Is the Programme/Curriculum Team more or less accountable in an online approach? What is the output from the validation - a detailed report or a confirmation that the process is complete? Does it matter? “The administrative tail is wagging the academic dog!”
Peer Review, Externality & Constructive Exchange To what extent is the input adding to the process and to the end product? Which approach (online or ‘traditional’) offers the better opportunity for constructive exchanges? To what extent do the two approaches involve student input? How do the approaches facilitate a truly two-way conversation? How do the approaches impact on the nature and clarity of language used? In an online validation approach do the externals have a formative rather than a summative role (or both)? ““Validations are always so confrontational!”
Timeliness, Efficiency and Economies Robustness versus responsiveness: in the two validation approaches is it possible to have both or is one at the expense of the other? So is online really quicker, more efficient and cheaper? How does the role of the QA administrator/officer change in an online approach (more tasks or less or just different)? Does an online approach to validations increase capacity – even more validations possible! Is this a good thing? Which process allows Programme/Course Teams to explore new ideas and to make changes to their product? Does an online approach enable greater input from students and other stakeholders or is this unnecessarily slowing things down? “Validation slows everything up!”
Documentation What is the purpose of the validation documentation (is it just for the validation itself or beyond)? Does one approach generate more paperwork than the other? Should validation documentation be just for approval purposes? Or does it need to be student friendly? What do validation Panels really need to see? Which validation approach creates a more appropriate QAA audit trail? Is this what we should be aiming for? Is the ‘richness’ of detail and contextual information lost in the online validation approach? “I had to produce so much documentation and it is only a minor change!”
Curriculum Design, Approval and Sustainability Who should be involved in the curriculum design process? Consultation with external Panel members in an online validation process – is it at an appropriate level? How can the QA administrator/officer most effectively manage all the contributions to curriculum design in an online validation process? How do the two validation approaches ensure both the fitness for purpose of the new programme as well as its long term sustainability? At what point in the validation process should the Programme/Course Leader have the final say? “We have 50 students all waiting to take this new programme – honestly!”
Quality of Product & Academic Standards Does the validation process add to the quality of the final product in terms of Learning, Teaching and Assessment Strategies? Or is the validation process principally one of ‘signing off’ paperwork? To what extent do the two validation approaches emphasise that validation is simply a ‘first stage’ of the Quality Assurance process? How does the validation process ensure the production of high quality public information? To what extent do the two validation approaches help to ensure the reputation of the University? Does either approach better highlight the role of the QA administrator/officer as a facilitator in the validation process (i.e. more than just a minute taker!) “Have you got everything minuted? Is there tea and coffee booked & are we having lunch?”
Accountability Regardless of type of process accountability lies with the Panel Chair supported by the Quality Officer/Administrator. In return the Panel Chair and QualityColleagues, it is 4.45 and Officer/Administrator must have a a decision needs to be process that supports them. made…
Peer Review, Externality & Constructive Exchange Constructive but critical peer review (both external and internal) has to be at the core of any process and be received at a point in that process where it can have an impact and bestI don’t like blogging because I don’t want to appear ensure the highest quality product for negative. students.
Timeliness, Efficiency and Economies In a perfect world we could define a standard timeframe for validation. However, curriculum design requires thought, research, consultation and evaluation. Institutions do notIt took six weeks just to get an assessment employ staff as just curriculum changed. designers.
Documentation Documentation should be appropriate for both student and validation processes so as to avoid unnecessary duplication. Documentation should not beWe will validate what you wasted on validation. are going to deliver nowyou deliver what we have validated.
Curriculum Design, Approval and Sustainability Curriculum design and approval are more effective as an integrated process, allowing wider stakeholder Everyone we spoke to before the validation involvement with no ‘nasty surprises’said it was fine, yet why at the end. have we been set conditions?
Quality of Product & Academic Standards Our validation processes must be answerable to external quality agencies and this is unavoidable. Should this stop us from doing things Will this stand up to differently?muster when the QAA audits us?