Successfully reported this slideshow.
Your SlideShare is downloading. ×

Carlton Wood | Operational Processes, Technolgy and Support.pptx

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 16 Ad
Advertisement

More Related Content

Similar to Carlton Wood | Operational Processes, Technolgy and Support.pptx (20)

More from EADTU (20)

Advertisement

Recently uploaded (20)

Carlton Wood | Operational Processes, Technolgy and Support.pptx

  1. 1. Designing Online Assessment Operational Processes, Technology and Support Carlton Wood - The Open University UK EADTU empower webinar week, 16 June 2022
  2. 2. Online Assessment Special Interest Group • Establishment of SIG January 2021; First meeting March 2021; 5 meetings in 2021- 2022 • 18 members institutions from 14 countries • 37 HE practitioners + 18 contributors of good practice • 15 Good Practice case studies • 4 Focus areas • Assessment Futures • Assessment Design • Trust, Privacy and Ethics • Operational processes, technology and support https://online-assessment.eadtu.eu https://doi.org/10.5281/zenodo.6563226
  3. 3. Assessment Future is already present • Online Assessment is here to stay • Transition phase from exam hall to online home exams • Distance Teaching sector long history of home-based assessment • EADTU member institutions best placed to lead • Great variety of hybrid solutions to online assessment
  4. 4. Operational processes, technology and support • Must start with thanks to all the contributors to the SIG • Without their generous inputs of time, thoughts and practises then there would not be anything to report • This presentation will cover Chapter 3 and I understand that fellow authors have already spoken in this empower session • All inputs were gathered as previously described – either from Case Studies provided from SIG members or via Padlet contributions • Distillation of common themes • Aim to highlight key ideas / practises that could be carried forward
  5. 5. Operational processes, technology and support (continued) The over-arching challenges that were identified that were faced by all institutions were: • How did institutions organise themselves to deliver the required operational processes? • Could face-to-face provision be adapted for on-line delivery? • To what extent was technology deployed to assist in the delivery of operational process? • How did Universities support staff and students whilst delivering the process?
  6. 6. 1) How did institutions organise themselves to deliver the required operational processes? • Rapid onset of the pandemic required urgent rapid responses • General uneasiness expressed which suggested that participants in the SIG somehow felt that they may be an outlier “do universities have already frameworks and guidelines for online examinations?” • No organisation had all the answers • Little evidence provided that anyone had generated a previously prepared process that it could swiftly bring out and deploy case study from Open University UK
  7. 7. 1) How did institutions organise themselves to deliver the required operational processes? ….cont. • The previous Good Practice identified 6 reasons or themes as to why this particular model appeared to work well. These themes were: • common purpose, • urgency, • listening and being heard, • student partnerships, • professionalising communication • cross university membership • Similar themes were presented from most of the members although there were obviously nuances dependent upon individual circumstances • Tension “creating meaningful assessments, measuring high thinking levels, which do not take up too much faculty time to grade and give feedback”
  8. 8. 2) Could face-to-face provision be adapted for on-line delivery? • It was clear that many institutions had good examples where assessment was delivered well in a face-to-face environment • The challenge was to what extent could the existing assessment be adapted to make it deliverable and useful in an on-line environment case study from Universidad Técnica Particular de Loja (UTPL) in Ecuador
  9. 9. 2) Could face-to-face provision be adapted for on-line delivery? • Some evidence of identifying individuals had experience of on-line delivery and so bringing these individuals together • Within the same University different approaches were required. Despite the University level overviews that were set up in the majority of institutions to handle process, it was clear that a ‘one size fits all’ was never going to work • Degree of flex so individual academic areas did not have to follow a single set unified process. • Sometimes these decisions were left to faculties “large scale proctoring took place in one faculty, open book exams in some of the other faculties” • Interaction between the academic side of Universities and Information Technology (IT) section of the same institution • “education development department sets out a framework, checklist, it goes through IT-department for working protocols to see if that's all fine”
  10. 10. 3) To what extent was technology deployed to assist in the delivery of operational process? To bring process change, application and deployment of technology featured heavily. Five themes emerged • Sheer range of solutions that were employed • Solutions developed in house versus those available commercially • Deployment of technology –proctoring and plagiarism tools • Changes that were introduced could increase concern/anxiety amongst students • Difficulties that were encountered with technology • Use of technology in different ways – “use of Zoom as a tool to support the student and visual supervision of the process in a synchronous way; Through this, I can verify the identity of the student” and secondly “oral examinations are replaced by online "zoom" sessions”
  11. 11. 3) To what extent was technology deployed to assist in the delivery of operational process? (cont) • Clear that respondents were happy to utilise commercially available software whilst others either used “in-house” solutions that was already developed • Some took the time to (rapidly) develop new additional in-house solutions - “development of a completely new application for secure and reliable online exams, AvEx, through the collaborative work of experts in technology and methodology” case study from Universidad Técnica Particular De Loja | Ecuador
  12. 12. 3) To what extent was technology deployed to assist in the delivery of operational process? (cont) • One development that demonstrated a wider perspective than just a single university was reported from Finland and relies on a national based approach to developing a technologically based solution “a national platform for e- exams is in use and being developed further” • This example is not an isolated one either and both the TeSLA project mentioned in the introduction to the report and the development and sharing of software solutions is highly encouraging • Technologies for proctoring or for detecting plagiarism by students – strongly linked to the trust element described in a previous session • All the technological solutions had potential to cause students stress - students need to be carefully guided in the 'technical' process to reduce stress. Provide an opportunity for a 'try out' [is also helpful] – [and] give clear instructions and also feedback on what is happening during the process”
  13. 13. 4) How did Universities support staff and students whilst delivering the process? • A major challenge that all institutions faced during the pandemic was how best to support both staff and students in a period of extensive change • Delivering this support in a non face-to-face manner added another level of complexity • This Case Study from the Open University of the University of Jyväskylä in Finland highlights key aspects case study from Open University of the University of Jyväskylä in Finland
  14. 14. 4) How did Universities support staff and students whilst delivering the process? (cont.) • Some of the support that Universities were offering was reactive but at other points Universities required students to take steps themselves to ensure that on-line assessments stood more chance of taking place successfully • Some institutions generated support that was equally applicable to both students and staff • Examples included providing technical support via telephone during live examination periods
  15. 15. Lessons learned going forwards • Timescales - if time is short do much of what is described in the report - if more time is available then identifying what tools and processes would best fit your university would be helpful • How prepared are the staff for any proposed changes? • Have a variety of support mechanisms to offer and to make available (UniDistance Vademecum example) • “Lot of supportive documents, protocols proposals, GDPR instructions, use cases presented by experienced teachers to other teachers; still in many ways teachers struggled” • We should question whether taking what we do in a face to face environment and simply converting that to on-line delivery is the most effective approach, it did in some circumstances and not in others • “We have built a pilot project to evaluate pedagogical challenges with online exams. This will lead to a report and guidelines and final strategy and regulations for next academic year”
  16. 16. Questions? carlton.wood@open.ac.uk

×