Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Paul Bouniol: Online Proctoring Forecast to Develop More than Ever Before

25 views

Published on

Online Proctoring Forecast to Develop More than Ever Before

Published in: Education
  • Be the first to comment

  • Be the first to like this

Paul Bouniol: Online Proctoring Forecast to Develop More than Ever Before

  1. 1. Online Proctoring Forecast to develop more than ever before? Paul Bouniol, LanguageCert Academic Consultant [Paul.Bouniol@peoplecert.org] ©Eaquals Eaquals International Conference | Madrid | 11-13 April 2019 #eaquals19madrid
  2. 2. Online Proctoring: Forecast to develop more than ever before? From e-learning and CB testing to OP OP explained Evaluating OP PeopleCert & OP Future developments Conclusions ©Eaquals #eaquals19madrid
  3. 3. Proctoring explained ©Eaquals #eaquals19madrid Proctoring = Invigilation: “Watching people taking a test to make sure they don’t cheat” Cheating: • Using prior knowledge of test content • Getting test answers by illicit means • Impersonation or substitution • Copying • Recording test content & responses Thecheatingtriangle(D’Souza, 2017) opportunity rationalisation incentive
  4. 4. Online Proctoring explained Online (or Remote) Proctoring: • Using a person and/or system to support invigilation • Where the proctor is not physically present • Uses online connectivity to observe/ analyse/ record test taker behaviour ©Eaquals #eaquals19madrid
  5. 5. Where does Online Proctoring come from? OPbuilds on online learning and CB testing Some benefits of CB testing: • Fast results • Auto marking • Analytical feedback • No print/shipping costs • More secure (fewer people in delivery chain) • Offers flexibility & choice • Much emphasis on design of online learning, much less on e- invigilation (Ketab, 2015) ©Eaquals #eaquals19madrid
  6. 6. What is Online Proctoring addressing? CB testing centres have to be in multiple locations with equipment, premises and staff on site Online Proctoring removes requirements for • equipment • location • invigilator present While supporting • security • cost effectiveness • fair & accurate assessment ©Eaquals #eaquals19madrid
  7. 7. Types of Online Proctoring Live Online Proctoring • trained proctor in a remote location • monitors the candidates in real time Recorded Proctoring • exam recorded and played back later to red-flag suspicious activity Advanced Automated Proctoring • uses face recognition for authentication & audio-video and screen recorded • system monitors feeds for suspicious activity - use of advanced video and audio analytics • detects candidates talking, tilting heads, missing faces, noises, etc. ©Eaquals #eaquals19madrid
  8. 8. A Case Study: PeopleCert Brief profile • PeopleCert: organisation administering advanced automated proctoring since 2011 • Develops and manages globally recognised qualifications • Delivers standardised exams on behalf of governments, professional associations, educational institutions etc. • Maintains certification programmes on behalf of a wide range of organisations • Qualifications development and management rely on a robust system and applications ©Eaquals #eaquals19madrid
  9. 9. A Case Study: PeopleCert Highlights • Has so far delivered more than 5 million exams across over 200 countries • Uses over 10,000 examination locations worldwide • Has a network of more than 8,000 subject matter experts, associates and invigilators • Administers Paper & Computer-based exams in 25 languages • Operates 24/7/365 and provides multilingual customer support • Possesses an award winning online proctoring solution (‘Innovation Award’ at the Business IT Excellence Awards) ©Eaquals #eaquals19madrid
  10. 10. A Case Study: PeopleCert Comprehensive Portfolio of 500+ Qualifications ©Eaquals #eaquals19madrid Business and IT Qualifications • AXELOS Best Management Practice (ITIL, PRINCE2, MSP, MoV, MoR, P3O, RESILIA) • COBIT (ISACA), DevOps (Dev Ops Institute), • DMI (Digital Marketing Institute) • Lean IT (Lean IT Association) • Lean Six Sigma (IASSC) • ECDL/ICDL (ECDL Foundation), • Quality Software Developer (PeopleCert and SIG) • Professional Valuers (Appraisers) • Management Systems Professionals • Other certificates under development
  11. 11. A Case Study: PeopleCert A picture is worth 1,000 words: Tests within the past 12 months ©Eaquals #eaquals19madrid
  12. 12. Comprehensive Portfolio LanguageCert Qualifications ©Eaquals #eaquals19madrid • A PeopleCert Group company • Awarding Organisation dedicated to language skills assessment & certification • Develops own language qualifications • Partners with renowned organisations worldwide to offer high-quality language skills assessment and certification • Deploys PeopleCert’s state-of-the-art, innovative and flexible exam administration technologies and systems About LanguageCert
  13. 13. Comprehensive Portfolio LanguageCert Qualifications ©Eaquals #eaquals19madrid ENGLISH LanguageCert International ESOL LanguageCert Young Learners ESOL SPANISH LanguageCert USAL esPro TURKISH LanguageCert Ankara University TÖMER PB & CB exams available Portfolio to be enriched & more languages on the way
  14. 14. Online Proctoring Main features • Addition to CB assessment • Allows candidates to sit secure tests in any location • Prevents/identifies attempts to cheat in tests and deters impersonation, copying, learned responses • Candidates monitored online by means of webcam, mic & access to their screen • Monitoring software tracks actions and behaviour • However candidates need access to: - a suitable device (laptop/desktop PC, tablet, mobile) - an internet connection (256 + kbps speed) - a functional webcam and a mic ©Eaquals #eaquals19madrid
  15. 15. Online Proctoring Candidate’s Steps ©Eaquals #eaquals19madrid
  16. 16. Online Proctoring Candidate’s Steps • Candidate downloads exam application (Locks down browser to mitigate content theft and cheating) • Two kinds of prevention features: - Navigation Prevention Features - Infringement Prevention Features ©Eaquals #eaquals19madrid
  17. 17. Online Proctoring Navigation & Infringement Prevention Features Navigation Prevention Features: • Prevents right-click • Prevents function keys • Prevents some key combos • Hides Taskbar and Desktop • Hides menus and icons • Prevents window minimization • Prevents Copy/Paste • Prevents printing ©Eaquals #eaquals19madrid Infringement Prevention Features: • Terminates blacklisted applications (e.g. screen capture & keyloggers) • Prevents launch of applications during the exam session • Terminates communications software • Prevents execution on virtual machines • Prevents simultaneous tests • Allows single monitor only
  18. 18. Online Proctoring Initial Candidate’s Checks ©Eaquals #eaquals19madrid
  19. 19. Online Proctoring Initial Proctor’s checks ©Eaquals #eaquals19madrid
  20. 20. Online Proctoring Live Proctoring • Proctor to candidate ratio 2:8 • Ensures at least one proctor monitors candidates at all times • Proctors use a 40-inch screen and observe up to 8 candidates at once • All candidate and system actions monitored, logged and timestamped ©Eaquals #eaquals19madrid
  21. 21. ©Eaquals Eaquals International Conference, Riga, 27 – 29 April 2017 www.eaquals.org 21
  22. 22. Online Proctoring Initial instructions to Candidates ©Eaquals #eaquals19madrid
  23. 23. Online Proctoring Monitoring Software • PeopleCert uses its own communication and monitoring software • Proctors can communicate with candidates and identify notable behavior • Software analyses: • Voice detection / Candidate Talking • Candidate Tilting Head • Missing face • Multiple faces detected • Background noises • Alerts represented with icons on the proctor’s interface • All alerts tagged, timestamped and logged ©Eaquals #eaquals19madrid Multiple faces detected Candidate talking Candidate Tilting head No face detected Background Noise detected Low Battery
  24. 24. Online Proctoring Auditing Authorized auditors : • Select and replay any test • Playback recordings at different speeds • View automated tags - captured by the system or added by proctor • Play the video at time stamp • Tag and add comments on video ©Eaquals #eaquals19madrid
  25. 25. Is Online Proctoring the future? • CB will grow & flexibility in assessment delivery is required • Demand to address individual needs (genuine flexibility/availability) • Tests will occur more often at candidate’s home, employment etc. • Candidates expected to use range of tools ©Eaquals #eaquals19madrid
  26. 26. Developments in Online Proctoring Further developments expected & research • Enhance the test taker's authentication by the use of biometric inputs • Facial recognition to be used to avoid impersonation • Keyboard behavior analysis will further develop • Broader use of ways to identify patterns of unusual behavior/response ©Eaquals #eaquals19madrid
  27. 27. Conclusions, Online Proctoring ... and Languages • An addition to traditional test venues • Supports accessible testing in remote areas, for people who can’t travel etc. • Enables a more personalised provision in future • Combines technology and a friendly face to provide high quality service • Language exams about to be launched as an OP option ©Eaquals #eaquals19madrid
  28. 28. Thank you* * Special thanks to Roger Johnson & Marios Molfetas for their input and support ©Eaquals #eaquals19madrid
  29. 29. References and useful sources ©Eaquals #eaquals19madrid Amigud, A., Arnedo-Moreno, J., Daradoumis, T., Guerrero-Roldan, A.E. (2018), "An integrative review of security and integrity strategies in an academic environment: current understanding and emerging perspectives", Computers & Security. Atoum, Y., Chen, L., Alex X. Liu, A.X., Hsu, S.D.H., Liu, X. (2017), "Automated Online Exam Proctoring", Multimedia IEEE Transactions, vol. 19, 7: 1609-1624. Berkey, D., & Halfond, J. (2015). Cheating, student authentication and proctoring in online programs. New England Journal of Higher Education. D'Souza, K. A. and Siegfeldt, D. V. (2017), A Conceptual Framework for Detecting Cheating in Online and Take‐Home Exams. Decision Sciences Journal of Innovative Education, 15: 370-391. Karim, M. N., Kaminsky, S. E., & Behrend, T. S. (2014). Cheating, reactions, and performance in remotely proctored testing: An exploratory experimental study. Journal of Business and Psychology, 29: 555-572.
  30. 30. References and useful sources (Cont’d) ©Eaquals #eaquals19madrid Ketab, S., Clarke, N. & Haskell-Dowland, P. (2015). E-Invigilation of E-Assessments. Conference: Proceedings of INTED2015 Conference 2nd-4th March 2015, Madrid, Spain. Lilley, M., Meere, J., & Barker, T. (2016). Remote live invigilation: A pilot study. Journal of Interactive Media in Education, 1: 1-5. Plaus, K., Boren, L., Brazell, T. P., Wickett, J., & Weber, A. (2015). Remote proctoring test delivery: A report on options and considerations. Washington, DC: Institute for Credentialing Excellence. Siegmann, S. M., Moore, D., & Aquino, C. P. (2014) Exam performance in a hybrid course: A model for assessing online and in class delivery modes. The Online Journal of Distance Education and e‐Learning, vol. 2, 4: 70- 80. Wright, N. A., Meade, A. W., & Gutierrez, S. L. (2014). Using invariance to examine cheating in unproctored ability tests. International Journal of Selection and Assessment, 22, 12-22.

×