Testing is Changing - Questionmark & ProctorU Briefing

869 views
753 views

Published on

Improve the security of your online assessments using Questionmark and ProctorU's assessment technologies. This presentation touches on some of the secure assessment delivery methods available such as online invigilation.

Published in: Services, Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
869
On SlideShare
0
From Embeds
0
Number of Embeds
40
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Testing is Changing - Questionmark & ProctorU Briefing

  1. 1. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Testing is Changing: Practical and Secure Assessment in the 21st Century June, 2014
  2. 2. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Agenda Slide 2 8.45– 9.30 ​Registration 9.30 – 9.45 Welcomes and introductions ​9.45 – 10.55 Testing is Changing: Practical and Secure Assessment in the 21st Century Demonstration ​10:55 – 11:15 ​Networking Break ​11.15 – 12.20 Flexibility and Security – From Test Authoring to Delivery and Reporting Demonstration 12.20 – 12.30 ​Q&A
  3. 3. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. • Founded in 1988 • Offices in US, UK, Canada; Distribution partners in 27 countries • Questionmark provides technologies and services that enable organizations to measure knowledge, skills, and attitudes securely for certification, regulatory compliance and improved learning outcomes. Background • 1,000 current enterprise customers worldwide • Corporate and Government (Learning/HR function); Higher Education; Awarding Bodies and Test Publishers Customers About Questionmark
  4. 4. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. • Assessment Management System • Questionmark OnDemand (SaaS/cloud) • Questionmark Perception (installable software) • Multilingual • Localized for dozens of different languages and cultures • Global 24/7 Support Products and Services About Questionmark
  5. 5. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Che Osborne VP of Sales Kerry Williams Regional VP Sales, Europe Ivan Forward Assessment Solutions Consultant Chloe Mendonca Marketing Introductions: Questionmark Slide 5
  6. 6. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Don Kassner Matt Jaeh Introductions: Slide 6
  7. 7. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. How Testing is Changing in the 21st Century Presented by Don Kassner – President, ProctorU
  8. 8. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.  33% of all job openings and nearly 50% of all new jobs created between 2008 and 2018 will require a postsecondary degree or credential.  How can organizations keep up with the demand? An Educated Work Force http://www.mcgraw-hillresearchfoundation.org/wp-content/uploads/the-return-on-investment-from-adult-education-and- training.pdf#page=6
  9. 9. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.  70% of companies are planning to implement mobile learning in the next 2 years. Learning in the 21st Century  e-Learning - FOUR out of FIVE companies are reducing costs, increasing the speed of engagement with learning and reducing time spent away from the business. http://towardsmaturity.org/article/2013/01/15/towards-maturity-focus-report-mobile-learning-work/ http://corporateelearningconsortium.com/wp-content/uploads/2012/12/Leadership-research.pdf 70%
  10. 10. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.  Certification testing is evolving from linear-exams to performance-based assessments  Employers want something that is outcome-based Evolution of Certification
  11. 11. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.  Long distance and associated travel costs  Not 24/7  Cannot meet scalable demand  Requires time away from business  Requires time-consuming post-exam review  Cannot catch integrity breaches in real-time  No live technical support Secured Assessments Onsite Proctoring Automated Remote Proctoring
  12. 12. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Secured Assessments  Ability to address integrity breaches as they occur  Live technical support  24/7 operation  Multiple location redundancy  Convenient testing from home / workplace  Provides a human element of customer service Live Online Proctoring
  13. 13. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Identity Verification Identity Verification Attendance Verification  Government-issued identification  Keystroke analysis  Public Records challenge- based quiz  Facial-recognition technology  Fingerprint scanner (session entry verification, not identity)
  14. 14. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.  Password-protected exams  Secure Learning Management Systems  Multiple test banks  Limiting or eliminating “Slippery Slope” exams Exam Structures
  15. 15. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.  Tools to combat major sources of online plagiarism  Create a culture of assessment integrity among administrators and test-takers  Tighten proctoring requirements Additional Preventative Measures
  16. 16. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Demonstration Slide 16
  17. 17. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. What questions do you have? Slide 17
  18. 18. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Break
  19. 19. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Ensuring Assessment Security Secure Technology and Good Practice
  20. 20. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Software as a Service Installable Software Assessment Management System
  21. 21. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. European Data Centre Slide 21 ISO 2700 and 9001 accredited State-of-the-art datacentre SaaS / Rapid Deployment 24/7 Monitoring Global 24/7 support available 99.9+% Uptime
  22. 22. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Protection of candidate/student PII (Personally Identifiable Information) Protection of valuable assessment content • Average corporate test: 15,000 GBP • Average certification test: 100,000 to 200,000 GBP Protect integrity of test/exam results • A lot could be on the line… • Reputation • Life and Limb Key drivers for secure assessment delivery Slide 22
  23. 23. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Slide 23 Stakes of assessment Licensure exams Certification exams Summative Assessments Placement tests Pre-certification exams Job Task analysis surveys Needs analysis surveys Post-course test Pre-course test Formative quizzes Course Evaluations Satisfaction surveys Stakesofassessment Motivationtocommitfraud
  24. 24. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Pre- employment Public Certifications & Licensing Regulatory Compliance Sales and Technical Channel Verification Threat Level in Higher Stakes Short Term with Low Trust Relationships Long Term with High Trust Relationships LargePrograms SmallProgram Higher threats require more “Oversight” and so cost more Lower threats require less “Oversight” and so cost less to administer Slide 24
  25. 25. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Impersonation Content Theft Cheating Key Threats to be Addressed in High-stakes Exam Delivery Slide 25
  26. 26. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Tight controls over the access to content Shuffling items and choices – limit exposure of item pool Not exposing the scoring algorithm beyond the content repositories/databases Securing, and only providing limited access to, the content repositories/databases Enforceable candidate agreements Vigorous follow up on infractions Mitigate Leakage of Content Slide 26
  27. 27. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Randomization …of choices …of questions Slide 27 Helpful for… • Re-use of items in practice tests/quizzes • Mitigate cheating/copying from “wandering eyes”
  28. 28. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Demonstration Secure, Collaborative Authoring Slide 28
  29. 29. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Enforceable candidates agreement; formal honesty contracts Monitoring (Proctoring/Invigilation) Secure browsers/players on candidate devices Mitigate Cheating Slide 29
  30. 30. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Questionmark Secure One Setting – Multiple Platforms Slide 30 Windows Mac iPad Printing Screen capture Task switching Control/system keys Right-click / context menu options View source Accidentally exiting exam Prevents….
  31. 31. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Test Centers Events (Classrooms or Conventions) Dedicated Kiosk with multiple cameras Remote real-time webcam Record & Review / 360 or webcam Un-Monitored Options for Monitoring Tests Slide 31
  32. 32. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. •Personable, consistent •Monitor vulnerable to unfair influence Test Centers •Convenient, personable, consistent •Monitor vulnerable to unfair influence Events (Classrooms or Conventions) •Monitor is less vulnerable •New and not yet widespread Dedicated Kiosk with multiple cameras •Monitor is less vulnerable •Easy to deploy for use at home Remote Real-time webcam •Authentication and forensic benefits •Nothing for content theft Record & Review 360 or webcam •Works for employees •Nothing for content theft Un-Monitored Monitoring Tests Securely Slide 32 IDFraud Protect Content Minimizes Cheating √ √ √ √ √ √ √ X √ X X √ √ √ √ √ √ √
  33. 33. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Reporting and Analytics Slide 33
  34. 34. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Analysis How are our Assessments doing? How are our Participants doing? Did the learning investment impact performance? Where are the skills and knowledge gaps in my organization? Are my questions and tests valid and reliable? What did participants think of the learning experience? What did participants learn from the training or course? How do I coach participants to improve performance? How did one group of test takers compare to another?
  35. 35. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Analyze Time to Complete Exam Slide 35 Valuable completion time analysis useful to: • Identify potential cheating • Flag potential content theft • Determine if allotted time is sufficient for completing the assessment
  36. 36. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.  How do aggregate results compare at one center vs. another? Analyze Results by Test Centre/Location Valuable for spotting problems with invigilation
  37. 37. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Demonstration Reporting and Analytics Slide 37
  38. 38. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Providing a Great Candidate Experience Slide 38
  39. 39. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Turnkey web portal solutions for awarding bodies Configured and branded to your organisation’s needs Powered by comprehensive, scalable and secure platform Questionmark Portals Slide 39
  40. 40. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. An Example Portal Slide 40
  41. 41. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. OnDemand Portals On Your Design Your Logo and Your Color Scheme Slide 41
  42. 42. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Simple For Candidates To Register Simple registration Slide 42
  43. 43. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Simple To Sign In Simple Sign In Slide 43
  44. 44. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Menu Options Different Menu Options for Guests and Candidates Slide 44
  45. 45. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Highlight Key Messages Quick links to important messages Slide 45
  46. 46. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. News Feeds Latest News from your news feeds Slide 46
  47. 47. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Your Pages You manage your page content Slide 47
  48. 48. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Certificate Slide 48
  49. 49. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.  Awarding body needs to reach a wide global audience; physical test centers do not reach global audience and are too costly  Awarding body still needs exams to be monitored  Candidates can take exams online anywhere in the world, but must book an appointment with a remote monitoring service first  Remote monitoring service is responsible for  Visually identifying/authenticating candidate/securing exam environment  Generating exam launch URL  Monitoring exam in real-time Customer Story: Awarding body portal Slide 49
  50. 50. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Candidate Portal – Candidate Process
  51. 51. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Candidate Portal – Monitor Process Slide 51 Registration ID Questionmark OnDemand Tenant ID Launch URL
  52. 52. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Configurable, branded portal pages Self Registration Automated scheduling Payment processing Keep candidates informed Certificate Generation How portals benefit certification programs
  53. 53. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. What questions do you have? Slide 53
  54. 54. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Complete your evaluations for prize drawing! Slide 54
  55. 55. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. www.questionmark.com/whitepapers Recommended White Papers Slide 55
  56. 56. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Keep in Touch Slide 56 blog.questionmark.com www.questionmark.com/newsletter twitter.com/questionmark linkedin.com Look for Questionmark under “Groups” and join youtube.com/questionmarkvideo flickr.com/questionmarkpictures facebook.com Become a “fan” for news, updates and more
  57. 57. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Learn more about ProctorU Slide 57
  58. 58. Copyright © 1995-2014 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Thank you for attending! Slide 58

×