Approaches to assessing students


Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • What we are really here to talk about today is the preservation of academic integrity. I doubt that anyone in this audience involved in online learning would argue that integrity is an issue of concern, one that as faculty we wrestle with each semester. Why do we concern ourselves with academic integrity? What is our role as academics to ensure that academic integrity is upheld?
  • We should care about academic integrity because it is the very principle upon which academia is built. The quality of our institution, the respect of our degree programs, hinges upon academic integrity.
  • What is academic integrity? In addition to these 4 attributes I would also like to add accountability. Only if we hold ourselves accountable for our own actions can we safeguard integrity.
  • Why should we implement safeguards against academic dishonesty? First, we are required to do so by the government. Clearly institutions will have to find a solution to the proctoring of assessments in their distance learning programs. We do it because we have to.
  • But perhaps we should take a philosophical approach to academic integrity, and safeguard it not because we have to, but because we should. From my philosophical perspective, all that I have worked for my entire career, and all that I invest in my students is of little value if I do not safeguard the integrity of higher education. Otherwise, I have a $70000 piece of paper worth absolutely nothing. Even of more concern is the inability to trust the expertise of colleagues. We are building the future of our next generation on a fallacy.
  • Dondald McCabe is well-known for his surveys of cheating among college students. His work began in the mid 1990s, and has continued through today. In this time period he has amassed a large amount of data (a sample size the envy of any scientist) that shows the frequency of academic dishonesty by infraction type and the perceived severity of this infraction. Note that values are the percent of students who admitted to the behavior at least once in the last year. One in five students admitted to copying from another student, while one in three admitted to getting information about an exam from their peers who already took it.
  • Brain regions exhibiting increased activity in the Opportunity condition, as compared with the No Opportunity condition, broken down by group (honest vs. dishonest) and outcome type (win vs. loss). BA, Brodmann area. fMRI data are projected onto a reference anatomical image. (A) Increased activity in bilateral DLPFC is associated with decisions to lie (Opportunity Wins > No-Opportunity Wins) in dishonest subjects. (B) Increased activity in bilateral ACC/SMA, DLFPC, VLPFC, DMPFC, and right parietal lobe is associated with decisions to refrain from lying (Opportunity Losses > No-Opportunity Losses) in dishonest subjects. (C) Increased activity in bilateral VLPFC is associated with decisions to accept honest wins (Opportunity Wins > No-Opportunity Wins) in honest subjects. No significant effects were observed in association with decisions to refrain from lying (Opportunity Losses > No-Opportunity Losses) in honest subjects.According to the “Will” hypothesis, honesty results from the active resistance of temptation, comparable to the controlled cognitive processes that enable the delay of reward. According to the “Grace” hypothesis, honesty results from the absence of temptation,
  • Andrew Jackson view of proctoring
  • Approaches to assessing students

    1. 1. Approaches to Assessing StudentsWhile Ensuring Academic Integrity and Student Convenience WCET 2011 Conference, October 27, 2011 Denver, Colorado Panelists: April Cognato, Michigan State University (MI) Deb Gearhart, Troy University (AL) Mark Sarver, EduKan (KS) Moderator: Teresa Theisen, Minnesota State Colleges and Universities (MN) Facilitator: Scott Woods, University of Phoenix (AZ)
    2. 2. OverviewVarious methods are used in distance education toachieve a higher level of academic integrity.In the presentations, you will learn from severaladministrators who are striving to improve academicintegrity, avoid implementation obstacles, and provideconvenient methods for authenticating distance-learningstudents.Materials and activities will focus on lessons learned,success stories, and best practices in academic integrity.
    3. 3. April CognatoApril Cognato has been an assistant professor of biology atMichigan State University since 2006 where she is involvedin the development and implementation of traditional andonline curricula.Concurrently, she is a contributing author to two non-majors biology textbooks, and author of online digitalassets for McGraw-Hill Higher Education.She has been an undergraduate biology instructor since1996.Cognato received her B.S. from University of California,Davis and an M.S. and Ph.D. from Texas A & M University.
    4. 4. . Academic Integrity
    5. 5. One Falsehood Spoils a Thousand Truths --Ashanti Proverb April Cognato, Ph.D. Michigan State University
    6. 6. Academic Integrity……is the very principle upon whichcolleges and universities are built
    7. 7. Academic Integrity……is “a commitment, even in the faceof adversity, to five fundamentalvalues: honesty, trust, fairness, respect,and responsibility” Center for Academic Integrity
    8. 8. Academic Integrity Matters• Higher Education Opportunity Act 2008 requires that distance learning meets the same expectations as in- class instruction• Requires student authentication in distance learning
    9. 9. Academic Integrity Matters“Without academic integrity, there canbe no trust or reliance on the effectiveness,accuracy, or value of a Universitysteaching, learning, research, or publicservice activities.” –UC Davis Code of Academic Conduct The credibility of ouracademic system is at stake
    10. 10. Academic Integrity is in Crisis Admitted to cheating at Perceived severity of least once in the last behavior (moderate toDishonest behavior year (2002-2005) serious infraction) n = 71,071Learning what is on a 33% 64%test from someone whotook itCopying from anotherstudentHelping someone cheat 63% 20% 10% 90% 92% 89%Using unauthorized 8% 90%notesUsing an 5% 90%electronic/digital deviceFalse excuse to delay 16% 58%test McCabe, D. (2005) Cheating Among College and University Students: A North American Perspective. International Journal for Academic Integrity Volume 1, No.
    11. 11. “If it should ever come to that, and you…”Cheating is considered dishonest. Itknow no one has no proof whatsoevercounts as stealing and lying. There arethat you cheated: deny, deny, deny!some cases, however, where cheating on(Convince yourself as well be youra test might be argued to asadversaries you did not cheat,are testsacceptable. Sometimes there remainingfirm and the result of As long as theythanthat are confident.) politics, rather cannot prove it but can only wonder or hearpracticality.”someone elses words against your own,youre in the clear”.
    12. 12. Our Immoral BrainsOpportunity to lie Opportunity to lie No opportunity to lie Decision: LIE Deciding whether Decision: Honest or not to lie Greene, JD and JM Paxton (2009) Patterns of neural activity associated with honest and dishonest moral decisions. PNAS 106 (30) 12506-12511
    13. 13. Moral decisions……depend more on the absence oftemptation than on the active resistanceof temptation.Greene, JD and JM Paxton (2009) Patterns of neural activity associated withhonest and dishonest moral decisions. PNAS 106 (30) 12506-12511
    14. 14. Our Responsibility• Remove temptation• Provide a secure testing environment
    15. 15. Case 1Michigan State University
    16. 16. Remote Proctor ProA Comprehensive Proctoring Solution Dr. April Cognato, Ph.D. Michigan State University
    17. 17. Online Course• Fundamentals of Genetics o Fully online o 150 students o Single summer session (7 weeks) o Not proctored, remote• Assessments o Problem Sets, short answer, word problems o Summative assessments, weekly “mid-term” like exams
    18. 18. Academic DishonestyInfractions included:• Plagiarism from internet sources• Copying answer key (verbatim!!) provided by other students• Submitting, obtaining the key, resubmitting with the key (LMS bug)Suspected many; failed 6 students in 2009, 2010
    19. 19. A Comprehensive Remote Proctoring System Desired End User Desired System Features Features1. Flexible 1. Authenticates student2. Cost-effective identity3. Easy to employ for 2. Monitors student faculty and students activity4. Tech support 3. Secures computer access5. Customer support 4. Compact, portable 5. LMS compatible
    20. 20. Remote Proctor Pro Solution Authenticates o Biometric data Secures computer o Lockdown browser Monitors students o 360° camera Compact & portable o Desktop, USB, disassembles Records o Video record of exam session Reports o Exam policy violations
    21. 21. Remote Proctor Pro How it Works1. Assembly2. Enrollment3. Authentication4. Assessment5. Post-assessment Review
    22. 22. Bioauthentication
    23. 23. Post-assessment Review
    24. 24. Remote Proctoring SolutionsProvided students with two options:• Remote Proctor Pro (RPP)• National College Testing Association (NCTA), Consortium of College Testing Centers (CCTC) In-person Proctoring
    25. 25. RPP vs. Testing Center Feature RPP CCTC Authentication X XSystem Monitoring X X Browser security X Human Compact/portable X Convenient XEnd-user Cost-effective X User friendly X X Tech support X Customer support X X
    26. 26. Cost ComparisonTesting Center Hours Fee CTCC 2011 RPPMSU M 8AM-5PM $30/exam $180 W 12PM-5PMFerris State M 9AM-4PM $20/hourUniversity $240 W 9AM-4PMGrand Rapids CC M 7:30AM-7:30PM $10/hour W7:30AM-7:30PM $120Kalamazoo CC M 8:30AM-9PM $20/test $120 $100 W 8:30AM-9PMCentral Michigan M 9AM-1AM $30/testUniversity $180 W 9AM-1AMLake Superior State M 8:30AM-4PM $15/hour $180 W 8AM-12PMSouthwestern $20/examMichigan College M 8AM-5PM $120 W 8AM-5PM
    27. 27. Remote Proctored Exams• 86 students opted to use the RPP• 6 exams over 7 weeks• Semi-synchronous exam periods• 2 hour time limit• Multiple choice, short answer, essay (word problems)• Survey of proctoring experience delivered for credit at the end of the course
    28. 28. Remote Proctor Pro Protects IntegrityLMS Bug 4% 4% 8% Impossible/Diffic ult to Cheat Easy to Cheat Knows someone 84% who cheated No reponse
    29. 29. Student Testimony“It’s a great way to have the freedom to take the exam onyour own time in whatever setting you want” “It’s easy to use and you can take [the test] when you want”“The remote proctoring device was easy and convenient…you couldtake your exam in your own home and at whatever time you wanted.”“You couldn’t cheat and I took my test in the comfort of my own home.” “Remote proctoring tech support was VERY NICE and HELPFUL.”“The lowered cost compared to the in-person proctoring exam was a plus.” “It looks like something out of Star Wars!”
    30. 30. Take Home Points• Remote Proctor Pro system effectively protects academic integrity• RPP is a comprehensive proctoring system o Authenticates student identity o Monitors student activity o Secures computer o Records exam session o Reviews and Reports• RPP meets the needs of the end user o Easy to use (faculty and students) o Convenient o Cost effective o Responsive tech support o Attentive customer service
    31. 31. Future Best Practices• Inform students of proctoring requirement (and associated costs) at the time of registration.• Deploy device at least 2 weeks prior to beginning of course.• Require a practice assessment the first week.• Institute a backup exam plan.• Establish good communication with your CMS IT.• Provide students with clear instructions for Technical Support.• Anticipate technically challenged students!
    32. 32. Pedagogical Best Practices • Tutorial on academic integrity • Synchronized assessments • Question pools • Random delivery of questions and answers • Exam review controlledTesting center at the University • Required exam of Central Florida NY Times July 5, 2010 proctoring
    33. 33. Enrollment
    34. 34. Image SourcesLogo: Center forAcademic Integrity http://catholicno(http://www.academici m/dougleschan/wp- m/2010/11/low-tech-way-to- content/uploads/2011/0 prevent-cheating-on.html com/2007_07_01 8/integrity-150x150.gif _archive.html
    35. 35. Deb GearhartDeb Gearhart is the director of eCampus for Troy University.Previously Gearhart served as the founding director of E-Education Services at Dakota State University inMadison, South Dakota and was there for the 11 years.Before joining Dakota State she spent 10 years with theDepartment of Distance Education at Penn State.She earned a M.Ed. in Adult Education with a distanceeducation emphasis and an M.P.A. in PublicAdministration, both from Penn State. Gearhart completedher Ph.D. program in Education, with a certificate indistance education, from Capella University.
    36. 36. Case 2Troy University
    37. 37. • On average eTROY proctors over 7500course exams a term• A need to assist students withproctoring options• Adopted an academic operatingprocedure for online course proctoring in2008.• Needed both human and technologybased proctoring options.
    38. 38. The educational-technology firm istrusted by over 100colleges, universities andcorporations as the best way toreduce incidents of academicdishonesty by connecting studentsface-to-face with a human proctor.
    39. 39. See the student See what they are doing Know who they are
    40. 40. • Students interact with a live proctor in real time.• Our identity authentication method is the strongest in the industry.• ProctorU is easy to use for students and faculty.• Students enjoy the convenience of testing at home and instructors rest easy knowing the integrity of their exam is secure.
    41. 41. Mark SarverMark Sarver has served in many capacities in highereducation. He is the CEO of EduKan, the onlineeducation consortium for the six western communitycolleges in Kansas. Sarver has taught courses inleadership, organizationalbehavior, accounting, international business, franchisedevelopment, marketing and management. Sarversexperience in higher education strategy andassessment spans virtually all aspects of institutionalmanagement from admissions and marketing, tofinancial analysis, to program and course assessment.
    42. 42. Case 3EduKAN
    43. 43. Troy UniversityNorthwestern UniversityThe University of Florida The University of ArizonaThe University of IllinoisBob Jones UniversityChaminade University of HonoluluKansas State UniversityLawson LearningNational American UniversityOregon State UniversitySaint Louis UniversityThomas Edison State CollegeThe University of West FloridaThe University of Louisiana at LafayetteThe University of MississippiThe University of New EnglandThe University of Tennessee Health SciencesWashington State UniversityNorthern Virginia Community College (ELI)Others…
    44. 44. “I have completed countless onlinecourses. Students provide me withpasswords and user names so I canaccess key documents and online exams.In some cases, I have even contributed toweekly online discussions with otherstudents in the class”. “IDENTITY THEFT ON DEVICES AND OVER NETWORKS CREATES THE FASTEST GROWING WHITE COLLAR CRIME. BIOMETRIC SIGNATURE ID HAS AN ANSWER.” Patents: 7663614, 8004491 Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID
    45. 45. Agenda• About EduKan• BSI Company Background• Briefing on Biometrics and Multi factor Authentication• Edukan case study – Dr Mark Sarver CEO• Q&A Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID 47
    46. 46. Biometric Signature ID Corporation Dallas-based privately held software development company The way you move your mouse (pointing device) is unique and can make a positive ID using dynamic (behavioral) biometrics  Gesture (dynamic) biometrics capture speed, length, angle, height direction of how we write not what we write “3 identity proofing software products” that require no hardware or installation unlike other static biometrics (fingerprint etc…) Satisfies security need in multiple sectors for remote identity proofing and access to the PC and smart devices  DEA, SSA, HHS, DE, Homeland Security, financial services, white house, cloud computing, e-commerce, FFIEC… Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID 48
    47. 47. Static versus Dynamic Biometrics• Static biometrics are physiological or anatomical i.e. fingerprints, iris scans, facial recognition, DNA swabs etc.• Static biometrics require “hardware”.• Dynamic biometrics measure behavioral characteristics and do not require the use of hardware In our case we capture a students movements as they draw with their mouse, finger, stylus, touch screen (pointing device). Movements such as speed, length, angle, height, direction are unique to each student. Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID 49
    48. 48. Multi-Factor Authentication Is the new “Gold Standard”Something You Have Something You Know Who you AreCredit card, Tokens, Unique characteristics - static PINS, Passwords, ImagesSmart Card, devices OR Dynamic Biometrics Weak Strong Very Weak Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID 50
    49. 49. Closed Loop Technology – Always 2 Levels BioSig-ID Click-ID CSQ – Complex Questions + + -Enrollment – Two levels are always required for enrollment in BioSig-ID and Click-ID. In rarecircumstances Click-ID and CSQ’s are used only when user cannot enroll in BioSig-ID.Validation - If user exceeds the number of attempts to validate with BioSig-ID, they are directedto Click-ID to validate. If successful, user creates a new enrollment with BioSig-ID. This is thesame process if the user has enrolled with Click-ID and CSQ’sCost Savings - Can be used for password resets – reducing help desk calls and overhead. Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID 51
    50. 50. EduKan Case Study - Primary Goals EduKan worked with Biometric Signature ID to run a proof-of-technology pilot because: 10,000 hours for proctoring and related costs Proctoring may create a competitive disadvantage Proctoring may create physical presence in some states Meets HEOA and HLC requirements Wanted to assess their biometric technology as a cost-effective way of addressing online student security and reduction of proctoring Measure student acceptance of new identity verification software Uses a biometric system where no hardware is required Tests a system where students create their unique password by just drawing shapes/letters/numbers with their mouse Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID
    51. 51. EduKan Methodology During March-April, 2011, 174 students from multiple classes volunteered to verify their identity from their own computers using BioSig-ID Test subjects/students were asked to enroll, create a profile and verify their identity at various times for a total of 6 times before the final exam. At the final exam they were asked to validate their identity and receive a password to access the exam Full audit trails were analyzed of all activity and an online survey was administered Nearly 6,300 enrollments/verifications were completed by the participants in 60 days Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID
    52. 52. Current Verification Methods Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID
    53. 53. EduKan Overall FindingsBased on the results of this technology pilot, the BioSig-ID verification softwareusing just a mouse was a useful and practical tool for remote identity proofing.During the pilot study, it required little administration (only 9 help desk calls from174 students) and will help EduKan comply with newer, more stringentrecommendations of the HEOA and HLC.EduKan intent is to replace proctored exams with BioSig-ID and have started fulldeployment with over 1,600 students Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID
    54. 54. 97% Expressed Ease of Use Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID
    55. 55. 16% did not view the video? Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID
    56. 56. For Testing: 97% of Students Prefer BioSig-ID Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID
    57. 57. Typical Responses Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID
    58. 58. Features Students Liked from EduKan Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID
    59. 59. Going Forward Full implementation with deep integration into LMS (Pearson Learning Studio – eCollege) Must change the way we teach, evaluate and assess All 4,000 students Implementing U-Sign Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID
    60. 60. Contact information Stephanie Piche, Director Marketing 310-729-2997 Jeff Maynard, President 972-436-6862 Dr. Mark Sarver, CEO EduKan 620-204-0374Test drive the software Signature ID completes very successful proof of technology with Houston Community College District the 4th largest in the nation with 98% satisfaction• Signature ID white paper on How to reduce Academic Dishonesty (highlights uSignOnline)• third party test results announced proving BioSig-ID software exceeds the national NIST standard for accuracy by a 3 fold factor;• 97-accuracy-against-identity-fraud-from-the-tolly-group/ Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID 62
    61. 61. User Acceptance + 3rd Party Testing Accuracy Studies Summary of Results 100.0% 99.78% 100 97.3% 99.97% 94.0% 75 2 minutes 50 19 Seconds 25 Enrollment *Ease of Ease of Stop Imposters Access Granted Average Average Attainment Enrolling Validation With Passwords To Legitimate Users Enrollment Validation V2.0 Validations – University Pilots (multiple campuses), other users, Independent 3rd party testing subjects by The Tolly Group =32,000 total enrollment and validation transactions Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID
    62. 62. BSI: Recognition, Validation and Acceptance Awards: BSI awarded the “2010 New Product Innovation of the Year Award Frost & Sullivan employs 1,800 analysts in 40 global offices… for North America Signature Biometrics” from Frost and Sullivan Third Party Testing: Software denies >99.9% of imposters from successful log in, results exceed NIST standards Rigorous Evaluation: State of Texas recognition and investment as “breakthrough technology” User Acceptance 98% of first time users performing over 50,000 actions had a positive experience Patented Technology: BSI has University of Texas Systems – received two US patents for their User Acceptance Studies gesture biometrics technologies 9 campuses, grad/undergrad, 46 classes, 167 students, 6 weeks, 10 authentications at (Patent -7663614,8004491) random, different ages, locations, disciplines Copyright ©2011 All Rights Reserved - Confidential Biometric Signature ID 64
    63. 63. Questions andConclusions