Assessment

1,162 views
1,063 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,162
On SlideShare
0
From Embeds
0
Number of Embeds
39
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Assessment

  1. 1. Robert Gibson, Ed.D. eLearning Institute October 9, 2008 Emporia State University Effective Online Assessment Strategies
  2. 2. Is Cheating a Problem on Your Campus? Research suggests it may be a very big problem… perhaps bigger than you realize Effective Online Assessment Strategies
  3. 3. How to Cheat Online <ul><li>Why do students cheat? </li></ul><ul><li>Everybody does it </li></ul><ul><li>“ If I don’t cheat, my grade suffers” </li></ul><ul><li>“ Unrealistic” demands by faculty </li></ul><ul><li>Opportunity presents itself </li></ul>Effective Online Assessment Strategies
  4. 4. How to Cheat Online <ul><li>In a recent survey of high school students: </li></ul><ul><ul><li>80% of high-achieving students admitted to cheating at least once </li></ul></ul><ul><ul><li>50% said they did not believe cheating was necessarily wrong </li></ul></ul><ul><ul><li>95% of the cheaters said they’ve never been caught </li></ul></ul><ul><li>According to Duke’s Center for Academic Integrity , 75% of college students confess to cheating at least once </li></ul><ul><li>Technology makes it easy. Students use the Internet, pagers, and cell phones to cheat </li></ul><ul><ul><li>http://chronicle.com/free/2008/09/4705n.htm </li></ul></ul><ul><ul><li>http://tinyurl.com/4jler4 </li></ul></ul><ul><ul><li>Using “Slugs” </li></ul></ul>Effective Online Assessment Strategies
  5. 5. How to Cheat Online <ul><li>In a study of 500 middle and high school students, Rutgers management Professor Don McCabe found that more than two thirds said working with classmates didn't constitute cheating, and half thought it was fine for parents to do their homework </li></ul><ul><li>Males report significantly higher incidences of cheating than do females </li></ul><ul><li>No difference between freshmen and sophomores </li></ul><ul><li>No difference between full- and part-time students </li></ul><ul><li>Residential college students perceive cheating to be more socially acceptable than do off-campus students </li></ul>Effective Online Assessment Strategies
  6. 6. How to Cheat Online <ul><li>Business majors tend to score lower on measures of moral development and reasoning than some other majors (particularly when compared to liberal arts students) </li></ul><ul><li>Cheating behaviors are a function of situational factors that appear to be either independent of or are at least poorly correlated to the level of moral development </li></ul><ul><li>Data suggest that instructors who repeatedly use the same examination each semester encourage cheating. Thornton & Peterson (2000) </li></ul>Effective Online Assessment Strategies
  7. 7. How to Cheat Online Blogging Made Easy with Wordpress … and the faculty?
  8. 8. How to Cheat Online <ul><li>A 25-question survey was completed by 742 faculty: </li></ul><ul><li>Faculty do not perceive academic dishonesty to be a serious problem! </li></ul><ul><li>Faculty claimed to be familiar with current policy and procedure and were unconcerned with implementation </li></ul><ul><li>Most faculty did not regularly follow institutional policy; rather, handled incidents of cheating and plagiarism on an individual basis. They believed responsibility for reducing academic dishonesty lay primarily with students and individual faculty </li></ul>Effective Online Assessment Strategies
  9. 9. How to Cheat Online <ul><li>86% of faculty have suspected and 65% have been certain of academic dishonesty in their classrooms </li></ul><ul><li>800 faculty at 16 institutions surveyed: </li></ul><ul><ul><li>40% indicated they never reported cheating </li></ul></ul><ul><ul><li>54% indicated they seldom reported cheating </li></ul></ul><ul><ul><li>6% indicated they often reported cheating </li></ul></ul><ul><ul><li>Concerns regarding potential legal action or lengthy student appeals dissuaded many faculty from reporting this problem </li></ul></ul>Effective Online Assessment Strategies
  10. 10. Continuum for Cheating <ul><li>Excuses </li></ul><ul><li>Blaming the Technology </li></ul><ul><li>Dishonest Collaboration </li></ul><ul><li>Exploiting the Technology </li></ul><ul><li>Hacking the Technology </li></ul><ul><li>Low </li></ul><ul><li>High </li></ul>Effective Online Assessment Strategies
  11. 11. Continuum for Cheating <ul><li>Excuses, excuses… </li></ul><ul><ul><li>Exploit unclear course expectations </li></ul></ul><ul><ul><li>Exploit faculty disorganization </li></ul></ul><ul><li>Prevent Excuses </li></ul><ul><ul><li>Include clear statements on syllabus (consider syllabus quiz) </li></ul></ul><ul><ul><li>Indicate or link to institution policies </li></ul></ul><ul><ul><li>Establish or link to institution honor code </li></ul></ul><ul><ul><li>Demonstrate honesty </li></ul></ul>Effective Online Assessment Strategies
  12. 12. Continuum for Cheating <ul><li>Blaming the Technology… </li></ul><ul><ul><li>“ Virus” claim </li></ul></ul><ul><ul><li>“ Computer crashed” claim </li></ul></ul><ul><ul><li>“ Quiz did not submit” claim </li></ul></ul><ul><li>Prevent Excuses </li></ul><ul><ul><li>Include clear statements on syllabus </li></ul></ul><ul><ul><li>Shift responsibility to students (consider E-test helper guide) </li></ul></ul><ul><ul><li>Provide links to Helpdesk, tutorials, helper guides </li></ul></ul>Effective Online Assessment Strategies
  13. 13. Continuum for Cheating <ul><li>Dishonest Collaboration… </li></ul><ul><ul><li>Print the test </li></ul></ul><ul><ul><li>Split the work </li></ul></ul><ul><ul><li>Recycle papers </li></ul></ul><ul><li>Prevent Dishonest Collaboration </li></ul><ul><ul><li>Consider no-print/no-copy script (Princeton) </li></ul></ul><ul><ul><li>Use random question pools </li></ul></ul><ul><ul><li>Use assessment timers </li></ul></ul><ul><ul><li>Use SafeAssign (plagiarism detection) </li></ul></ul><ul><ul><li>Utilize proctors for high-stakes, online assessments </li></ul></ul><ul><ul><ul><li>http://www.ncta-testing.org/cctc/ </li></ul></ul></ul>Effective Online Assessment Strategies
  14. 14. Continuum for Cheating <ul><li>Exploiting the Technology… </li></ul><ul><ul><li>False identity </li></ul></ul><ul><ul><li>Print and distribute assessments </li></ul></ul><ul><ul><li>Utilize slugs and other proxies </li></ul></ul><ul><ul><li>Paper mills </li></ul></ul><ul><li>Prevent Exploitation </li></ul><ul><ul><li>Consider using systems that defeat printing and copying </li></ul></ul><ul><ul><li>Consider using secure testing software (SecureExam) </li></ul></ul><ul><ul><li>Use SafeAssign (plagiarism detection) </li></ul></ul><ul><ul><li>Do not place high-stakes assessments online </li></ul></ul>Effective Online Assessment Strategies
  15. 15. Continuum for Cheating <ul><li>Hacking the Technology… </li></ul><ul><ul><li>Packet-sniff passwords </li></ul></ul><ul><ul><li>Exploit predictable passwords </li></ul></ul><ul><ul><li>Attempt to gain access to the server itself </li></ul></ul><ul><li>Prevent Hacking </li></ul><ul><ul><li>http://www.birthindex.com </li></ul></ul><ul><ul><li>Use strong passwords </li></ul></ul><ul><ul><li>Avoid unsecured networks </li></ul></ul><ul><ul><li>Don’t rely on online grade book and backup often </li></ul></ul>Effective Online Assessment Strategies
  16. 16. Designing Objective Questions : Primer Effective Online Assessment Strategies
  17. 17. Objective Question Mistakes <ul><li>Multiple-multiple choice (cognitive load is redirected to understanding the question-choice permutations) </li></ul><ul><li>Example: When writing a learning objective, you should always include: </li></ul><ul><ul><li>An action verb </li></ul></ul><ul><ul><li>A criterion measure </li></ul></ul><ul><ul><li>The tools used </li></ul></ul><ul><ul><li>A and B </li></ul></ul><ul><ul><li>B and C </li></ul></ul><ul><ul><li>B, C, and E. </li></ul></ul>Effective Online Assessment Strategies
  18. 18. Objective Question Mistakes <ul><li>The long correct option (cues the correct choice) </li></ul><ul><li>Example: Most beginning book writers will seek out: </li></ul><ul><ul><li>Writing software </li></ul></ul><ul><ul><li>Peer review </li></ul></ul><ul><ul><li>An agent who can represent them to a publisher </li></ul></ul><ul><ul><li>Good advice </li></ul></ul>Effective Online Assessment Strategies
  19. 19. Objective Question Mistakes <ul><li>Excessively long item choices (assesses reading comprehension as much as subject matter knowledge, and it’s difficult to remember the choices) </li></ul><ul><li>Example: Used hypodermic needles should be disposed by: </li></ul><ul><ul><li>Bagging them in a leak-proof plastic bag and calling Environmental Health and Safety for Disposal. </li></ul></ul><ul><ul><li>Dropping all the disposables into plastic containers free of punctures or warps and then calling Environmental Health and Safety. </li></ul></ul><ul><ul><li>Collecting the disposables into a large metal bin that is leak-proof and free of punctures, and then calling Environmental Health and Safety </li></ul></ul><ul><ul><li>Carrying the sharp objects to a Satellite Accumulation Area and storing them into a leak-proof plastic tub, and then calling Environmental Health and Safety. </li></ul></ul>Effective Online Assessment Strategies
  20. 20. Objective Question Mistakes <ul><li>Ridiculous or obvious foils (Humorous and an anxiety reducer. However, using them basically makes your multiple-choice test item a true-false item) </li></ul><ul><li>Example: Who is best known for his/her contributions to mathematics? </li></ul><ul><ul><li>Paris Hilton </li></ul></ul><ul><ul><li>Alfred North Whitehead </li></ul></ul><ul><ul><li>Maya Angelou </li></ul></ul><ul><ul><li>Sponge Bob </li></ul></ul>Effective Online Assessment Strategies
  21. 21. Objective Question Mistakes <ul><li>Foils without parallel construction. (Items using the same grammatical structure. Cognitive resources directed to understanding the question and not applying knowledge) </li></ul><ul><li>Example : A lazy coach is a manager who: </li></ul><ul><li>Provides positive feedback but avoids negative feedback. </li></ul><ul><li>Provides negative feedback but no positive feedback. </li></ul><ul><li>When an annual review for the employee comes up, he gives feedback, but never at any other time. </li></ul><ul><li>A manager who is the type of person who never provides any feedback. </li></ul>Effective Online Assessment Strategies
  22. 22. Objective Question Mistakes <ul><li>“ All of the above” or “none of the above” choices (Avoid these foils unless you are attempting to remind the student that all/none of these apply. If the student knows one of the options to be true, it reduces her choices to that one or the all of the above choice.) </li></ul><ul><li>Example : Coaching is best done: </li></ul><ul><ul><li>During the annual review. </li></ul></ul><ul><ul><li>During the semiannual review. </li></ul></ul><ul><ul><li>When a coachable moment presents itself. </li></ul></ul><ul><ul><li>All of the above. </li></ul></ul><ul><ul><li>None of the above. </li></ul></ul>Effective Online Assessment Strategies
  23. 23. Objective Question Mistakes <ul><li>Negative questions (Readers overlook the not word in the question. Avoid these items when possible, especially if using a negative in the stem.) If negatives must be used, capitalize, underscore, bold, or otherwise highlight </li></ul><ul><li>Example : Which of the following is not recommended for the victim of an accident: </li></ul><ul><ul><li>Not leaving the victim unattended. </li></ul></ul><ul><ul><li>Not moving the victim. </li></ul></ul><ul><ul><li>Notifying the authorities </li></ul></ul><ul><ul><li>Not talking to the victim. </li></ul></ul>Effective Online Assessment Strategies
  24. 24. <ul><li>Place as much of the question in the stem as possible, rather than duplicating material in each of the options </li></ul><ul><li>Avoid using ALWAYS and NEVER in the stem. Test-wise students are likely to rule out such universal statements from consideration </li></ul><ul><li>Research supports using from three to five foils </li></ul><ul><li>Avoid clues to the correct answer in the grammatical construction of the question </li></ul>Objective Question Mistakes Effective Online Assessment Strategies
  25. 25. <ul><li>Multiple-Choice-True-False Questions. </li></ul><ul><li>Create two true and two false items, and add reasons why the item is true or false. </li></ul><ul><li>Example : The Colorado Supreme Court banned the use of cell phones in elevators: </li></ul><ul><ul><li>True, it was deemed a safety hazard. </li></ul></ul><ul><ul><li>True, it was judged a public nuisance </li></ul></ul><ul><ul><li>False, because it violated freedom of speech. </li></ul></ul><ul><ul><li>False, there was a hung jury </li></ul></ul>Test Item Alternatives Effective Online Assessment Strategies
  26. 26. <ul><li>Scenario-based . Smaller than a case study but longer than a standard multiple-choice stem, scenarios put the student into situations that they must solve. </li></ul><ul><li>Example : You have just given Ariel her performance review. You ask her to sign her evaluation ( Meets Expectations ), but she believes she is underrated. What should you do? </li></ul><ul><ul><li>Give Ariel the option of signing or being suspended for 30 days. </li></ul></ul><ul><ul><li>Note and date her disagreement in writing, and have her sign it. </li></ul></ul><ul><ul><li>Set several performance based 30-day goals that will upgrade your evaluation if she achieves them. </li></ul></ul><ul><ul><li>Verbally acknowledge her right to disagree, and indicate that you are obligated to submit the evaluation anyway. </li></ul></ul>Test Item Alternatives Effective Online Assessment Strategies
  27. 27. <ul><li>Testlets ( Context-Dependent Item Sets ) A test problem or scenario is presented and a set of related test items are asked. </li></ul><ul><li>Example : Mark is a blood lab technician who has worked at the University Health Sciences Center for the last five years. Mark has just found out he has HIV.  Is Mark covered by ADA? </li></ul><ul><ul><li>a. Yes, HIV is covered by ADA </li></ul></ul><ul><ul><li>b. Yes, as long as he is taking medicine for HIV. </li></ul></ul><ul><ul><li>c. No, only AIDS is covered by ADA, not HIV. </li></ul></ul><ul><ul><li>d. No, neither HIV nor AIDS are covered by ADA. </li></ul></ul>Test Item Alternatives Effective Online Assessment Strategies
  28. 28. <ul><li>Continued… </li></ul><ul><li>Must the university allow Mark to continue to work with blood since he has HIV? </li></ul><ul><ul><li>a. No, the university is required to reassign Mark to a position where he is not working in a medical facility. </li></ul></ul><ul><ul><li>b. No, the university is required to reassign Mark to a position where he is not working with blood. </li></ul></ul><ul><ul><li>c. Yes, unless Mark's HIV posses a demonstrable safety risk to him or others, the university must continue to allow Mark to work with blood. </li></ul></ul><ul><ul><li>d. Yes, the university cannot make HIV employees switch positions. </li></ul></ul>Test Item Alternatives Effective Online Assessment Strategies
  29. 29. <ul><li>Alternate-Choice. A two-option MC question that works better with high achieving examinees (grad students and majors) who might readily eliminate several obvious foils in a four-item MC test. Best used to generate more items than you would with a standard four item MC test. </li></ul><ul><li>Example: Which is a proper closing line for an autumn haiku? </li></ul><ul><ul><ul><li>a. An island song </li></ul></ul></ul><ul><ul><ul><li>Like a floating river </li></ul></ul></ul><ul><ul><ul><li>Rain Rain Fall Fall </li></ul></ul></ul><ul><ul><ul><li>b. An island song </li></ul></ul></ul><ul><ul><ul><li>Like a floating river </li></ul></ul></ul><ul><ul><ul><li>With falling rain a ‘quiver. </li></ul></ul></ul>Test Item Alternatives Effective Online Assessment Strategies
  30. 30. Alternative Forms of Assessment <ul><li>Self Assessment </li></ul><ul><li>Reflective Journal Writing </li></ul><ul><li>Discussions/Blogs/Wikis </li></ul><ul><li>Peer Assessment </li></ul><ul><li>Electronic Portfolios (Assess.) </li></ul><ul><li>Case Studies </li></ul><ul><li>Role Playing/Simulation/Games </li></ul><ul><li>Teach-Back/Oral Projects </li></ul><ul><li>Group Projects </li></ul>Native Blackboard Tool Self-and-Peer Assessment Baylor Journal/Learning Objects Discussion Board/LO Self-and-Peer Assessment No native Blackboard tool Student Group Tool Virtual Classroom or chat Virtual Classroom or chat Student Group Tool Effective Online Assessment Strategies
  31. 31. Contact and Conclusion <ul><li>Dr. Robert Gibson </li></ul><ul><li>Technology and Computing Services </li></ul><ul><li>Emporia State University </li></ul><ul><li>[email_address] </li></ul><ul><li>http://rob-gibson.net </li></ul>32 Effective Online Assessment Strategies

×