THE SOLUTION Low-stakes, Online, Repeatable Learning Checks
WHY?• Online assessment • K-12, and other standardized tests already online (Bennett, 2002) • Easier access for students (Brooks, Nolan, & Gallagher, 2003) • Can be used to enhances self-regulated learning (Nicol & Milligan, 2006) • Can be used to remediate deficiencies, automate homework grading, and guide study time toward problem solving (Toback, Mershin, & Novikova, 2005)
WHY?• Repeatable quizzing • Keller Plans (1968) • Personalized Systems of Instruction = UDL (Brothen & Wambach, 2003) • Found most valuable tool in online microeconomics course (Brown & Liedholm, 2004) • Encourages mastery learning (Brooks, Nolan, & Gallagher, 2003) • Mastery learning extremely successful (Bloom, 1976) • Formative assessment with feedback promotes student learning (Nicol & Milligan, 2006) • Shown to aid in decreasing DFW rates (Twigg, 2004)
DECISIONS TO MAKE• Give correct answer or not?• With or without feedback?• Graded or ungraded?• Timed or untimed?• Number of questions each time?• Limited or unlimited number of attempts?
STEPS1. Build item pool2. Enter item pool into Blackboard3. Set up repeatable quiz parameters on Blackboard Let Blackboard to the rest!
SIDE NOTE…Most instructors and test bank writers do NOT write good test questions & Many students are test savvy
SIDE NOTE…• Some hints for writing good test questions • Align the questions with learning outcomes • Match the question type to the Bloom‟s taxonomy level • Avoid using negatives in the question • Avoid giving information in the stem that can give the answer to another question • Avoid clues in the stem (e.g., grammatical)
SIDE NOTE…• General test question patterns that test savvy students pick up on • The middle answer (or C) is usually correct • The longer answer is usually correct • The „different‟ answer is usually correct • “All of the above” or “None of the above” is usually the correct answer
LESSONS LEARNED• Use Respondus to forbid screen capturing• Shorten time for quizzes• Limit number of attempts• Larger item pools are better• Simple item analysis limitations• Scores positively correlated with effort (Johnson, Joyce, & Sen, 2002)