Timing is everything - Using psychology research to make your assessments more effective

Timing is Everything Using psychology research to make your assessments more effective John Kleeman Questionmark Chairman Twitter: @johnkleeman 2012 Users Conference New Orleans March 20 - 23Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Would you like to reduce learner forgetting curves? 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 2 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Agenda Introduction Spacing of learning Using spacing to make assessment more effective Retrieval practice Using retrieval practice to make assessment more effective Other interesting research – self-explanations, feedback Where to find out more Questions 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 3 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Typical Psychology Method Group A Group B Learn in Learn in one way another way Wait a Wait a period period Measure Is learning Measure learning improved? learning 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 4 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Spacing of Learning 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 5 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

SpacingMassed Spacing learning aids retention 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

One experiment : Kornell 2009  25 students learning difficult English words with flashcards  Brilliant -> Effulgent  Righteousness -> Rectitude  Massed  Learn ¼ words each day for 4 days.  Review before test  Spaced  Spend same time  Look at all words each day for 4 days  Review before test 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 7 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Results  Spacing more effective than massed / cramming  Most participants predicted the opposite 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 8 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

A classic experiment  Shebilske + colleagues, 1999  Used video game useful for pilot training  20 people learn massed  10 lessons 2 days  20 people learn spaced  10 lessons 10 days  Control for differences 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 9 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Experiment results  Massed group  Mean score 2,156 after 10 lessons  Distributed group  Mean score 3,189 after 10 lessons  48% better  Participants were tested a week later and the effect remained 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 10 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Spacing effect  Spacing  Repeat learning material, separated over time  Effective for long term retention  Cramming  Repeating learning intensely (e.g. before exam)  Effective to pass an exam, not for long term  Interleaving topics  More effective than massing them 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 11 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Using assessments to take advantage ofthe spacing effect 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Use assessments as spaced learning  Quizzes + feedback = learning  Can be spaced to give repetition  Some ideas:  Quiz of the week  Question of the day  Quiz with evaluation survey  Quizzes as post-learning reinforcement  Quizzes on mobile devices 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Use assessments to promote spaced learning  One exam at end of course  Encourages cramming  Does not measure long term retention  Tests or quizzes during course  Potentially promote revision & learning at spaced intervals  Which increase retention  Potential techniques  Mock/practice exams  Regular quizzes with small amounts of credit  Splitting up exam into series over time 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 14 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Evidence? One study from Angus & Watson 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 15 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Your questions on spacing 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 16 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Retrieval Practice 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 17 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Retrieval helps you learn  Francis Bacon, 1620 If you read anything over twenty times, you will not learn it by heart so easily as if you were to read it only ten, trying to repeat it between whiles, and when memory failed, looking at the book. 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 18 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Learning Receive Working Long term information memory memory 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 19 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Learning useful if can retrieve • Question  Research shows Cue • Other stimulus retrieval practice helps future retrieval • Retrieve from long term memory  Successful retrieval Search modifies memory making it more • Act or respond retrievable in future Act 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 20 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Roediger & Karpicke experiment 2006  120 students divided into groups randomly  Given reading comprehension passages used in TOEFL  Some students  Study for 14 mins  Some students  Study for 7 mins  Recall material for 7 mins  Compared results after 5 mins and 1 week 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 21 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

How much learners retained 90% 80% 70% 60% 50% Restudy 40% Quiz 30% 20% 10% 0% After 5 mins After 1 week 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 22 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Marsh & others 2007. Experiments with multiple choice questions 60% 50% 40% 30% No quiz Quiz 20% 10% 0% SAT II Bloom Definition Bloom Application 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 23 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Karpicke & Blunt experiment 2011  80 students in 4 groups, studying a science text  Study  Repeated Study  Concept Mapping (mind maps)  Retrieval Practice  Surveyed learners on expectation of retention  Tested a week later to see how much retained 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 24 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

How much learners retained Actual retention Learner expectation of 80% retention 70% 80% 70% 60% 60% 50% 50% 40% 40% 30% 30% 20% 20% 10% 10% 0% 0% 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 25 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Larson experiment 2009 Quizzes reduce forgetting curve In real-life, not in the laboratory Medical residents (doctors in training) 20 doctors in each group Quiz group Restudy group Day 1 Study Study Day 1 Quiz Re-study 2 weeks later Quiz Re-study 2 weeks later Quiz Re-study 6 months later Final exam Final exam 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Part of the results Quiz group Re-study group 62% 48% 44% 36% 19% Initial Quiz Quiz 6 months 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 27 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Applying retrieval practice results for assessments 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 28 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Retrieval practice key findings  Retrieval practice helps retention of learning  Applies to facts  And also concepts  Retrieval practice includes  Recall  Formative quizzes/tests  Self-assessment  Explain or read back of material  Timing of retrieval  Works best when spaced out  Strongest effect when retrieval is difficult  Feedback very helpful  Repetition of retrieval  Research suggests 5-7 spaced repetitions helpful 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 29 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Computer quizzes are effective retrieval practice because …  Learners do not realize they will forget  Quizzes encourage/force retrieval practice  Quizzes give corrective feedback  And also:  Randomize so easy to repeat  Mobile devices  Accessible  Measure that practice has happened 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 30 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

If you are a practitioner …  Give quizzes after learning sessions  Focus questions on important things to remember  Make quizzes effortful (delay & difficulty)  Question types:  Recall (e.g. short answer) best  Multiple choice work too  Include feedback, usually the correct answer  Encourage retrieval (don’t allow peek ahead)  Spacing helps significantly 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 31 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

If you are a manager …  How easy is it for learners to take quizzes?  How easy is it for your instructors to quiz learners?  Could you make it easier? 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 32 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Your questions on retrieval 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 33 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Self-explanations 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 34 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Self-explanations Learners who can self-explain something learn it better  Inferences  Explain why something is  What the principle behind an example is Atkinson + others did an experiment in mathematics learning proving self-explanation questions can help learning 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Conventional, non self-explanation question 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 36 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Self-explanation question that helps learning better 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 37 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Self-explanations  Their conclusions  More effective than other kinds of questions  Works best in disciplines like maths and science where there are clear principles  Gives good scaffolding to help people learn why and not what  Possible application  Consider questions which give the answer and ask participants to identify the reason why it is so 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 38 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Psychological Research on Feedback 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 39 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Where does feedback fit in? Current State Search / Adjust Retrieval Evaluation Response 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 40 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Roediger & Butler. Feedback on multiple choice questions Retention after one week 60% 50% 40% 30% 20% 10% 0% No quiz Quiz, no feedback Quiz, immediate Quiz, delayed feedback feedback 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 41 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Feedback tip #1 – give feedback in formative assessments  Students only know wrong if you tell them  Include correct answer  In most contexts, just telling people they are wrong doesn’t help learning  Feedback most useful on incorrect answers  Some value on correct answers, especially if learner guesses 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 42 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Feedback tip #2 – do retrieval first  Usually force people to retrieve before you give feedback  Don’t allow them to peek ahead to see the right answer  This deprives them retrieval practice  Exception when building understanding 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 43 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Feedback tip #3 – get people to pay attention  Feedback only useful if learners pay attention  Give feedback so that people will attend  Consider attention span  Vary feedback methods (e.g. graphics)  Review whether to give feedback on correct answers  Monitor effect of feedback  Keep feedback simple, clear and not too long 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 44 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Feedback tip #4 – delay feedback if you can  Slight delay on feedback is usually better than immediate feedback for retention  Probably due to spacing effect in learning  End of test / after each block feedback better than after each question feedback  Research not conclusive 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 45 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

In a Perfect World 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 46 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Learning in the Actual World They don’t get it All! 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 47 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Learning Curve Purpose: • Content repetition • Memory retrieval practice • Strengthens memory recall Ouch! The Forgetting Curve Knowledge/Skills Experience Experience Experience Questions Questions Learning Learning Time Learning 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 48 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

In the Actual World They Forget It! 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 49 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Slow the Forgetting Curve Purpose: • Memory retrieval practice • Strengthens memory recall • Reduce Forgetting Curve Knowledge/Skills Experience Experience Experience Questions Questions Questions Questions Learning Learning Time Learning 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 50 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Questions through Learning Process Reduce Forgetting Create Intrigue Knowledge/Skills Strengthen Memory Questions Questions Questions Questions Questions Time 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 51 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

“Desirable Difficulty”  Something that  Slows initial learning  Increases long-term retention and transfer  Makes you think you are learning less well, but in fact allows you to retain information  Examples  Testing rather than re-studying  Spaced learning  Interleaving tasks  Delayed feedback  Paradox : to learn better, we may need to think we are learning less well  Paradox : The best time to reinforce learning is when we are just about to forget something 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 52 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Where to find out more 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 53 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Two recommended books Building Expertise  Applying the Science of Learning Ruth Colvin Clark Richard Mayer 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Dr Will Thalheimer  Expert at translating research into practice  “Learning takes place over time—real learning doesn’t usually occur in one-time events”  Two Questionmark white papers www.questionmark.com/whitepapers  Learning Benefits of Questions  Providing Feedback to Learners  Good material on own site www.work-learning.com  Spacing Learning over Time 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 55 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

References – spacingWho Title Journal Date Description URLKornell Optimising Learning Using Appl. Cognit. Psychol 2009 Spacing expemiment Here Flashcards: Spacing Is More Effective Than CrammingShebilske, Interlesson Spacing and Task- Journal of 1999 Space Fortress experiment Has to beGoettl, Related Processing During Experimental purchasedCorrington, Complex Skill Acquisition Psychology: AppliedDayAngus & Does regular online testing British Journal of 2009 Correlation study See here for infoWatson enhance student learning in the Educational numerical sciences? Robust Technology Vol 40 No evidence from a large data set 2, 255-272Thalheimer Spacing Learning over time 2006 Good general explanation www.work- learning.com 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

References – retrieval practiceWho Title Journal Date Description URLRoegiger & Test-Enhanced Learning Psychological Science 2006 Widely cited paper showing HereKarpicke benefits of testing vs studyingMarsh, The memorial consequences Psychonomic Bulletin 2007 Shows benefits of multiple HereRoediger, of multiple-choice testing & Review choice questionsBjork&BjorkLarsen, Test-enhanced learning in medical Medical Education 2009 Shows experiment on HereButler & education. medical education andRoediger retrieval practiceRoediger & The critical role of retrieval Trends in Cognitive 2010 8 page review article on HereButler practice in long-term retention Sciences retrieval and retentionKarpicke & Retrieval Practice Produces More Science Magazine 2011 Clear paper comparing Select from hereBlunt Learning than Elaborative retrieval with study and (non-commercial) Studying with Concept Mapping mind mapsKleeman Recent cognitive psychology International CAA 2011 Some more detail from me Here research shows strongly that Conference quizzes help retain learning. What does this mean for CAA? 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

References – self-explanations & feedback Who Title Journal Date Description URL Atkinson, Renkl Transitioning From Studying Journal of 2003 Paper on self- Here & Merrill Examples to Solving Problems Educational explanations Psychology Valerie J. Shute Focus on Formative Feedback ETS (Educational 2007 45 page article giving ETS website Testing Service) advice on feedback Thalheimer Providing Learners with Questionmark 2008 Two part white paper on Questionmark Feedback white paper effective feedback website: Part 1 Part 2 Fazio, Huelser, Right/wrong feedback: Memory 2010 Research showing just Here Johnson & Consequences for learning telling students right or Marsh wrong without correct answer unhelpful 2012 Users Conference  New Orleans Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. QuestionmarkSlide 58 is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Your questions  You can also email me at john@questionmark.com or follow me on Twitter @johnkleeman 2012 Users Conference  New OrleansSlide 59

Timing is everything - Using psychology research to make your assessments more effective

by Questionmark

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 703

Statistics

Timing is everything - Using psychology research to make your assessments more effective Presentation Transcript

http://blog.questionmark.com	700
http://blogs.questionmark.com	3