Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

2012 Computer-Assisted Assessment


Published on

My presentation at the 2012 CAA conference.

An attempt to summarise how all the Moodle computer-marked assessment components fit together.

  • Be the first to comment

  • Be the first to like this

2012 Computer-Assisted Assessment

  1. 1. CMA in Moodle:Past, Present and FutureTim HuntLeading Technical DeveloperThe Open University
  2. 2. Overview of this talkThis talk will focus on the ‘Present’1 Overview2 Questions3 Activities4 Conclusions
  3. 3. What are we trying to achieve?A CMA system must address• pedagogic• administrative• technicalconsiderations
  4. 4. Overview
  5. 5. Demonstration
  6. 6. Map of Moodle’s CMA tools
  7. 7. Questions
  8. 8. The quiz activity
  9. 9. The question engine
  10. 10. The question bank
  11. 11. The quiz activity
  12. 12. Administrative parts of Moodle
  13. 13. Other graded activities
  14. 14. Plugins• The various conceptual parts of Moodle are made up of -plugins -core Moodle functionality• Plugins make it easy to add custom functionality• They break down the functionality into units that can be combined in different ways
  15. 15. Questions
  16. 16. Question behaviours• How the student interacts with a question• For example 1 The Student gives a response and clicks ‘Check’ 2 They immediately get told if they were right, with feedback 3 If they were wrong, the can click ‘Try again’ 4 If they get it right on a later try, they get less marks 5 After 3 incorrect tries, they are just told the answer• Some parts of this are configurable
  17. 17. Question behaviours (to date)• Interactive with multiple tries – as on last slide• Deferred feedback• Immediate feedback• Deferred feedback with CBM• Immediate feedback with CBM• Adaptive mode• Adaptive mode (no penalties)• Manually graded
  18. 18. Question types – run-time• I described the behaviour without giving any details of the question• Question types say what the question is -what is shown -how the student responds -how it is graded -how the feedback is generated
  19. 19. Question types – authoring• Moodle questions are authored using web forms• Teachers can define the question, within the template provided by the question type• Questions are stored in Moodle’s question bank -an area for each course / quiz -hierarchical categories in each area
  20. 20. Question types (to date)• All question types combine a few basic interactions -entering words -entering numbers / symbolic notations -selection -positioning things – points, molecules, parts of a diagram• Moodle now has question types for all of these separately -… but what about combinations?
  21. 21. Question import/export formats• A way to get questions -in to Moodle from other systems -out of Moodle• Also useful for bulk authoring
  22. 22. Question display options• Information should, or should not, be visible at given times -marks -right/wrong -different types of feedback• There are settings for this• and natural limits -e.g. no feedback until an answer is submitted
  23. 23. Putting questions together• In principle, any question can use with any behaviour• In practice, it is not that simple -e.g. essays can only be manually graded• Still, many interactive questions can be assembled
  24. 24. Activities
  25. 25. The quiz – set up• Sequence of questions to be answered -specific question or random selection -fixed order or shuffled -answered in sequence, or freely• Which students should have access• When and where attempts can be made
  26. 26. Quiz access rulesAll the different ways students may, or may not, berestricted from accessing the quiz• Open / close dates• Time limit• Number of attempts• IP address• Secure browser• Enforced delays between attempts• Acknowledge academic honesty statement first
  27. 27. The quiz – results• Question outcomes combined to give overall results• Results reported• Final grades sent to the grade-book
  28. 28. Quiz reportsAllow the outcomes to be viewed, analysed and downloaded• Grades• Responses• Statistics – standard test psychometrics – CTT• Efficient manual grading
  29. 29. Lesson activity• Like a “choose your own adventure” book• Since Moodle 1.2 – not yet using the question bank/question engine
  30. 30. Other possibilities• Could be built in Moodle• Or IMS LTI could be used to connect to external tools
  31. 31. Summary
  32. 32. How does Moodle measure up?A CMA system must address• pedagogic• administrative• technicalconsiderations
  33. 33. Pedagogy• Behaviours allow a range of learning tasks -Gibbs and Simpson conditions for feedback to promote learning → Interactive behaviour -Certainty Based Marking• Question types -designed to allow detailed feedback• Options for constructing quizzes• Quiz reports
  34. 34. Administration• Moodle provides a lot of this -courses, users, groups, authentication, enrolment -course as an sequence of activities -gradebook, event calendar, notifications• Question bank• Quiz set-up UI• Quiz access rules• Quiz reports
  35. 35. Technical design• Small core + plugins• Moodle as a framework -runs on Windows/Linux/Mac, Mysql/Postgres/... -concepts of courses, users, groups, uploaded files...• Simple but flexible database design for questions• Open source project -software as as a way to capture the learning of a community of practice
  36. 36. Where should Moodle do more?• More different activities using questions -lesson should use question bank / engine -question practice -question authoring / peer review as a learning task• Better facilities for collaborative authoring of questions -sharing between quizzes / courses / sites -versioning• More flexibility for building quizzes out of questions
  37. 37. Three things to remember• Question = behaviour + type• Plugins to organise software with lots of functionality• Moodle quiz comes from 10 years of community of practice -Future contributions to the open source project welcomeDocumentation at the OU with open source software
  38. 38. Map of Moodle’s CMA tools