On-line and Objective Testing: Frequently Asked Questions and Concerns Hugh Davis and the e3an Project Team
What are the advantages?   <ul><li>Lecturers can assess students more frequently </li></ul><ul><li>Students can monitor th...
What types of question can we use?   <ul><li>Multiple Choice </li></ul><ul><li>Multiple Response </li></ul><ul><li>Text Ma...
Doesn’t it take too long to construct useful objective questions?   <ul><li>Yes if you try to do it all yourself </li></ul...
Who made the questions in the database and how can I be sure of their validity?   <ul><li>The questions were made by pract...
Who owns the copyright of questions on the database?   <ul><li>The person who wrote them retains copyright. </li></ul><ul>...
How will the questions be distributed to me?   <ul><li>The database will be available on-line – initially hosted at Southa...
What will happen to the database after the project finishes ? <ul><li>The worst case is that you will have a copy on CD, w...
Will I be able to interchange my questions between systems – or will all my work be lost when the current software goes ou...
My university QA will not allow use of objective tests  for examinations <ul><li>Start by using them for courseworks / lab...
What happens if students get hold of the whole database?   <ul><li>They might learn how to answer all the questions you wi...
The trouble with Objective Questions is that they tend to address factual content encouraging surface learning rather than...
Why are you including traditional exam style and short answer questions in the database? Surely we can’t deliver these on-...
Students may get right answer for wrong reason - reinforcing misconceptions   <ul><li>Possibly true </li></ul><ul><li>Mayb...
What happens if students guess - &quot;Multiple Guess Test&quot;   <ul><li>There is considerable research to demonstrate t...
Aren’t the students already over assessed? <ul><li>Probably they get too much  summative  assessment (things that are form...
There is too high a reliance on stability of technology! What happens if the computers fail during an exam?   <ul><li>Doin...
Do students have required IT skills?   <ul><li>To drive a browser?  </li></ul><ul><li>If they don’t, then do something abo...
Do staff have required IT skills?   <ul><li>Often not.  </li></ul><ul><li>Designing questions and setting up tests all nee...
Does my institution have enough computers?   <ul><li>Maybe not if you are trying to sit the whole first year down to forma...
It takes too long to (pre-)organize for use of on-line testing.   <ul><li>This is one of the biggest problems.  </li></ul>...
When on-line, can’t the students cheat by looking things up on the web or by conferencing with other people? <ul><li>This ...
Upcoming SlideShare
Loading in...5
×

香港六合彩

536

Published on

在秦雨向香港六合彩介绍了我和赵玉后,香港六合彩一脸色相地望着赵玉说:哟,飞哥,没想到婶子也真的很不赖嘛!
我努力抑制着自己一拳结果了香港六合彩的欲望,笑着说:是嘛?过奖过奖,地摊儿货而已.

Published in: Business, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
536
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

香港六合彩

  1. 1. On-line and Objective Testing: Frequently Asked Questions and Concerns Hugh Davis and the e3an Project Team
  2. 2. What are the advantages? <ul><li>Lecturers can assess students more frequently </li></ul><ul><li>Students can monitor their own progress </li></ul><ul><li>Students can get detailed and timely feedback </li></ul><ul><li>Speed of testing can ensure wider syllabus coverage </li></ul><ul><li>Tests can incorporate wide variety of media </li></ul><ul><li>Tests can be taken at a distance </li></ul><ul><li>Using randomized tests, tests can be taken at different times </li></ul><ul><li>Not prone to human error - objective </li></ul><ul><li>Saves staff time (in long run) </li></ul><ul><li>Diagnostic reports and analysis </li></ul><ul><li>Links to student management systems? </li></ul>
  3. 3. What types of question can we use? <ul><li>Multiple Choice </li></ul><ul><li>Multiple Response </li></ul><ul><li>Text Match </li></ul><ul><li>Fill in Blanks </li></ul><ul><li>Numeric </li></ul><ul><li>Selection (for each statement select yes/no, true/false, etc) </li></ul><ul><li>Hotspot (click in a graphic) </li></ul><ul><li>And many others…. </li></ul>
  4. 4. Doesn’t it take too long to construct useful objective questions? <ul><li>Yes if you try to do it all yourself </li></ul><ul><li>That is the advantage of a Testbank – you can start with a collection of hopefully appropriate questions </li></ul><ul><li>You can alter the questions as needed and add your own questions </li></ul>
  5. 5. Who made the questions in the database and how can I be sure of their validity? <ul><li>The questions were made by practicing academics </li></ul><ul><li>The topics to be tested within any particular theme were chosen by the theme working group </li></ul><ul><li>The questions have all been peer reviewed </li></ul><ul><li>The questions will be tested and updated as necessary </li></ul><ul><li>You can always add your own questions </li></ul>
  6. 6. Who owns the copyright of questions on the database? <ul><li>The person who wrote them retains copyright. </li></ul><ul><li>By allowing the questions into the database the owner of the copyright allows other academics to use the questions freely for the purpose of student testing </li></ul><ul><li>Questions may not be re-published or sold. </li></ul>
  7. 7. How will the questions be distributed to me? <ul><li>The database will be available on-line – initially hosted at Southampton, and accessible by registered users </li></ul><ul><li>The database will also be distributed to registered academics on CD with suitable run-time software. </li></ul><ul><li>Questions may be imported to your favourite system or printed, or viewed directly in html. </li></ul>
  8. 8. What will happen to the database after the project finishes ? <ul><li>The worst case is that you will have a copy on CD, which you may continue to grow on your own questions. </li></ul><ul><li>e3an are actively researching models for continuation </li></ul><ul><ul><li>IEE </li></ul></ul><ul><ul><li>Publisher </li></ul></ul><ul><ul><li>Subscription </li></ul></ul><ul><ul><li>Continuation funding </li></ul></ul>
  9. 9. Will I be able to interchange my questions between systems – or will all my work be lost when the current software goes out of date? <ul><li>We will be keeping the questions in an XML format (IMS QTI) </li></ul><ul><li>QTI is a generic way of representing questions </li></ul><ul><li>Many packages are implementing import QTI functionality </li></ul><ul><li>New packages are likely to use QTI as their data format. </li></ul>
  10. 10. My university QA will not allow use of objective tests for examinations <ul><li>Start by using them for courseworks / labs / example classes </li></ul><ul><li>Use them for formative assessments </li></ul><ul><li>Move to a more realistic university ;-) </li></ul>
  11. 11. What happens if students get hold of the whole database? <ul><li>They might learn how to answer all the questions you wished to ask. Would this be so bad? </li></ul><ul><li>Realistically there are too many questions for this to be an issue – it would be more of a problem if they got hold of the exact test you were about to set. </li></ul><ul><li>Most packages allow you to randomise the numeric data in a question or to randomise the questions themselves (order of answers – or selection of questions) </li></ul>
  12. 12. The trouble with Objective Questions is that they tend to address factual content encouraging surface learning rather than underlying concepts and it is difficult to test deeper understanding (synthesis, analysis, evaluation). <ul><li>Maybe true (although there is much in the literature attempting to demonstrate otherwise) </li></ul><ul><li>But how much of your current assessment deals with higher order skills? </li></ul><ul><li>Objective tests may only be part of the total assessment strategy </li></ul>
  13. 13. Why are you including traditional exam style and short answer questions in the database? Surely we can’t deliver these on-line? <ul><li>Not everything is easily tested using objective tests </li></ul><ul><li>Students may benefit from practice on exams </li></ul><ul><li>Staff may benefit by looking at other people’s exams </li></ul><ul><li>Although we cannot mark these questions automatically, we can use them to set example sheets – and release the example answers at a suitable time, all on-line if we wish. </li></ul>
  14. 14. Students may get right answer for wrong reason - reinforcing misconceptions <ul><li>Possibly true </li></ul><ul><li>Maybe this happens in other assessments too? </li></ul><ul><li>Hopefully the larger number of questions will lead to the same learning matter being tested in more than one way. </li></ul>
  15. 15. What happens if students guess - &quot;Multiple Guess Test&quot; <ul><li>There is considerable research to demonstrate that marks gained from such tests correlate very well with marks gained from other assessment methods </li></ul><ul><li>If the test is well designed guessers will end up getting 1 in n questions right – where n is the number of choices available </li></ul><ul><li>It may be useful to scale the final marks in someway to remove the 20 – 25% that might be available by guessing – but not much evidence to support this </li></ul><ul><li>Negative marking wrong answers is generally discredited. </li></ul>
  16. 16. Aren’t the students already over assessed? <ul><li>Probably they get too much summative assessment (things that are formally marked) </li></ul><ul><li>But how much formative assessment (where they can check their own progress and get feedback ) do they get? </li></ul><ul><li>Objective tests are excellent for allowing students (and staff) to check their progress at timely moments during a course – in time to allow action to be taken. </li></ul>
  17. 17. There is too high a reliance on stability of technology! What happens if the computers fail during an exam? <ul><li>Doing formal examinations at computers may be problematic for all sorts of reasons – although there are institutions that are brave enough to do this. </li></ul><ul><li>Multiple Choice questions on paper along with an optical mark reading package to scan the results introduces some extra effort, but probably worthwhile for formal examinations. </li></ul><ul><li>For less formal testing situations, most delivery systems allow for a test to be restarted where technology has failed. </li></ul><ul><li>The gains of using technology generally outweigh the problems </li></ul>
  18. 18. Do students have required IT skills? <ul><li>To drive a browser? </li></ul><ul><li>If they don’t, then do something about it quick! </li></ul><ul><li>Some familiarity with the delivery environment will generally assist the students feel comfortable and thus to optimise their performance </li></ul>
  19. 19. Do staff have required IT skills? <ul><li>Often not. </li></ul><ul><li>Designing questions and setting up tests all need IT skills (in addition to assessment skills) that need staff to be IT literate </li></ul><ul><li>In places where large scale CAA is successfully used there is generally some in house technical assistance available specifically for this purpose </li></ul><ul><li>Staff are intelligent. They will learn if they think it will benefit them. </li></ul>
  20. 20. Does my institution have enough computers? <ul><li>Maybe not if you are trying to sit the whole first year down to formal exams at the same time (do you have enough invigilators either?) </li></ul><ul><li>Is it essential that the whole cohort do your test at the same time? – do they do their courseworks at the same time? </li></ul>
  21. 21. It takes too long to (pre-)organize for use of on-line testing. <ul><li>This is one of the biggest problems. </li></ul><ul><li>In this area, significant pre-organisation can pay back significantly when it comes to assessment while also improving feedback to the student </li></ul><ul><li>But you have to do the work up-front. </li></ul>
  22. 22. When on-line, can’t the students cheat by looking things up on the web or by conferencing with other people? <ul><li>This situation is no different to most courseworks, labs and project work we set. Perhaps it is even desirable sometimes! </li></ul><ul><li>Secure browsers that prevent access to other sites and other software are available where proof of individual effort is required </li></ul><ul><li>But can you be sure that they do not have someone sitting beside them when they do the test – especially distance learners? </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×