Successfully reported this slideshow.

Can Confidence Assessment Enhance Traditional Multiple-Choice Testing?

2

Share

Loading in …3
×
1 of 19
1 of 19

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Can Confidence Assessment Enhance Traditional Multiple-Choice Testing?

  1. 1. Can Confidence Assessment Enhance Traditional Multiple-Choice Testing? Josef Kolbitsch, Martin Ebner, Walther Nagler, Nicolai Scerbakov
  2. 2. Technology enhanced Learning http://flickr.com/photos/powerbooktrance/289992273//
  3. 3. uLearning http://www.flickr.com/photos/sovietuk/141381675
  4. 4. „Students sometimes accomplish their results of a Mutiple-Choice Test by a combination of partial knowledge and guesswork.“
  5. 5. Confidence Assessment „Students do not only have to choose an answer to a multiple-choice question, but also have to indicate how confident they feel about their given answer.“
  6. 6. additional confidence paramter
  7. 7. Lecture Design - lecture „Application of Operating and Information Systems“ - recommended to all students of Graz University of Technology - 720 students attend - 432 did the final exam
  8. 8. Test Design 100 questions were generated with following influence by confidence:
  9. 9. Interpretation of the parameter Points if Points if Confidence Interpretation correct incrorrect 100% I know it 10 -5 I am not sure, but I think 66% 7 -3 I know it I am not sure, but I 33% 3 -2 assume something I did not know it and I 0% 0 0 have to guess
  10. 10. Test Groups - Control group (260 students) - Experimental group with Confidence (172 students) - Experimental group without Confidence (172 students) - Experimental group scaled (172 students)
  11. 11. Overview about groups Minimum Maximum Range of Type of Test Score Score Points Experimental group with -50 100 150 Confidence Experimental group 0 100 100 without Confidence Experimental group scaled 0 100 100 Control group 0 100 100
  12. 12. General findings $!!quot;# ,!quot;# +!quot;# !quot;#$quot;%&'(quot;)*+),'-./0/)1$*#quot;) *!quot;# )!quot;# :72;9<2#=90>3.0?.# (!quot;# :72;#=90>3.0?.# :72;#=90>3.0?.@#1?/A.3# '!quot;# =90249A#B49<C# &!quot;# %!quot;# $!quot;# !quot;# -./0# 12/03/43#5.67/890# -.37/0# -93.#
  13. 13. Is there a difference between the results of the MCT with confidence assessment and the result of the Control Group? )!quot;!!#$ /012$345678598$ (%quot;!!#$ /012$345678598quot;$:9;<87$ (!quot;!!#$ /0124=1$345678598$ 3451>4<$?>4=@$ !quot;#$quot;%&'(quot;)*+),&-.quot;%&/) '%quot;!!#$ '!quot;!!#$ &%quot;!!#$ &!quot;!!#$ %quot;!!#$ !quot;!!#$ !*&!$#$ &&*'!$#$ '&*(!$#$ (&*)!$#$ )&*%!$#$ %&*+!$#$ +&*,!$#$ ,&*-!$#$ -&*.!$#$ .&*&!!$#$ !quot;#$quot;%&'(quot;)*+)0'123-3),$*#quot;)
  14. 14. Is there a difference between the scaled results of MCT with confidence assessment and the result of the Control Group? +!quot;!!#$ ./01$234567487$ *!quot;!!#$ ./01$234567487quot;$98:;76$ ./013<0$234567487$ )!quot;!!#$ 2340=3;$>=3<?$ !quot;#$quot;%&'(quot;)*+),&-.quot;%&/) (!quot;!!#$ '!quot;!!#$ &!quot;!!#$ %!quot;!!#$ !quot;!!#$ !,&!$#$ &%,(!$#$ (%,*!$#$ *%,-!$#$ -%,%!!$#$ !quot;#$quot;%&'(quot;)*+)0'123-3),$*#quot;)
  15. 15. Is there a relationship between the confidence parameter and the correctness of the answer? $!!quot;# ,!quot;# +!quot;# *!quot;# !quot;#$quot;%&'(quot;)*+)/%$*##quot;$&),%-.quot;#-) !quot;#$quot;%&'(quot;)*+),%-.quot;#-) )!quot;# (!quot;# '!quot;# &!quot;# %!quot;# -./012/32#456728# $!quot;# 9/3.::23;#</8=2:8# !quot;# $!!quot;# ))quot;# &&quot;# !quot;# 0*%12quot;%$quot;)3'45quot;-)
  16. 16. Discussion - Result of Experimental Group with Confidence are slightly worse than those of Control Group - Result of Experimental Group scaled are slightly better than those of Control Group - Relationship between confidence parameter and percentage of wrong answers are highly relevant
  17. 17. „By adding a confidence parameter to a MCT the result of the MCT deteriorates“
  18. 18. SOCIAL LEARNING Computer and Information Services Graz University of Technology Graz University of Technology Martin Ebner http://elearning.tugraz.at http://elearningblog.tugraz.at

×