Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Can Confidence Assessment
   Enhance Traditional
 Multiple-Choice Testing?


Josef Kolbitsch, Martin Ebner, Walther Nagler...
Technology enhanced Learning




http://flickr.com/photos/powerbooktrance/289992273//
uLearning




http://www.flickr.com/photos/sovietuk/141381675
„Students sometimes accomplish their results
 of a Mutiple-Choice Test by a combination of
    partial knowledge and guess...
Confidence Assessment



        „Students do not only have to choose an
      answer to a multiple-choice question, but a...
additional confidence paramter
Lecture Design


         - lecture „Application of Operating and
           Information Systems“
         - recommended t...
Test Design

              100 questions were generated with
               following influence by confidence:
Interpretation of the parameter


                                            Points if Points if
   Confidence        Inte...
Test Groups

        - Control group (260 students)
        - Experimental group with Confidence
          (172 students)
...
Overview about groups


                              Minimum Maximum   Range of
        Type of Test
                    ...
General findings

                               $!!quot;#


                                ,!quot;#


                  ...
Is there a difference between the results of the MCT with
confidence assessment and the result of the Control
Group?


   ...
Is there a difference between the scaled results of MCT
with confidence assessment and the result of the Control
Group?


...
Is there a relationship between the confidence parameter
and the correctness of the answer?

                             ...
Discussion

      - Result of Experimental Group with
        Confidence are slightly worse than
        those of Control ...
„By adding a
  confidence
   parameter
     to a MCT
    the result
  of the MCT
deteriorates“
SOCIAL LEARNING

Computer and Information Services
  Graz University of Technology

            Graz University of Technol...
Can Confidence Assessment Enhance Traditional Multiple-Choice Testing?
Upcoming SlideShare
Loading in …5
×

Can Confidence Assessment Enhance Traditional Multiple-Choice Testing?

2,244 views

Published on

Presentation at ICL 2008

Published in: Education, Technology
  • Login to see the comments

Can Confidence Assessment Enhance Traditional Multiple-Choice Testing?

  1. 1. Can Confidence Assessment Enhance Traditional Multiple-Choice Testing? Josef Kolbitsch, Martin Ebner, Walther Nagler, Nicolai Scerbakov
  2. 2. Technology enhanced Learning http://flickr.com/photos/powerbooktrance/289992273//
  3. 3. uLearning http://www.flickr.com/photos/sovietuk/141381675
  4. 4. „Students sometimes accomplish their results of a Mutiple-Choice Test by a combination of partial knowledge and guesswork.“
  5. 5. Confidence Assessment „Students do not only have to choose an answer to a multiple-choice question, but also have to indicate how confident they feel about their given answer.“
  6. 6. additional confidence paramter
  7. 7. Lecture Design - lecture „Application of Operating and Information Systems“ - recommended to all students of Graz University of Technology - 720 students attend - 432 did the final exam
  8. 8. Test Design 100 questions were generated with following influence by confidence:
  9. 9. Interpretation of the parameter Points if Points if Confidence Interpretation correct incrorrect 100% I know it 10 -5 I am not sure, but I think 66% 7 -3 I know it I am not sure, but I 33% 3 -2 assume something I did not know it and I 0% 0 0 have to guess
  10. 10. Test Groups - Control group (260 students) - Experimental group with Confidence (172 students) - Experimental group without Confidence (172 students) - Experimental group scaled (172 students)
  11. 11. Overview about groups Minimum Maximum Range of Type of Test Score Score Points Experimental group with -50 100 150 Confidence Experimental group 0 100 100 without Confidence Experimental group scaled 0 100 100 Control group 0 100 100
  12. 12. General findings $!!quot;# ,!quot;# +!quot;# !quot;#$quot;%&'(quot;)*+),'-./0/)1$*#quot;) *!quot;# )!quot;# :72;9<2#=90>3.0?.# (!quot;# :72;#=90>3.0?.# :72;#=90>3.0?.@#1?/A.3# '!quot;# =90249A#B49<C# &!quot;# %!quot;# $!quot;# !quot;# -./0# 12/03/43#5.67/890# -.37/0# -93.#
  13. 13. Is there a difference between the results of the MCT with confidence assessment and the result of the Control Group? )!quot;!!#$ /012$345678598$ (%quot;!!#$ /012$345678598quot;$:9;<87$ (!quot;!!#$ /0124=1$345678598$ 3451>4<$?>4=@$ !quot;#$quot;%&'(quot;)*+),&-.quot;%&/) '%quot;!!#$ '!quot;!!#$ &%quot;!!#$ &!quot;!!#$ %quot;!!#$ !quot;!!#$ !*&!$#$ &&*'!$#$ '&*(!$#$ (&*)!$#$ )&*%!$#$ %&*+!$#$ +&*,!$#$ ,&*-!$#$ -&*.!$#$ .&*&!!$#$ !quot;#$quot;%&'(quot;)*+)0'123-3),$*#quot;)
  14. 14. Is there a difference between the scaled results of MCT with confidence assessment and the result of the Control Group? +!quot;!!#$ ./01$234567487$ *!quot;!!#$ ./01$234567487quot;$98:;76$ ./013<0$234567487$ )!quot;!!#$ 2340=3;$>=3<?$ !quot;#$quot;%&'(quot;)*+),&-.quot;%&/) (!quot;!!#$ '!quot;!!#$ &!quot;!!#$ %!quot;!!#$ !quot;!!#$ !,&!$#$ &%,(!$#$ (%,*!$#$ *%,-!$#$ -%,%!!$#$ !quot;#$quot;%&'(quot;)*+)0'123-3),$*#quot;)
  15. 15. Is there a relationship between the confidence parameter and the correctness of the answer? $!!quot;# ,!quot;# +!quot;# *!quot;# !quot;#$quot;%&'(quot;)*+)/%$*##quot;$&),%-.quot;#-) !quot;#$quot;%&'(quot;)*+),%-.quot;#-) )!quot;# (!quot;# '!quot;# &!quot;# %!quot;# -./012/32#456728# $!quot;# 9/3.::23;#</8=2:8# !quot;# $!!quot;# ))quot;# &&quot;# !quot;# 0*%12quot;%$quot;)3'45quot;-)
  16. 16. Discussion - Result of Experimental Group with Confidence are slightly worse than those of Control Group - Result of Experimental Group scaled are slightly better than those of Control Group - Relationship between confidence parameter and percentage of wrong answers are highly relevant
  17. 17. „By adding a confidence parameter to a MCT the result of the MCT deteriorates“
  18. 18. SOCIAL LEARNING Computer and Information Services Graz University of Technology Graz University of Technology Martin Ebner http://elearning.tugraz.at http://elearningblog.tugraz.at

×