SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.
SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.
Successfully reported this slideshow.
Activate your 14 day free trial to unlock unlimited reading.
Can Confidence Assessment Enhance Traditional Multiple-Choice Testing?
4.
„Students sometimes accomplish their results
of a Mutiple-Choice Test by a combination of
partial knowledge and guesswork.“
5.
Confidence Assessment
„Students do not only have to choose an
answer to a multiple-choice question, but also
have to indicate how confident they feel
about their given answer.“
7.
Lecture Design
- lecture „Application of Operating and
Information Systems“
- recommended to all students of Graz
University of Technology
- 720 students attend
- 432 did the final exam
8.
Test Design
100 questions were generated with
following influence by confidence:
9.
Interpretation of the parameter
Points if Points if
Confidence Interpretation
correct incrorrect
100% I know it 10 -5
I am not sure, but I think
66% 7 -3
I know it
I am not sure, but I
33% 3 -2
assume something
I did not know it and I
0% 0 0
have to guess
10.
Test Groups
- Control group (260 students)
- Experimental group with Confidence
(172 students)
- Experimental group without
Confidence (172 students)
- Experimental group scaled (172
students)
11.
Overview about groups
Minimum Maximum Range of
Type of Test
Score Score Points
Experimental group with
-50 100 150
Confidence
Experimental group
0 100 100
without Confidence
Experimental group scaled 0 100 100
Control group 0 100 100
13.
Is there a difference between the results of the MCT with
confidence assessment and the result of the Control
Group?
)!quot;!!#$
/012$345678598$
(%quot;!!#$
/012$345678598quot;$:9;<87$
(!quot;!!#$ /0124=1$345678598$
3451>4<$?>4=@$
!quot;#$quot;%&'(quot;)*+),&-.quot;%&/)
'%quot;!!#$
'!quot;!!#$
&%quot;!!#$
&!quot;!!#$
%quot;!!#$
!quot;!!#$
!*&!$#$ &&*'!$#$ '&*(!$#$ (&*)!$#$ )&*%!$#$ %&*+!$#$ +&*,!$#$ ,&*-!$#$ -&*.!$#$ .&*&!!$#$
!quot;#$quot;%&'(quot;)*+)0'123-3),$*#quot;)
14.
Is there a difference between the scaled results of MCT
with confidence assessment and the result of the Control
Group?
+!quot;!!#$
./01$234567487$
*!quot;!!#$
./01$234567487quot;$98:;76$
./013<0$234567487$
)!quot;!!#$
2340=3;$>=3<?$
!quot;#$quot;%&'(quot;)*+),&-.quot;%&/)
(!quot;!!#$
'!quot;!!#$
&!quot;!!#$
%!quot;!!#$
!quot;!!#$
!,&!$#$ &%,(!$#$ (%,*!$#$ *%,-!$#$ -%,%!!$#$
!quot;#$quot;%&'(quot;)*+)0'123-3),$*#quot;)
15.
Is there a relationship between the confidence parameter
and the correctness of the answer?
$!!quot;#
,!quot;#
+!quot;#
*!quot;#
!quot;#$quot;%&'(quot;)*+)/%$*##quot;$&),%-.quot;#-)
!quot;#$quot;%&'(quot;)*+),%-.quot;#-)
)!quot;#
(!quot;#
'!quot;#
&!quot;#
%!quot;# -./012/32#456728#
$!quot;# 9/3.::23;#</8=2:8#
!quot;#
$!!quot;# ))quot;# &"# !quot;#
0*%12quot;%$quot;)3'45quot;-)
16.
Discussion
- Result of Experimental Group with
Confidence are slightly worse than
those of Control Group
- Result of Experimental Group scaled are
slightly better than those of Control
Group
- Relationship between confidence
parameter and percentage of wrong
answers are highly relevant
17.
„By adding a
confidence
parameter
to a MCT
the result
of the MCT
deteriorates“
18.
SOCIAL LEARNING
Computer and Information Services
Graz University of Technology
Graz University of Technology
Martin Ebner
http://elearning.tugraz.at
http://elearningblog.tugraz.at