Computer based testing vs. paper based testing Nynke R.Bos,Martijn van der Weerd, Roel Sijstermans Academic Medical Center – University of Amsterdam The NetherlandsBackgroundThe ‘Klinische Lijn’ (KL) exam is the final exam students have to passbefore they start their internship. The KL is a paper based examconsisting of 130 case based questions. Developing a computer basedversion of the KL offers the possibility of adding multimedia to the KLexam. Also, the organization of the KL is a complex logistical process,since eight different medical specialties are involved in providing andgrading the questions. Digitalizing the KL would therefore reducelogistical problems.MethodsThe first step in digitalizing the KL is to establish whether there are differences between setting a paper based testand a computer based test. 187 students were assigned to do half of the test on computer and the other half onpaper. Group A (n=99) did the first part on the computer and the second half on paper. Group B (n=88) did the firstpart on paper and the second part on the computer. The other 189 students did the whole test on paper at adifferent location, due to a shortage of available computers.A comparison was made on word count, but only with those questions that needed an elaborative answer. Also, thep-values for each version and each question were calculated to establish whether there was a version of the testthat proved easier. Results During the exam, students had the choice to start with either the digital or the paper part of the test. Nearly all students chose to start with the digital form. 69.8% of the total number of students (n=376) passed the KL exam. The word count proved that students used more words in the digital part of the exam than in the paper based version (36.2 ± 15.5 vs. 22.3 ± 4.9) (P<.05) An increased word usage would suggest a higher p-value for the digital test. The average p-values are more or less the same suggesting that a higher word usage on the computer does not affect the difficulty level of the computer based test. (0.60 ± 0.18 vs. 0.59 ± 0.17) Some p-values of the paper version made by group B were higher compared to those of group A and the control group. Students in group B started with the second part of the test, which might explain this difference.ConclusionStudents use more words on a computer based test than on a paper based test. No significant differences werefound between the p-values, making each test evenly difficult.Students seemed to prefer the digital settings over the paper based version. It is important that the sequence of thequestions is the same for all students, in order to have equal p-values for all questions.