Conﬁdence: Analysis of the conﬁdence data indicates a range of behaviours
from students having low conﬁdence in correct answers indica8ng
uncertainty in knowledge or ability, through to students with high
conﬁdence incorrect answers, exemplifying the Dunning-Kruger Eﬀect. The
advantage of a diagnos8c test may be to encourage these students to
confront their level of knowledge.
In the 2nd year diagnos8c test, ques8on 5 was answered correctly by 26
students and was based on VSEPR. The frequency of points on the
conﬁdence scale is similar and in the moderately conﬁdent range (3 – 5). It
diﬀers for low and high conﬁdence depending on whether the ques8on was
answered correctly or incorrectly. Ques8on 10 was answered correctly by
the majority of students, but has a very diﬀerent proﬁle to Ques8on 5.
Changing how we write exam ques6ons: Construc8ng correct structures is
viewed as an essen8al process that ‘expert’ chemists do to understand a
molecule. It is also viewed as one that is more cogni8vely demanding and
may involve several steps: iden8fy bonding and non-bonding electrons,
create a 2-D representa8on, translate that into a 3-D representa8on2,
determine whether some subs8tuents will be ‘diﬀerent’ (e.g. axial and
equatorial subs8tuents), iden8fy any symmetry the molecule may have. By
separa8ng the molecular structure iden8ﬁca8on step from the mNMR
thinking step in examina8on ques8ons, students are less likely to be
overwhelmed with extraneous informa8on while s8ll working through the
demands of structure determina8on. This also allows ease of applica8on of
an ‘error carried’ approach to marking.
Old Style: Predict the theore8cal low temperature 19F NMR spectrum of PF5.
Revised Style: (a) Use VSEPR theory to draw the structure of PF5. [1 mark]
(b) : Predict the theore8cal low temperature 19F NMR spectrum of of PF5.
Diagnosing Alterna8ve Concep8ons in NMR
Eilesh Tolly-Brewster, Sam Goodwin and Katherine Haxton*
School of Chemical & Physical Sciences, Keele University
Introduc6on: diagnos8c tests can provide insight into prior knowledge and alterna8ve concep8ons held by students before teaching. This can then be useful in
direc8ng students towards addi8onal learning resources, and tailoring taught sessions to the level of the students. Mul8ple choice ques8ons (MCQs) can be
carefully designed to probe common errors or alterna8ve concep8ons, and in some cases the ability of the student to apply their prior knowledge to unfamiliar
scenarios through recall or extrapola8on1. Our 1st year diagnos8c test covers introductory topics in spectroscopy including basic NMR and is intended to
evaluate the prior learning of the students, as well as probing some known alterna8ve concep8ons that they may hold from pre-university teaching. Our 2nd
year diagnos8c test covers VSEPR (mainly hypervalent p-block compounds) and NMR (I=1/2 nuclei), aimed at students ajer 2nd year organic NMR teaching but
before inorganic mul8nuclear NMR teaching. The 2nd year test includes ques8ons that require the students to apply their knowledge of NMR to unfamiliar
scenarios, and is ideally placed to iden8fy alterna8ve concep8ons arising from 1st year teaching.
Methodology: Alterna8ve concep8ons and common errors in understanding
and applying NMR theories have been iden8ﬁed by:
- analysis of wriken exam answers
- focus groups and interviews
- analysis of free text answers to preliminary diagnos8c test
MCQs have been created to test speciﬁc alterna8ve concep8ons. Each
distractor is mapped onto a speciﬁc alterna8ve concep8on or common error.
Ques8ons range from straighlorward tests of knowledge (familiar, recall),
through to ques8ons that test conceptual understanding1 (unfamiliar,
MCQs are coupled with free-text space to explain answers/show working,
and a 7-point conﬁdence scale from ‘not conﬁdent’ through to ‘highly
conﬁdent’. Analysis of free-text comments allows for greater understanding
of any issues, and whether the ques8ons are themselves confusing.
Common Errors: Analysis of 2nd year mul8nuclear exam answers to
categorise common errors was undertaken and 12 scripts analysed, reasons
for lost marks recorded by category of error. ‘Other’ refers to an error that
was speciﬁc to the design of the ques8on so would unlikely to be seen again.
‘Omission’ refers to not answering part of the ques8on. All answers analysed
had mul8ple errors.
The majority of errors came from incorrect molecular structures, highligh8ng
that molecular shape remains a barrier to student learning at higher levels of
studies2. Some integra8on and splinng errors were directly linked to
incorrect molecular structure, others were incorrect for the postulated
2nd Year Diagnos6c Test (DT): The rela8onship between DT results and exam
ques8on marks was probed (Graph 2). Four groups of students were
iden8ﬁed based on the magnitude and direc8on of change in marks. Groups
C and D indicate students showing improvement from DT to exam, with D
(increase >37%) showing par8cularly good achievement in the exam. B
represents students with limited change in mark (±7%), and A represents
students who’s marks decline from DT to exam which may be for a variety of
reasons not necessarily related to the content. The range in marks is such
that the DT is of limited prognos8c value, however plans for future years
involve direc8ng students with low DT scores to addi8onal learning
1. Holme, Luxford and Brandriet, J. Chem. Educ., 2015, 92, 1477- 1483
2. Tiekmeyer et al, J. Chem. Educ. 2017, 94, 282
Online Material: want to read more or try some
diagnos8c test ques8ons?
Please visit: bit.ly/Diagnos6cNMR
0 20 40 60 80 100
Exam Ques6on Score
How Much People Think They Know How Much People Know
Correct Answer Learning?
1 2 3 4 5 6 7
Q5 (26/75 correct) Q10 (61/75 correct)
1 2 3 4 5 6 7
Q5 Incorrect Q5 correct
0 20 40 60 80
Omission (part of ques8on)
Other (ques8on design issue)
% of errors in answers analysed