Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Haxton diagnostic #ViCEPHEC17


Published on

Poster for Variety in Chemistry Education 2017 on Diagnostic tests for aiding teaching of NMR. See related blog post:

Published in: Education
  • Be the first to comment

  • Be the first to like this

Haxton diagnostic #ViCEPHEC17

  1. 1. Confidence: Analysis of the confidence data indicates a range of behaviours from students having low confidence in correct answers indica8ng uncertainty in knowledge or ability, through to students with high confidence incorrect answers, exemplifying the Dunning-Kruger Effect. The advantage of a diagnos8c test may be to encourage these students to confront their level of knowledge. In the 2nd year diagnos8c test, ques8on 5 was answered correctly by 26 students and was based on VSEPR. The frequency of points on the confidence scale is similar and in the moderately confident range (3 – 5). It differs for low and high confidence depending on whether the ques8on was answered correctly or incorrectly. Ques8on 10 was answered correctly by the majority of students, but has a very different profile to Ques8on 5. Changing how we write exam ques6ons: Construc8ng correct structures is viewed as an essen8al process that ‘expert’ chemists do to understand a molecule. It is also viewed as one that is more cogni8vely demanding and may involve several steps: iden8fy bonding and non-bonding electrons, create a 2-D representa8on, translate that into a 3-D representa8on2, determine whether some subs8tuents will be ‘different’ (e.g. axial and equatorial subs8tuents), iden8fy any symmetry the molecule may have. By separa8ng the molecular structure iden8fica8on step from the mNMR thinking step in examina8on ques8ons, students are less likely to be overwhelmed with extraneous informa8on while s8ll working through the demands of structure determina8on. This also allows ease of applica8on of an ‘error carried’ approach to marking. Old Style: Predict the theore8cal low temperature 19F NMR spectrum of PF5. [6 marks] Revised Style: (a) Use VSEPR theory to draw the structure of PF5. [1 mark] (b) : Predict the theore8cal low temperature 19F NMR spectrum of of PF5. [5 marks} Diagnosing Alterna8ve Concep8ons in NMR Eilesh Tolly-Brewster, Sam Goodwin and Katherine Haxton* @kjhaxton School of Chemical & Physical Sciences, Keele University Introduc6on: diagnos8c tests can provide insight into prior knowledge and alterna8ve concep8ons held by students before teaching. This can then be useful in direc8ng students towards addi8onal learning resources, and tailoring taught sessions to the level of the students. Mul8ple choice ques8ons (MCQs) can be carefully designed to probe common errors or alterna8ve concep8ons, and in some cases the ability of the student to apply their prior knowledge to unfamiliar scenarios through recall or extrapola8on1. Our 1st year diagnos8c test covers introductory topics in spectroscopy including basic NMR and is intended to evaluate the prior learning of the students, as well as probing some known alterna8ve concep8ons that they may hold from pre-university teaching. Our 2nd year diagnos8c test covers VSEPR (mainly hypervalent p-block compounds) and NMR (I=1/2 nuclei), aimed at students ajer 2nd year organic NMR teaching but before inorganic mul8nuclear NMR teaching. The 2nd year test includes ques8ons that require the students to apply their knowledge of NMR to unfamiliar scenarios, and is ideally placed to iden8fy alterna8ve concep8ons arising from 1st year teaching. Methodology: Alterna8ve concep8ons and common errors in understanding and applying NMR theories have been iden8fied by: - analysis of wriken exam answers - focus groups and interviews - analysis of free text answers to preliminary diagnos8c test MCQs have been created to test specific alterna8ve concep8ons. Each distractor is mapped onto a specific alterna8ve concep8on or common error. Ques8ons range from straighlorward tests of knowledge (familiar, recall), through to ques8ons that test conceptual understanding1 (unfamiliar, extrapola8on). MCQs are coupled with free-text space to explain answers/show working, and a 7-point confidence scale from ‘not confident’ through to ‘highly confident’. Analysis of free-text comments allows for greater understanding of any issues, and whether the ques8ons are themselves confusing. Common Errors: Analysis of 2nd year mul8nuclear exam answers to categorise common errors was undertaken and 12 scripts analysed, reasons for lost marks recorded by category of error. ‘Other’ refers to an error that was specific to the design of the ques8on so would unlikely to be seen again. ‘Omission’ refers to not answering part of the ques8on. All answers analysed had mul8ple errors. The majority of errors came from incorrect molecular structures, highligh8ng that molecular shape remains a barrier to student learning at higher levels of studies2. Some integra8on and splinng errors were directly linked to incorrect molecular structure, others were incorrect for the postulated structure. 2nd Year Diagnos6c Test (DT): The rela8onship between DT results and exam ques8on marks was probed (Graph 2). Four groups of students were iden8fied based on the magnitude and direc8on of change in marks. Groups C and D indicate students showing improvement from DT to exam, with D (increase >37%) showing par8cularly good achievement in the exam. B represents students with limited change in mark (±7%), and A represents students who’s marks decline from DT to exam which may be for a variety of reasons not necessarily related to the content. The range in marks is such that the DT is of limited prognos8c value, however plans for future years involve direc8ng students with low DT scores to addi8onal learning resources. References: 1. Holme, Luxford and Brandriet, J. Chem. Educ., 2015, 92, 1477- 1483 2. Tiekmeyer et al, J. Chem. Educ. 2017, 94, 282 Online Material: want to read more or try some diagnos8c test ques8ons? Please visit: 0 10 20 30 40 50 60 70 80 90 100 0 20 40 60 80 100 DT Score Exam Ques6on Score A B C D How Much People Think They Know How Much People Know Dunning-Kruger Effect High Confidence, Wrong Answer Low Confidence, Correct Answer Increasing Confidence, Correct Answer Learning? 0 10 20 30 1 2 3 4 5 6 7 % Students Q5 (26/75 correct) Q10 (61/75 correct) 0 10 20 30 40 1 2 3 4 5 6 7 % students Q5 Incorrect Q5 correct 0 20 40 60 80 VSEPR Splinng Integra8on Omission (part of ques8on) Other (ques8on design issue) % of errors in answers analysed