LAK 2018 Paper - Early lessons from a pilot study
•Download as PPTX, PDF•
1 like•102 views
Quantext is a text analysis tool designed for teachers to analyze student responses from assignments. An initial pilot study involved 8 teachers from different courses who provided feedback. This informed improvements to the user interface and additional needs like integrating with learning management systems. The document outlines several use cases for Quantext, including marking responses, identifying common errors, exploring student understanding, and facilitating discussion. It also discusses ongoing challenges like accurately measuring semantic similarity between student answers.
Recommended
More Related Content
Similar to LAK 2018 Paper - Early lessons from a pilot study
Similar to LAK 2018 Paper - Early lessons from a pilot study (15)
Recently uploaded
Recently uploaded (20)
LAK 2018 Paper - Early lessons from a pilot study
- 1. Quantext Analysing student responses: early lessons from a pilot study JENNY MCDONALD, ADON C. M. MOSKAL, IRINA ELGORT & CATHY GUNN
- 2. SESSION OUTLINE WHAT IS QUANTEXT? MOTIVATION PILOT STUDY USE CASES AND WORKFLOWS CHALLENGES: SEMANTIC SIMILARITY WHERE TO FROM HERE
- 3. What is Quantext A text analytic tool designed for teachers… https://quantext.org https://github.com/quantext/quantext
- 4. MOTIVATION Probe what is happening at the site of learning¹ through the systematic analysis of teacher and student text. 1. Knight & Littleton, 2015
- 5. 8 teachers from six different courses, spanning the humanities, sciences, health sciences and commerce, and with cohorts of under 100 students to well over 1000 and including 2 MOOCs PILOT STUDY Feedback from participants to date: Informed interface development Demonstrated the need to cater for mini-essays (4/6 courses) in addition to SAQs Clear requirement for visualisation options Range of data management and interface usability improvements Integration with LMS Use case workflows ….
- 6. Use case 1 :: Marking student responses The flow of blood in the systemic and pulmonary circulations is the same. Please explain why.
- 7. Use case 1 :: Marking student responses idk ?
- 8. Use case 1 :: Marking student responses high pressure resistance
- 9. Use case 1 :: Marking student responses the resistance and pressure are the same
- 10. Use case 1 :: Marking student responses in series oxygenated blood entering
- 11. Use case 1 :: Marking student responses Because the systemic and pulmonary circuits are in series with each other, so the flow through both needs to be the same
- 12. Use case 1 :: Marking student responses I’m tired now
- 13. Use case 2 :: Identify common errors Artery walls are thick, strong and not very compliant. Why is this? (Hint: think about how the structure of the arterial wall relates to its function)
- 14. Student responses ...they have thick tunica media containing mostly smooth muscle and connective tissue which is high in collagen and elastin. ...blood … is under high pressure and flow in the arterial system is pulsatile. Student A Because they carry blood under high pressure and carry pulsatile blood. Student B Use case 2 :: Identify common errors
- 15. Lecture 10: Final lecture ...In the veins, they, they carry almost all of the volume of the blood and yet their pressure is very low. So why is there this difference between the arteries and the veins in the way that they’re working? And the reason is because the structure of the arteries and the veins is very different from one another... Use case 2 :: Identify common errors
- 16. Use case 3 :: Explore understanding Describe what is meant by total peripheral resistance
- 17. Use case 3 :: Explore understanding the resistance in the peripheral vessels that the heart has to overcome when pumping blood to ensure that the blood reaches the entire circulatory system this is the resistance to blood flow that is provided by the friction on the blood by the vessel walls total vascular resistance in the body from the arteries, arterioles and capillaries (venule and venous resistance is so low compared to arterial resistance that it is negligible)
- 18. Use case 4 :: Explore discussion
- 19. Import questions and student short answers/mini essays Quick comparison within and between cohorts Analyse in context with reference to teaching materials Label/categorise responses through iterative similarity (keyword/phrase & semantic) and sorting - multiple labels possible. Identify teaching insights Reporting and export PILOT => KEY FEATURES
- 21. SEMANTIC SIMILARITY WORD SENTENCE/FRAGMENT PARAGRAPH/RESPONSE/POST DOCUMENT ROLE INTENTION ATTENTION
- 22. Measuring semantic similarity SEMANTIC SIMILARITY heart Words represented by vectors which capture meaning in use - word embeddings pump 𝜃toe 𝜃 Cosine of angle 𝜽 between vectors
- 26. WHERE TO FROM HERE?
Editor's Notes
- Text is arguably central to teaching and learning; from deep questions designed to reveal how students understand and describe the world, to more surface-level questions relating to content knowledge. In addition, students’ communicative ability is often assessed through examination of syntax and style. Yet, particularly in large class settings, it is challenging for educators to know what and how their students are learning. The time and resource constraints of modern tertiary environments, especially in large class settings, mean it is often only at the time of marking final examinations that student understanding becomes clear to the teacher, by which time it is too late to respond to misconceptions.
- I’ve got time for one concrete example to show you what I mean....
- Conversational framework is useul because it captures the key idea which is a central dialogue between teacher-student (can also be between students etc) Nice too because it sits comfortably with some key lingustic theories – Wittgenstein – language is meaning in use and Bakhtin – the dialogic – what we say or write or speak about is always related, in a way, back to that which we have previously been exposed to..
- What have we found from pilot thus far?
- Here are the key features – you can explore yourself by going to Quantext.org – url at the end of these slides. Quantext does simple and obvious things like counting key words and identifying key word groups/phrases – also does semantic similarity using some fancier techniques – called words embeddings. Explain if time...
- Here are the key features – you can explore yourself by going to Quantext.org – url at the end of these slides. Quantext does simple and obvious things like counting key words and identifying key word groups/phrases – also does semantic similarity using some fancier techniques – called words embeddings. Explain if time...
- Here’s our development roadmap and presentations and reporting on outcome of pilot schedule for Feb March next year..