Successfully reported this slideshow.
Upcoming SlideShare
×

# Computer Aided Assessment (CAA) for mathematics

1,240 views

Published on

Chris Sangwin & Simon Hammond

Published in: Education, Technology
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

### Computer Aided Assessment (CAA) for mathematics

1. 1. Computer Aided Assessment (CAA) for mathematics Chris Sangwin & Simon Hammond Copyright c Last Revision Date: June 1, 2009
2. 2. 2 Introduction. ... NOT multiple choice questions ... • Computer aided assessment (CAA) • CAA with computer algebra • Practical issues Implementations • Pedagogical issues • Future directions
3. 3. 3 JEM - Joining Educational Mathematics eContentPlus Thematic Network http://jem-thematic.net/ Founder members (15): Universitat Politecnica de Catalunya, Helsingin Yliopisto, Tech- nical University, Jacobs University, Universiteit van Amster- dam, University of Birmingham, FernUniversitt Hagen, Maths for More, NAG Ltd, Liguori Editore, ISN Oldenburg GmbH, RWTH Aachen University, Univ. Nacional de Educacin a Dis- tancia, Universitat Oberta de Catalunya, Universidade de Lis- boa.
4. 4. 4 Use of objective tests Consider the following question: Example question 1 Determine the following integral: cos(x) sin(2x)dx. As a multiple choice question: ◦ (2/3) cos3 (x) + C ◦ −(2/3) cos(x) + (2/3) sin3 (x) + C ◦ −(2/3) cos(x) + (1/3) sin(x) sin(2x) + C ◦ Don’t know. How do we know the students don’t diﬀerentiate the candidate solutions to check?
5. 5. 5 Computer algebra marking Computer algebra systems can be used to mark work. This checks for algebraic equivalence. (x + 1)2 ≡ x2 + 2x + 1 Useful for marking many routine problems.
6. 6. 6 Fundamental idea if simplify(sa-ta) = 0 then mark := 1 else mark := 0
7. 7. 7 STACK System overview The STACK system: • internet based CAA system, • uses very simple Maxima (computer algebra), and L TEX (type setting) A • All components open source (e.g. GPL).
8. 8. 8 Demonstrating the STACK system http://www.stack.bham.ac.uk/
9. 9. 9 In learning and teaching We are assessing a student provided answer. This is an objective test. This is • not Multiple Choice Question; • not string/regex match. Other tests for the form of an answer.
10. 10. 10 Input of mathematics This is a fundamental but unsolved problem. There are a number of options 1. Strict CAS syntax. eg. 2*(x-1)*(x+1) 2. “informal” linear text syntax. eg. 2(x-1)(x+1) x(t-1) ? 3. Graphical input tool. eg. equation editor. 4. (Pen-based input ?) 5. (Geometry applet ?) Not all groups of students are equal.
11. 11. 11 Syntax innovations Diﬃcult to achieve! Babbage 1830’s “a profusion of notations [...] which threaten, if not duly cor- rected, to multiply our diﬃculties instead of promoting our progress” Babbage, C. (1827) sin2 (x) sin−1 (x) sin sin x = sin2 x (composition)
12. 12. 12 Structure in random problem sets In practice, the numbers often do not matter. Tuckey, C. O., Examples in Algebra, Bell & Sons, London, (1904) Too much randomization destroys structure. An underlying question space.
13. 13. 13 Workshop task Option A: Context: end of ﬁrst calculus course. (Age 18) Write 6 questions which test whether a student can diﬀerentiate elementary functions. E.g. Diﬀerentiate cos(3x) with respect to x. Option B: Context: age 11. Write 6 questions which test whether a student can add frac- tions.
14. 14. 14 Randomization 1. What could you randomize? 2. What would you randomize? 3. What are some likely incorrect answers? 4. What feedback would you like to provide? ... with a view to implementing these questions live.
15. 15. 15 Issues • Well-posed questions. • Fair questions. • Structure in question sets. Schemes of work, vs isolated questions. • Algebraic form of answers as a goal.
16. 16. 16 Feedback One third of feedback interventions decreased performance. Kluger, A. N. and DeNisi, A., Psychological Bulletin (1996). The nature of feedback determines its eﬀectiveness.
17. 17. 17 Processing answers Test for algebraic equivalence if simplify(sa-ta) = 0 then mark := 1 else mark := 0 Using mainstream CAS • Get a lot very quickly, Great for calculus and beyond. • Elementary algebra can be a problem. Maxima seems to be more suitable than most.
18. 18. 18 Every CAS is diﬀerent! Input Maple Maxima Axiom (numbers) 0.5-1/2 0.0 √ 0.0 0.0 4^(1/2) √4 2 2 1 1 1 4^(-1/2) 4 √ 4 2 √2 -4^(1/2) −4 2i 2√−1 sqrt(-4) 2i 2i 2 −1 (indices) a^n*b*a^m an bam an+m b bam an (a^(1/2))^2 √a a √a (a^2)^(1/2) a2 |a| a2 (collecting terms) 1+x^2-2*x x2 − 2x + 1 x2 − 2x + 1 x2 − 2x + 1 x/3+1.5*x+1/3 1.833x + .333 · · · 1.833x + 1 3 1.833x + 0.333 · · · 5 5x 5 3*x/4+x/12 6 x 6 6 x 5 1 5 5 3/(4*x)+1/(12*x) 6 x 6x 6x
19. 19. 19 Input Maple Maxima Axiom (brackets) -1*(x+3) −x − 3 −x − 3 −x − 3 2*(x+3) 2x + 6 2(x + 3) 2x + 6 2x−1 (2*x-1)/5+(x+3)/2 9 10 x + 13 10 5 + x+3 2 9 10 13 x + 10 (x-1)^3/(x-1) (x − 1)2 (x − 1)2 x2 − 2x + 1 x2 −2x+1 x2 −2x+1 (x^2-2*x+1)/(x-1) x−1 x−1 x−1 1 9x2 +3x 9x2 +3x (9*x^2+3*x)/(3*x) 3 x 3x 3x + 1 (other) log(x^2) ln x2 2 log(x) log x2 log(x^y) ln (xy ) y log(x) log (xy ) log(exp(x)) ln (ex ) x x exp(log(x)) x x x cos(-x) cos(x) cos(x) cos(x)
20. 20. 20 Issue: technical problems • Mixed data types in polynomials x/3 + 0.5? • Unary minus (no simpliﬁcation). 1 −1 1 − , , or . 1−x 1−x x−1 • Display, 1. Implicit multiplication, (xy, x · y, x × y) 2. i vs j, √ 1 3. x vs x 2 .
21. 21. 21 Language Do we have a way to talk about these ﬁne details? Unhelpful phrases: • simplify, 1 1000 e.g.22 = 4 or 22 = · · ·? • “move over”
22. 22. 22 Checking for properties To mark Example question 2 Give an odd function. 1. calculate f (x) + f (−x), 2. simplify, 3. check equality to zero.
23. 23. 23 Creating examples/instances Some questions ask for examples of objects. They require higher level thinking. Such questions are rare. (11.5 questions from 486 ≈ 2.4%) Pointon and Sangwin, 2003 Perhaps because they are time consuming to mark. STACK may mark some questions of this style. Exemplar questions
24. 24. 24 Students’ answers Students show great variety in their answer, and method. For example, 190 students were asked for two functions that satisfy f (1) = 0. Their answers were marked automatically.
25. 25. 25 The students (N = 190) gave 93 ‘diﬀerent’ answers. Frequency Frequency 1st Answer 2nd Answer x2 − 2x 45 x3 − 3x 29 x2 2 −x 31 x2 − 2x 10 x3 x3 3 −x 11 3 −x 9 x2 − 2x + 1 7 (x − 1)2 8 x4 x2 − 2x + 3 7 4 −x 8 (x − 1)2 5 x4 − 4x 5 2x2 − 4x 5 ex−1 − x 1 x3 x2 3 − 2 5 ex−1 + e−x+1 1 0 4 ln(x) − x 1
26. 26. 26 Two strategies emerged: JL: Ok, just take the parabola and shift it one. ··· B: I said, x − 1 = 0, then integrated it.
27. 27. 27 These problems can be used to generate (short) discussions. • sorting the data, • methods used, • ‘exotic’ examples. −1 f1 (x) = 0, f2 (x) = |x|(x − 2), f3 (x) = e (x−1)2 .
28. 28. 28 Automatic feedback Sophisticated automatic feedback may be provided by computer algebra systems. This • is immediate, • is based on properties of students’ answers, • could be positive and encouraging, • may be based on common mistakes, • may be based on common misconceptions.
29. 29. 29 Common misconceptions Computer algebra can also test for a type of incorrect answer. Misconceptions may be identiﬁed by • educational research, • previous teaching experience, • examining answers from previous students
30. 30. 30 Odd functions On examining the odd functions given by students, the majority of coeﬃcients (= 1) are odd, eg 3x5 , 5x7 , 7x5 − 3x. Students’ concept image of an odd function requires odd coeﬃ- cients. Furthermore, f (x) = 0 is odd, but was absent.
31. 31. 31 Functions that are odd and even. When asked for a function that was both odd and even 35% gave the correct answer (eventually), 35% failed to answer the question. Incorrect answers revealed that 24% of the students added an odd and even function. Examples include x + x2 , x2 + x3 , x5 − x6 . The computer algebra system can test for these misconceptions.
32. 32. 32 Student feedback What do you like about the system? Did you have any diﬃcul- ties? If so please describe them. Feedback & partial credit i like the way that you are given credit if your an- swer is partially correct and also given guidance on achieving the full mark for that question. I like the fact that feedback is immediate, but I do not like the fact that if I get an answer wrong I do not know where in my working I have made the error
33. 33. 33 Random questions The questions are of the same style and want the same things but they are subtly diﬀerent which means you can talk to a friend about a certain question but they cannot do it for you. You have to work it all out for yourself which is good. Syntax problems I feel the aim system is reasonably fair, however i have lost a lot of marks in quiz 3 for simple syntax errors
34. 34. 34 Give me an example... Recognising the turning points of the functions pro- duced in question 2 was impressive, as there are a lot of functions with stationary points at x=1 and it would be diﬃcult to simply input all possibilities to be recognised as answers.
35. 35. 35 Authoring questions In authoring, there is tension: 1. Ability to use all features of CAS. 2. Ease of writing questions. Not making question authors into programmers.
36. 36. 36 Conclusion Some important questions • For what purposes is this tool useful? • What properties do we want? – Not “looks correct”. – Not “select the correct answer”. • What feedback should we give?