JEM - Joining Educational Mathematics
eContentPlus Thematic Network
Founder members (15):
Universitat Politecnica de Catalunya, Helsingin Yliopisto, Tech-
nical University, Jacobs University, Universiteit van Amster-
dam, University of Birmingham, FernUniversitt Hagen, Maths
for More, NAG Ltd, Liguori Editore, ISN Oldenburg GmbH,
RWTH Aachen University, Univ. Nacional de Educacin a Dis-
tancia, Universitat Oberta de Catalunya, Universidade de Lis-
Use of objective tests
Consider the following question:
Example question 1
Determine the following integral:
As a multiple choice question:
◦ (2/3) cos3 (x) + C
◦ −(2/3) cos(x) + (2/3) sin3 (x) + C
◦ −(2/3) cos(x) + (1/3) sin(x) sin(2x) + C
◦ Don’t know.
How do we know the students don’t diﬀerentiate the
candidate solutions to check?
Computer algebra marking
Computer algebra systems can be used to mark work.
This checks for algebraic equivalence.
(x + 1)2 ≡ x2 + 2x + 1
Useful for marking many routine problems.
if simplify(sa-ta) = 0 then
mark := 1 else mark := 0
The STACK system:
• internet based CAA system,
• uses very simple
Maxima (computer algebra), and
L TEX (type setting)
• All components open source (e.g. GPL).
Demonstrating the STACK system
In learning and teaching
We are assessing a student provided answer.
This is an objective test.
• not Multiple Choice Question;
• not string/regex match.
Other tests for the form of an answer.
Input of mathematics
This is a fundamental but unsolved problem.
There are a number of options
1. Strict CAS syntax. eg. 2*(x-1)*(x+1)
2. “informal” linear text syntax. eg. 2(x-1)(x+1)
3. Graphical input tool. eg. equation editor.
4. (Pen-based input ?)
5. (Geometry applet ?)
Not all groups of students are equal.
Diﬃcult to achieve!
“a profusion of notations [...] which threaten, if not duly cor-
rected, to multiply our diﬃculties instead of promoting our progress”
Babbage, C. (1827)
sin2 (x) sin−1 (x)
sin sin x = sin2 x
Structure in random problem sets
In practice, the numbers often do not matter.
Tuckey, C. O., Examples in Algebra, Bell & Sons, London, (1904)
Too much randomization destroys structure.
An underlying question space.
Context: end of ﬁrst calculus course. (Age 18)
Write 6 questions which test whether a student can diﬀerentiate
E.g. Diﬀerentiate cos(3x) with respect to x.
Context: age 11.
Write 6 questions which test whether a student can add frac-
1. What could you randomize?
2. What would you randomize?
3. What are some likely incorrect answers?
4. What feedback would you like to provide?
... with a view to implementing these questions live.
• Well-posed questions.
• Fair questions.
• Structure in question sets.
Schemes of work, vs isolated questions.
• Algebraic form of answers as a goal.
One third of feedback interventions decreased performance.
Kluger, A. N. and DeNisi, A., Psychological Bulletin (1996).
The nature of feedback determines its eﬀectiveness.
Test for algebraic equivalence if simplify(sa-ta) = 0 then
mark := 1 else mark := 0
Using mainstream CAS
• Get a lot very quickly,
Great for calculus and beyond.
• Elementary algebra can be a problem.
Maxima seems to be more suitable than most.
Every CAS is diﬀerent!
Input Maple Maxima Axiom
√ 0.0 0.0
4^(1/2) √4 2 2
1 1 1
4 2 √2
-4^(1/2) −4 2i 2√−1
sqrt(-4) 2i 2i 2 −1
a^n*b*a^m an bam an+m b bam an
(a^(1/2))^2 √a a √a
(a^2)^(1/2) a2 |a| a2
1+x^2-2*x x2 − 2x + 1 x2 − 2x + 1 x2 − 2x + 1
x/3+1.5*x+1/3 1.833x + .333 · · · 1.833x + 1
1.833x + 0.333 · · ·
5 5x 5
x 6 6
5 1 5 5
3/(4*x)+1/(12*x) 6 x 6x 6x
Input Maple Maxima Axiom
-1*(x+3) −x − 3 −x − 3 −x − 3
2*(x+3) 2x + 6 2(x + 3) 2x + 6
x + 13
x + 10
(x-1)^3/(x-1) (x − 1)2 (x − 1)2 x2 − 2x + 1
x2 −2x+1 x2 −2x+1
(x^2-2*x+1)/(x-1) x−1 x−1
1 9x2 +3x 9x2 +3x
(9*x^2+3*x)/(3*x) 3 x 3x
3x + 1
log(x^2) ln x2 2 log(x) log x2
log(x^y) ln (xy ) y log(x) log (xy )
log(exp(x)) ln (ex ) x x
exp(log(x)) x x x
cos(-x) cos(x) cos(x) cos(x)
Issue: technical problems
• Mixed data types in polynomials
x/3 + 0.5?
• Unary minus (no simpliﬁcation).
1 −1 1
− , , or .
1−x 1−x x−1
1. Implicit multiplication, (xy, x · y, x × y)
2. i vs j,
3. x vs x 2 .
Do we have a way to talk about these ﬁne details?
e.g.22 = 4 or 22 = · · ·?
• “move over”
Checking for properties
Example question 2
Give an odd function.
1. calculate f (x) + f (−x),
3. check equality to zero.
Some questions ask for examples of objects.
They require higher level thinking.
Such questions are rare. (11.5 questions from 486 ≈ 2.4%)
Pointon and Sangwin, 2003
Perhaps because they are time consuming to mark.
STACK may mark some questions of this style.
Students show great variety in their answer, and method.
For example, 190 students were asked for two functions that
satisfy f (1) = 0.
Their answers were marked automatically.
Two strategies emerged:
JL: Ok, just take the parabola and shift it one.
B: I said, x − 1 = 0, then integrated it.
These problems can be used to generate (short) discussions.
• sorting the data,
• methods used,
• ‘exotic’ examples.
f1 (x) = 0, f2 (x) = |x|(x − 2), f3 (x) = e (x−1)2 .
Sophisticated automatic feedback may be provided by
computer algebra systems.
• is immediate,
• is based on properties of students’ answers,
• could be positive and encouraging,
• may be based on common mistakes,
• may be based on common misconceptions.
Computer algebra can also test for a type of incorrect answer.
Misconceptions may be identiﬁed by
• educational research,
• previous teaching experience,
• examining answers from previous students
On examining the odd functions given by students,
the majority of coeﬃcients (= 1) are odd,
3x5 , 5x7 , 7x5 − 3x.
Students’ concept image of an odd function requires odd coeﬃ-
Furthermore, f (x) = 0 is odd, but was absent.
Functions that are odd and even.
When asked for a function that was both odd and even
35% gave the correct answer (eventually),
35% failed to answer the question.
Incorrect answers revealed that 24% of the students added an
odd and even function.
x + x2 , x2 + x3 , x5 − x6 .
The computer algebra system can test for these misconceptions.
What do you like about the system? Did you have any diﬃcul-
ties? If so please describe them.
Feedback & partial credit
i like the way that you are given credit if your an-
swer is partially correct and also given guidance on
achieving the full mark for that question.
I like the fact that feedback is immediate, but I do
not like the fact that if I get an answer wrong I do
not know where in my working I have made the error
The questions are of the same style and want the
same things but they are subtly diﬀerent which means
you can talk to a friend about a certain question but
they cannot do it for you. You have to work it all
out for yourself which is good.
I feel the aim system is reasonably fair, however i
have lost a lot of marks in quiz 3 for simple syntax
Give me an example...
Recognising the turning points of the functions pro-
duced in question 2 was impressive, as there are a
lot of functions with stationary points at x=1 and
it would be diﬃcult to simply input all possibilities
to be recognised as answers.
In authoring, there is tension:
1. Ability to use all features of CAS.
2. Ease of writing questions.
Not making question authors into programmers.
Some important questions
• For what purposes is this tool useful?
• What properties do we want?
– Not “looks correct”.
– Not “select the correct answer”.
• What feedback should we give?