Matti Pauna

1. Workshop: Automatic Assessment in
Mathematics Education
Matti Pauna, University of Helsinki, JEM
1

2. Supporting learning with web-based
practicing and assessment exercises
2

3. Outline
!! Introduction and background
!! Using web based assessment in learning
!! Mobile assessment
!! Authoring exercises and building assignments
!! Case studies and results
!! Hands on
3

4. Background
!! WebALT project 2005 – 2006
!! JEM (Joining Educational Mathematics) network 2006 –
2009
!! Coordinator: University of Helsinki, department of
Mathematics and Statistics
!! Expertise on using TEL in the field of mathematics,
focusing on the use of interactive exercises for practicing
and assessment
!! WebALT: build a multilingual database of exercises using
language independent encoding grammar (OpenMath &
GF) and tools to author exercises
4

5. linalg1:determinant(linalg2:matrix(
linalg2:matrixrow(a,b),
linalg2:matrixrow(c,d))) Linguistic hints
for mood,tense
and directive
5

6. JEM Thematic Network
quot; ! JoiningEducational
Mathematics 2006-2009
quot; ! Continues dissemination
quot; ! EU funded network
quot; ! Gathers technology
developers, authors and
users of e-content
c!

7. http://jem-thematic.net
7

8. Exercises
!! Currently implemented in MapleTA, powered by the
Maple computer algebra system
!! Different types: multiple choice, open answer, fill in
blanks…
!! Most of the exercises allow answering by a mathematical
formula whose correctness the system evaluates
!! Exercises are algorithmic and randomized
!! Problems include hints and complete solutions
!! Grading is done automatically, using Maple to evaluate
student’s response against defined criteria
8

9. Exercises
!! To assess and practice basic
mathematical skills
!! Illustrative graphs and
images
9

10. Question Banks
!! Questions are divided into
banks according to the
MathTax classification
!! Manual and ad hoc
encoding, the system does
not provide support for
sufficient metadata
!! Repository supporting
LOM was developed for
better management of
eContent
10

11. Examples of use
!! Best suited for practicing
Homework Weekly tests basic skills which are the
tools required to build
understanding and
Diagnostic Remedial problem solving
e-tests studies !! High-schools, first year
college level mathematics
Distance Virtual online
learning classes
11

12. Some benefits of the system
!! Easy to use, only web browser is needed
!! Allows individual practicing at own time, place, and
pace in a “neutral” environment
!! Grading is done automatically by the computer
which saves time of the instructors
!! Immediate feedback for students
!! Students gain understanding of their learning
process and can concentrate on the weak points
and practice more
!! Diagnostic testing helps to design the teaching, for
example, dividing students into groups according to
their individual learning needs
12

13. Exercises on Mobile Devices
Smartphones
SMS quizzes
13

14. Quizzes in iPod or Smartphones
14

15. SMS Exercises
!! Questions, answers and feedback as text messages
!! Developed a Moodle module to cater quizzes via SMS
gateway
!! For true time and place independent practicing
!! Just in time classroom quizzes
!! In many countries mobile phones are much more
common than computers with internet access
15

16. Real time quizzes with phones as personal
response system (clickers)
16

17. Real time SMS quizzes activate students
!! Immediate feedback to correct mistakes right in the
beginning
!! Raises the level of attention: students follow the
presentation more carefully knowing that they have to
answer questions after it
!! Everybody gets involved
!! Eliminates the need for special devices, which often cause
configuration and other technical problems
17

18. Authoring interactive
exercises and quizzes
Exercise structure
Algorithms
Setting up quizzes
18

19. Question components
Question text
Correct answer
Question text with
algorithmic parameters
All of these elements are instantiated from
the algorithmic parameters
19

20. Authoring questions
!! Question bank editor
!! LaTeX macros
begin{question}{MultipleChoice}
name{example 1}
qutext{Given $f(x)=(x+3)^2$, find $f(x+5)$}
choice*{$(x+8)^2$}
choice{$x^2+6x+14$}
choice{$x^2+10x+24$}
choice{none of these}
end{question}
!! Editing the source (EDU) code
20

21. Source format
21

22. 22

23. Algorithm
$a = range(-5,5);
condition:ne($a,0);
$b = range(-3,3);
condition:ne($b,0);
condition:le(abs($b-($a)),5);
$f = maple(quot;($a)*x+($b)quot;);
$p1=maple(quot;[0,($b)]quot;);
$p2=maple(quot;[0,($b)-($a)]quot;);
$p3=maple(quot;[-1, ($b)-($a)]quot;);
$y1=($b)-($a);
$y2=if(lt($b,0), quot;($b)quot;, $b);
$deltay= -($a);
$plot = plotmaple(quot;plots:-display([plot($f, x = -5 .. 5, y = -5 .. 5, color = blue,
thickness=2), plots:-coordplot(cartesian, [-5 .. 5, -5 .. 5], color = [gray, gray],
linestyle = [solid, solid], grid = [11, 11])], axes = normal, tickmarks=[11,11],
labels=[``,``], numpoints=300)quot;);
23

24. Algorithm editor
24

25. Evaluating correctness of answer
Cannot analyze student’s response, partial grading not possible
25

26. Equations
!! Encoded in MathML
1.! LaTeX converted to MathML
2.! MathML source
3.! mathml function
$Deq = mathml(“$a*x^2+$b*x+$c”);
4.! Maple call
$Drad = maple(“printf(MathML[ExportPresentation]( sqrt(a
+b)<>sqrt(a)+sqrt(b) )) ”);
5.! Equation editor
!! Displayed as images on the web page
26

27. Equation editor
27

28. Setting up assignments
!! Collect questions into quizzes
!! Set policies
28

30. Case studies
30

31. 1. Exercises for Single Variable
Calculus
!! First year university course
!! About 100 exercises for limits, differentiation, integration,
series, etc.
!! Maple T.A. exercises used as homework assignments and
also a diagnostic test as self-assessment before the
course
!! Bonus points were given to students from homeworks
and they could take the assignment any number of times,
the best attempt counted

32. !quot;#$%&'()&*+
,-./0$1/+2$%+
3%4*0*&1#+54/&*+
'.*(1&,-./+$2+
64quot;*-quot;-/+

33. 2. High School exercises
!! Exercise sets for all 10 long term math courses in
the Finnish curriculum developed since 2005
!! Algorithmic exercises for training basic methods
and techniques
!! Students do not usually practice enough which is
needed to learn mathematics and solve more
demanding problems

34. 74/&*+,-./0$1/+
2%$)+8&9.%.1'+
'$3&*/+&1+(&#(+
/*($$quot;+)4'(+

36. Essay questions
!! For expressing mathematical thinking verbally
!! For self evaluation
!! Implemented as essay questions or essay parts in
exercises and they cannot be graded automatically

38. 3. Diagnostic Testing at Polytechnic
schools
!! To assess the level of mastering basic mathematics skills
of new students, both for students themselves and
instructors
!! Problem: Students have very varied background and
teaching needs to be designed accordingly
!! A diagnostic test is organized in computer lab, first week
of the semester, a brief introduction to the system was
given
!! Algorithmic questions are different for each student, so
cheating is harder

39. Results
:&/'%&5-0$1+;<=>+?+=@>+?+A=>++$2+3$&1'/BC+
39

40. Research: Kari Lehtonen
Diagnostic tests and practicing in
technical engineering
school.
Results in the test before and
after practicing.
Purple (four groups) n = 80, not
using online assignments.
Blue group n = 40, using online
assignments.
http://jem-thematic.net/en/node/872