Bokhove, C., & Drijvers, P. (2011). Effects of feedback conditions for an online algebra tool. In M. Joubert (Ed.) Proceedings of ICTMT10. Portsmouth: ICTMT10.

1. Effects of feedback conditions for
an online algebra tool
Christian Bokhove
Paul Drijvers
Fisme, Utrecht University
ICTMT10
5-8 July 2011
1

2. Background
 Christian Bokhove
12 yr Teacher maths/ict secondary school
 St. Michael College, Zaandam, Netherlands
 Phd Dudoc
www.dudocprogramma.nl
 Supervisors Paul Drijvers &
Jan van Maanen
2

3. Problem statement
 Transition secondary  higher education
 Lack of Algebraic expertise?
 Conceptual understanding & Basic skills
 „Math wars‟
 Entry exams
 Use of ICT
 “Use to learn” vs. “Learn to use”
 Position statement NCTM (2008): ICT can be a
valuable asset
 Potential of ICT for mathematics
3

4. Theoretical framework
 Algebraic expertise
 Formative assessment
 Black & Wiliam:
 Formative: assessment for learning
 Summative: assessment of learning
 Feedback
 Hattie & Timperley
 Types
4

5. Algebraic expertise
• Arcavi
• Kop & Drijvers
• Pierce & Stacey
• (Structure sense,
e.g. Hoch & Dreyfus)
5

6. Study design
Cyclical design
1. Pre-cycle: what tool with what characteristics?
2. Cycle 1: can it work?
3. Cycle 2: in what way can it work?
4. Cycle 3: does it work and why does it work?
6

7. Design research
Cyclic design
1. Pre-cycle: what tool with what characteristics?
2. Cycle 1: can it work?
3. Cycle 2: in what way can it work?
4. Cycle 3: does it work and why does it work?
7

8. Setup of third cycle
 324 students from 9 schools (N=324)
6vwo wiskunde B
 Intervention 6 clock hours in 6 parts
 Data collection
 Scores of paper
pre-post tests
 Scores and logs
 Questionnaires
8

9. Design principles
Design principles concerning feedback:
1. Crises
2. Formative scenarios
(Timing and fading)
3. Elaborate feedback
In this presentation I will only touch on 1 & 2
and focus on the results for the third principle.
9

10. Design principle: crises
John Keats
“Failure is, in a sense, the highway to success”
 Crises of learning (Van Hiele)
 Productive failure (Kapur)
 Impasse (VanLehn)
 Doll, Piaget, VanLehn, ….
10

11. Design principle: formative scenarios
 Hattie and Timperley
 Timing and fading (Renkl et al)
11

12. Design principle: elaborate feedback
 Black and William (1998)
 Assessment for learning
 Several feedback types
(Hattie and Timperley: FT, FP, FR, FS)
12

13. “Algebra with Insight”
Digital mathematical environment
http://www.fi.uu.nl/dwo/voho , developer: Peter Boon
13

14. IDEAS
General
characteristics
Diagnostic
messages
Rewrite
rules
Buggy
rules
IDEAS: webservice by Jeuring et al.
14

15. IDEAS – stepwise – condition c1
15

16. IDEAS – buttons – condition c2
16

17. Results for elaborate feedback
 Significant difference between the feedback
conditions when we look at the score for d1
(U=7680.00, p<0.001, r=-.321) with
condition c2 scoring higher than condition
c1.
 When considering the number of
attempts, this was significantly higher for
feedback condition c1 (Mdn=126.00)
(without extra buttons) than for feedback
condition c2
(Mdn=105.00), U=9904.50, p<.001, r=.202
.
17

18. Cases
18

19. Cases – stepwise feedback (CLIP)
Browser version
19

20. Cases
20

21. Cases
21

22. Cases – hints and solve (CLIP)
Browser version
22

23. Conclusion
 Condition c2 with self-regulatory feedback
scores better, less attempts.
 Task-related and self-regulatory (FT and FR)
feedback can be used in a formative way for
the learning of algebra. Students can use
the feedback to overcome difficulties and
check whether they are on the correct
solution path or not.
23

24. Future?
we
b
24

25. Questions/discussion
 Use of feedback. When, what, how?
 Insight & skills. When is there understanding?
 Qualitative & Quantitative research. Bridge?
 Design research & Effectstudies. Cooperate?
 Methodology: how „control‟ work at home?
Christian Bokhove
Twitter: http://twitter.com/cb1601ej
cbokhove@gmail.com
www.fi.uu.nl/~christianb
www.fi.uu.nl/dwo/en and www.fi.uu.nl/dwo/voho
Project: www.algebrametinzicht.nl
25