The document summarizes research on the effectiveness of online summer course programmes in mathematics and economics at Maastricht University. It presents findings on: 1) Students' prior knowledge and participation rates in the online courses. 2) The online teaching models used and student evaluations. 3) Pass rates and exam performance correlated with course participation. 4) Appropriateness of individual vs. collaborative learning approaches. The results suggest that the online summer courses helped improve student performance and preparedness for their studies. However, questions remain about the need for such courses and the role of online learning environments.

1. Effectiveness of an Online Summer course Programme
13th Edineb Congress
Thursday 15th of June 2005, Lisbon, 9.00-10.30
http://www.web-spijkeren.nl
Bart Rienties (University Maastricht)
b.rienties@algec.unimaas.nl
Dirk Tempelaar
d.tempelaar@ke.unimaas.nl
Joost Dijkstra
Joost.dijkstra@educ.unimaas.nl
Martin Rehm
m.rehm@algec.unimaas.nl
Geke Blok
g.blok@educ.unimaas.nl
22/05/2008

2. Prior Knowledge (UCM)
54% 32% Proportions and percents
23%
Signed numbers and variables : difficult items
43% 27% 27%
22/05/2008

3. Agenda
1. Introduction
2. Online remedial teaching model
3. Methodology and research design
4. Evaluation of courses
5. Study success
6. Didactical scenarios
7. Discussion
22/05/2008

4. Introduction - Assumptions
• Increasing internationalization of enrollment
(60% of students is foreign)
• Introduction of Bachelor-Master structure
• New accreditation procedures (Treaty of Bolognia)
• Differences in prior knowledge
• Completion rates 1st year differ
– 12% “
other”foreign students
– 50 % Dutch/German students
• Incentive problems physical summer course
• Problems at Master level even higher
22/05/2008

5. 2. Online Remedial Teaching Model
1. Online Availability 24/7
(Vrasides & Zembylas, 2003)
2. Adaptive
(Falmange et al., 2004)
3. Rapid feedback
(Draaijer, 2004, Vrasides & Zembylas, 2003)
4. Interactive
(Bryant et al. 2005, Ronteltap & Van der Veen, 2002)
5. Flexible Learning Methods & Assessment
(Marshall, 2003, Segers, 2004)
22/05/2008

6. Online Remedial Teaching Model
Electronic Learning Environment
Interaction
Student feedback Student
fee ck
db ba e
ad ac ed tiv
k fe
a pti ap
ve ad
Technology
tion
Inte
Fee
ck
24/7 online
rac
rac
dba
dba
Inte
tion
Fee
ck
Teacher
22/05/2008

7. 3. Methodology
22/05/2008

8. Prior knowledge test
Mathematics
• # attempts 524
• # completed tests 230
• # above threshold 39 (17%)
• # below threshold 191 (83%)
• # of students starting at Maastricht 101
– 55 joined (55%), mainly German “
Grundkurs”students
Economics
• # attempts 379
• # completed tests 211
• # above threshold 60 (25%)
• # below threshold 151 (75%)
• # of students starting at Maastricht ??
– 50 joined
From 01-04-2005 till 01-09-2005
22/05/2008

9. Start of Online Summercourse
Intro in Economics Mathematics
• weeks of online &
4 •4 weeks of individual learning
collaborative Problem Based •200 items
Learning •Entry assessment of true
• PBL-tasks
6 mastery
•Online chapters, video’ and
s •Learning path individual and
animated graphs from Parkin & unique!
Bade (2004) •Use of ALEKS
•Use of Discussion forum •Final Assessment
(Polaris)
•15-20 hours per week.
•Use of Weekly Assessment
(Blackboard)
•Final Assessment
•15-20 hours week
22/05/2008

10. Economics (Link)
22/05/2008

11. 22/05/2008

12. 22/05/2008

13. Mathematics
22/05/2008

14. “Ideal”individual learning-path
Based on outcomes of entry-
assessment, a student could be
evaluated at any point on the knowledge
space of topic X.
Student A can have a different learning
path than Student D to reach point f
Ideally, the learning materials and
teachings methods should adapt to the
knowledge/skills of each individual
student.
22/05/2008

15. ALEKS: learning pie
22/05/2008

16. Evaluation Mathematics
n=38
Table 1 End evaluation online summer course mathematics
This Summer course offered me a lot 4,6 (0,6)
The contents of the Summer course were inspiring 4,2 (0,5)
The format of the Summer course was good 4,4 (0,7)
The Summer course was well organized 4,5 (0,6)
The quality of the material in ALEKS is good 4,3 (0,6)
The material in ALEKS motivated me to keep up with the subject matter 3,9 (0,7)
Learning in an e-learning environment as ALEKS is not different from learning from a hard-copy book 2,4 (0,9)
It was fun that I could attend this Summer course via the internet 4,2 (0,8)
I gained enough knowledge and skills in mathematics to start with my study in Maastricht 3,5 (0,8)
It was easy to motivate myself to finish this Summer course 3,5 (1,1)
It was good that I could work on the subject matter at my own pace 4,5 (0,6)
I think that I have learned more by individually attending this course than I would have learned if I had to collaborate 3,9 (1,0) (P)
3,0 (0,9) (NP)
T-value= 1,53**
Questions via e-mail were answered well by the teacher 3,8 (0,8)
Give an overall grade for the quality of support you were given by ALEKS in this Summer course (1 = very bad - 10 = very good) 8,4 (1,0)
Give an overall grade for the quality of this Summer course (1 = very bad - 10 = very good) 8,7 (0,9)
Hours worked per week 17,5 (5,1) (P)
6,7 (4,0) (NP)
T-value= 14,10**
Source: Rienties, Dijkstra, Rehm, Tempelaar, Blok (2005), p. 249.
Note:All questions on Likert scale of 1 (=totally disagee) to 5 (totally agree) with exception to last three questions. Standard deviation between brackets. If no distinction is made
between the groups (P=Passed, NP= Not Passed), then Independent T-sample test is not significant and average of entire group is depicted.
* p < .05; ** p < .01 of Independent T-sample test.
22/05/2008

17. Evaluation Economics
Passed Failed T-value
Table 2 End evaluation online summer course Economics
n=25 n=10
This Summer course offered me a lot 4,3 (0,5) 3,5 (1,0) 2,394*
The contents of the Summer course were inspiring 4,2 (0,6) 3,9 (0,6) 1,246
The format of the Summer course was good 4,2 (0,8) 3,6 (0,7) 1,764
The Summer course was well organized 4,2 (0,8) 3,4 (0,7) 2,817**
The quality of the digital material was good 4,5 (0,6) 3,9 (0,7) 2,626*
The digital material motivated me to keep up with the subject matter 3,8 (1,0) 2,9 (0,6) 2,520*
Learning with an E-book is not different from learning from a hard-copy book 2,3 (1,1) 2,2 (1,0) 0,303
It was fun that I could attend this Summer course via the internet 4,0 (0,8) 3,2 (0,6) 2,994**
I am satisfied with what I learned in terms of knowledge, skills and insight 4,0 (0,7) 2,8 (0,6) 4,989**
I gained enough knowledge and skills in economics to start with my study in Maastricht 3,8 (0,6) 3,2 (0,4) 2,968**
The group in which I participated functioned well 4,0 (0,9) 3,1 (0,9) 2,664*
It was fun to collaborate with others in this Summer course 4,2 (0,7) 3,1 (1,1) 3,529**
Collaborating with others facilitated my understanding of the subject matter 3,9 (0,6) 3,2 (0,8) 3,015**
I think I was motivated to finish this Summer course because I could work in my own pace 3,8 (1,0) 2,4 (0,5) 5,231**
Give an overall grade for the functioning of the Online Summer course team (1 = very bad - 10 = very good) 8,3 (1,0) 7,4 (1,9) 1,857
Give an overall grade for the quality of the Online Summer course team (1 = very bad - 10 = very good) 8,4 (1,0) 8,1 (1,6) 0,659
Hours worked per week 14,6 (5,7) 6,0 (4,9) 13,406**
Source: Rienties, Dijkstra, Rehm, Tempelaar, Blok (2005), p. 248.
Note:All questions on Likert scale of 1 (=totally disagee) to 5 (totally agree) with exception to last three22/05/2008 deviation between brackets.
questions. Standard
* p < .05; ** p < .01 of Independent T-sample test.

18. 5.1 Study success QM
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
aj
in
il
K
ac
LK
,2
,2
ss
B1
Fa
G
hM
hM
A1
B1
tB
Pa
rm
O
rm
C
In
at
at
W
O
O
C
S
G
tM
G
tM
W
W
V
S
In
In
V
V
Figure 2: Passing rates QM1 (in %)
22/05/2008

19. Figure 3: Passing rates & exam participation QM1 (in %)
100%
90%
80%
70%
60%
SC Pass
50% SC Fail
No SC
40%
30%
20%
10%
0%
Going for exam Passing Exam Going & Passing
QM1 Exam QM1
22/05/2008

20. Figure 4: Partial scores in QM1 final exam (1-20)
16
14
12
10
SC Pass
8 SC Fail
NO SC
6
4
2
0
MathExam StatsExam
22/05/2008

21. Selection bias?
35
30
25
20
15
10
5
0
aj
il
in
K
LK
ac
ss
,2
,2
B1
Fa
G
hM
hM
B1
A1
tB
Pa
rm
O
rm
C
In
at
at
VW
O
O
C
S
G
G
tM
tM
W
W
S
In
In
V
V
Figure 5: Hours studying with ALEKS during QM1
22/05/2008

22. 5.2 Study success Economics
100
95
90
85
80
75
70
65
60
55
50
EconPrior NoEconPrior SC-pas s SC-f ail
Figure 6: Passing rates EcBus (%)
22/05/2008

23. Figure 7: Average grades final exam EcBus
(0-10)
7,4
7,2
7
6,8
6,6
6,4
6,2
6
5,8
5,6
5,4
EconPrior NoEconPrior SC-pass SC-fail
22/05/2008

24. 6. Didactical scenario
• Mathematics (individual) vs. Economics (collaborative/PBL)
• “ think that I have learned more by individually attending
I
this course than I would have learned if I had to
collaborate” 3.6 for the math summer course
:
• “ think I learned more in this Summercourse through
I
collaboration with others than I would have learned if I had
to work” 3.2 score for the economics summer course.
:
• Individual learning is regarded more suitable for the math
summer course
• Collaborative learning most suitable for the economics
summer course.
• Students’ opinion on the appropriateness of the didactical
scenario is dependent upon success in the summer course
22/05/2008

25. 6. Discussion
1. Are summer courses necessary if one
wants to become a truly international
university?
2. Is online assessment/collaboration a
good tool to monitor progress of
students?
3. What is the role of Virtual Learning
Environments in providing summer
courses?
22/05/2008

26. 22/05/2008

27. Effectiveness of an Online Summer course Programme
13th Edineb Congress
Thursday 15th of June 2005, Lisbon, 9.00-10.30
http://www.web-spijkeren.nl
Bart Rienties (University Maastricht)
b.rienties@algec.unimaas.nl
Dirk Tempelaar
d.tempelaar@ke.unimaas.nl
Joost Dijkstra
Joost.dijkstra@educ.unimaas.nl
Martin Rehm
m.rehm@algec.unimaas.nl
Geke Blok
g.blok@educ.unimaas.nl
22/05/2008