1. Computer Supported Collaborative Learning and its
impact on the Quality of Studying Economics
13th Edineb Congress
Friday 16th of June 2005, Lisbon, 9.00-10.30
http://www.web-spijkeren.nl
Bart Rienties (University Maastricht)
b.rienties@algec.unimaas.nl
Martin Rehm
m.rehm@algec.unimaas.nl
Wim Gijselaers
Mien Segers
22/05/2008

2. 22/05/2008

3. 22/05/2008

4. Agenda
1. Introduction
2. Theoretical considerations
3. Methodology
4. Results
5. Discussion
22/05/2008

5. Introduction
Why Computer Supported Collaborative Learning (CSCL) ?
1. Communication within the network explicit and coded
2. Asynchronous, more time to think and to reflect
3. Each student can study at his/her own time of preference
4. Large part of the online environment is actually CSCL
5. Interaction and collaboration is essential for learning
22/05/2008

6. What is new?
1. Pilot where CSCL was intensively used in a virtual learning
environment (VLE).
2. Most studies master-students or educational-science students,
focus the experiences with a novice group of students.
3. If higher cognitive processes towards the end of the course
have occurred, then this might provide proof for positive
learning effects of CSCL.
Central Question:
What is the impact of asynchronous electronic discussion
groups in a virtual learning environment on the quality of
the learning process of novice students?
22/05/2008

7. 2. Theoretical considerations
2.1 Unit of analysis
1. single unit (each individual sentence)
2. unit of meaning (one or more theme(s)
3. complete message
2.2 Segmentation and Reliability analysis
22/05/2008

8. 2.3 Model of Veerman and Veldhuis-Diermanse (2001)
Table 1 Catalysing of Threads
Message Example in Economics
Not Task-Related
- Planning quot;Shall we first complete Task 1,
before we go on with the next one?quot;
- Technical quot;Does anybody know how to add
a graph to my thread?quot;
- Social quot;Good summary!quot;
- Nonsense quot;Who wants to join us for a drink in
the pub?quot;
Task-Related
- New Idea
- Facts quot;The average rate of inflation in the
U.S. for 2004 is 2.7 %.quot;
- Experience / Opinion quot;I think that VAT-taxes should be
reduced to increase demand.quot;
- Theoretical Ideas quot;According to Perloff (2003), consumers maximize utility
subject to a budget constraint.quot;
- Explicitation quot;Moreover, this process necessitates that MR = MC.quot;
- Evaluation quot;Overall, combining the concept of utility and welfare, social
welfare is maximized when every individual can sets MR = MC,
without distortions.quot;
Note: Table 1 is based on Veerman and Veldhuis-Diermanse (2001), p.626. The examples in Economics are extracts from the discussion boards.
22/05/2008

9. 2.4 Online Remedial Teaching Model
1. Online Availability 24/7
(Vrasides & Zembylas, 2003)
2. Adaptive
(Falmange et al., 2004)
3. Rapid feedback
(Draaijer, 2004, Vrasides & Zembylas, 2003)
4. Interactive
(Bryant et al. 2005, Ronteltap & Van der Veen, 2002)
5. Flexible Learning Methods & Assessment
(Marshall, 2003, Segers, 2004)
22/05/2008

10. Online Remedial Teaching Model
22/05/2008

11. 3. Methodology
50 international fresh-men for bachelor
programme Economics.
Main advantages
1. Students not trained in “ academic
behaviour”
2. Lack academic (higher-cognitive)
thinking skills
3. Population has collaborated only
virtually with each other.
22/05/2008

12. 3.1 Hypotheses (Schellens & Valcke
(2005))
1. The proportion of task-oriented communication is larger
than the proportion of non-task-oriented communication.
2. Communication in the discussion groups will become more
task-oriented instead of non-task oriented.
3. In the context of non-task-oriented communication, more
‘social’ and ‘planning’ communication will be observed, at
the expense of interaction in relation to ‘
technical issues’
and ‘nonsense’ interaction.
4. The CSCL-environment fosters higher phases of
knowledge construction.
5. At the end of the research period, more communication
reflecting higher phases of knowledge construction will be
observed.
6. The more discussion activity in groups, the more phases of
higher knowledge construction will appear.
22/05/2008

13. 3.1 Hypotheses
Furthermore, based on section 2.4,
this research is also interested in
these two hypotheses:
7.The proportion of non-task-related
messages has a positive impact on
the activity of the group as well as
the knowledge construction.
8.The overall activity of a discussion
and the quality of the knowledge
construction increases with the
proportion of “ nonsense”
contributions.
22/05/2008

14. 3.2 Research design
1. 50 Students were selected based on
prior-knowledge test
2. e-PBL
3. 3 independent coders instead of 2
4. 1187 units analysed
5. 1206 units selected
6. Cronbach alpha > 0.75
7. Cohen’ kappa > 0.72
s
22/05/2008

15. 4. Results
Table 2 Task- and non-task oriented communication (in percentage)
Café a Theme Theme Theme All Schellens
f
Intro b Micro c Macro d themes e et al.
Non-task oriented 99.3 15.8 16.0 18.7 61. 3 5.8
Task oriented 0.7 84.2 84.0 81.3 38.7 94.2
a
n = 443
b
n = 139
c
n = 108
d
n =134
e
n = 822
f
n = 1428 (All themes data from Schellens and Valcke, 2005)
H0 Task > non task: Rejected
H1 Communication becomes more task-oriented: Rejected
Table 3 Distribution of non-task oriented communication in four categories (in
percentage, based on Veerman et al., 2001)
Café a Theme Theme Theme All Schellens
f
Intro b Micro c Macro d themes e et al.
Planning 7.7 50.0 11.8 8.0 9.7 20.6
Technical 8.6 18.2 29.4 16.0 10.1 9.2
Social 3.2 13.6 47.1 36.0 6.7 52.1
Nonsense 80.5 18.2 11.8 40.0 73.4 18.1
a
n = 440
b
n = 22
c
n = 17 H3 Social + Planning > Tech. + Nonsense : Rejected
d
n = 25
e
n = 504
f
n = 305 (All themes data from Schellens and Valcke, 2005)
22/05/2008

16. Higher cognitive learning?
Table 4. Types of task-oriented communication (in percentage, based on
Veerman et al., 2001)
Caféa Theme 1 b Theme 2 c Theme 3 d All themes e Schellens et
Intro Micro Macro al.f
Phase 1: New Info:
Facts 66.7 43.6 44.9 60.6 50.0 0.1
Phase 2: New Info:
Experience / Opinions
0.0 8.5 1.1 4.6 5.0 14.3
Phase 3: New Info:
Theory 33.3 12.0 20.2 13.8 15.1 29.6
Phase 4: Explicitation 0.0 34.2 25.8 19.3 26.4 15
Phase 5: Evaluation 0.0 1.7 7.9 1.8 3.5 41
a
n=3
b
n = 117
c
n = 89
d
n =109
e
n = 318
f
n = 1345 (All themes data from Schellens and Valcke, 2005)
H4 Task related: Phase 4-5 > Phase 1 - 3 : Rejected
H5 More Phase 3-5 at end of course: Rejected
22/05/2008

17. Table 5 Distribution of communication of groups with different levels of
discussion activities
Low discussion activity Median Discussion High Discussion
N <150 activity N = 234 activity N > 440
Total N % Total N % Total N %
Non-task-oriented 86 58.9 96 41.0 322 72.9
Task oriented 60 41.1 138 59.0 120 27.1
Non-task oriented
communication a
Planning 16 18.6 5 5.2 28 8.7
Technical 5 5.8 13 13.5 33 10.2
Social 2 2.3 20 20.8 12 3.7
Nonsense 63 73.3 58 60.4 249 77.3
Task-oriented
communication b
Phase 1: New Info: Facts 24 40.0 67 48.6 68 56.7
Phase 2: New Info: 3 5.0 9 6.5 4 3.3
Experience / Opinions
Phase 3: New Info: Theory 6 10.0 18 13.0 24 20.0
Phase 4: Explicitation 23 38.3 40 29.0 21 17.5
Phase 5: Evaluation 4 6.7 4 2.9 3 2.5
a
n = 504
b
n = 318 H6 More activity, more phase 3-5: Rejected
H7 More non-task, more activity & phase 3-5: Rejected
H8 More nonsense, more activity & phase 3-5: Rejected
22/05/2008

18. 6. Discussion
1. The proportion of task-oriented communication is larger than
the proportion of non-task-oriented communication.
1. Exact opposite result than Schellens & Valcke (2005)
2. Distance learning course non-relevant issues occur more often
3. Many new issues about (non-)academic life were discussed and new
friendships were build.
4. Intensively used as a means to get acquainted with each other
2. Communication in the discussion groups will become more
task-oriented instead of non-task oriented.
1. Intensively used as a means to get acquainted with each other
2. The results are in line with the findings of Schellens and Valcke (2005)
3. Hypothesis difficult to test as Café forum at the same time as working
on the respective tasks.
3. In the context of non-task-oriented communication, more
‘social’and ‘planning’communication will be observed, at the
expense of interaction in relation to ‘
technical issues’and
‘nonsense’interaction.
1. “Agree-button”in Polaris acts as a simple-to-use tool that takes over
the part of ‘
social’talk.
2. Prevalence of ‘Nonsense’ communication adds to the skewed outcome
of the result.
22/05/2008

19. 6. Discussion
4. The CSCL-environment fosters higher phases of
knowledge construction.
1. No experience in economics as well as academic thinking.
2. Goal of the online summer course to introduce basic principles
of Economics.
3. 26.4 % value for Explicitation, in comparison to the 15 % found
by Schellens and Valcke (2005) promising outcome
5. At the end of the research period, more communication
reflecting higher phases of knowledge construction will
be observed.
1. Results support this notion until midterm of the course
2. Tasks of Themes 1 and 3 solved by basic skills
3. Tasks of Theme 2 required a more technical analysis.
4. The decrease towards the end could be explained by the
approaching start of the actual study,
6. The more discussion activity in groups, the more
phases of higher knowledge construction will appear
1. Most differences in overall discussion activities on
contributions Café forum.
2. If person is feeling comfortable in the group, CSCL-
environment can increase the involvement of students
22/05/2008

20. 6. Discussion
7. The proportion of non-task-related messages has
a positive impact on the activity of the group as
well as the knowledge construction.
1. Minimum level of messages is necessary to establish a
critical mass of discussions,
2. Low discussion activity group relative low amount but on
higher cognitive level
8. The overall activity of a discussion and the quality
of the knowledge construction increases with the
proportion of “nonsense”contributions.
1. Non-task-related messages positive impact on the
overall activity of the group.
2. ‘
Nonsense’ link and overall formation of knowledge.
3. No clear picture with regard to the quality of the
knowledge construction
22/05/2008

21. Future research
1. Whether the online summer course in general and
the CSCL in particular has a temporary or structural
effect on the (prior) knowledge level and
competencies of students.
2. Whether the choice of participants influenced the
results.
3. Similar course set-up will be provided on a group of
master students, who have the expertise.
4. Research is needed on the motivation of
participants.
5. Subgroups participants enlarged
6. Didactical scenarios implemented at other institutes.
7. More specified and detailed information about the
subgroups
22/05/2008

22. Computer Supported Collaborative Learning and its
impact on the Quality of Studying Economics
13th Edineb Congress
Friday 16th of June 2005, Lisbon, 9.00-10.30
http://www.web-spijkeren.nl
Bart Rienties (University Maastricht)
b.rienties@algec.unimaas.nl
Martin Rehm
m.rehm@algec.unimaas.nl
Wim Gijselaers
Mien Segers
22/05/2008