This document discusses two views of online courses: as a means of reducing costs by distributing materials online, or as a means of improving learning through collaborative online environments. The author presents research from NJIT comparing online courses that emphasized individual work versus collaborative learning in small groups. Results showed higher exam scores, longer written assignments, and greater satisfaction for courses featuring collaborative learning online. Faculty also perceived greater learning when online discussions and group work were central. The author concludes that well-designed online collaborative learning can be as or more effective than traditional classrooms.
Online Courses: Collaborative Learning Improves Quality and Outcomes Over Passive Models
1. Online Courses as Effective Learning
Environments or "Digital Diploma Mills":
The Importance of Collaborative Learning
Keynote Address, Telelearning ‘99
Montreal Canada November
Copyright, Starr Roxanne Hiltz
New Jersey Institute of Technology
2. Two views of Online courses
A means of cutting the costs of “delivering”
educational materials to students, akin to
previous use of instructional television. Post
materials on the web; collect assignments; can
handle thousands of students.
OR: A means of improving the quality of
learning opportunities, by supporting learning
communities in an anytime/anywhere
environment (Learning Networks “TLN” or
“ALN”)- require small classes mentored by
skilled faculty members
3. Popular press: negative images of
online courses
Article title: “Wiring the Ivory Tower:
But will online courses lower
standards?”
Business Week, August 9, 1999
“No dorms, no sports fields, NO
COSTLY PROFESSORS”
4. Cites UNext:
Will spend $1 million a course for
video streamed lectures by “stars”
Use part time instructors to answer
email and grade assignments
NOT what is meant by LN!!
5. AFT/NEA Commissioned report
on Distance Learning
Asks, “What’s the difference” between
traditional and distance courses;
argues there is no proof that
distance/online is as good or better
6. Some Questions about “Differences”
This Talk Will Cover:
Is online collaborative learning superior
to the mass distribution of materials via
the Web?
How do both models compare to
traditional college level courses, in terms
of process and outcomes?
What are some guidelines for
maximizing the quality of research to
answer these questions?
7. Background
Over the last 15 years, NJIT has constructed a
series of computer-mediated communication
systems tailored to support “anytime/
anywhere” interaction among students and
instructors called “Virtual Classroom” [TM]
Used first in a variety of individual courses
and then for full degree programs; and
Developed various evaluation instruments and
approaches.
8. 8
Virtual ClassroomTM
Project at
NJIT
1993-1996 produced, delivered and evaluated
26 courses comprising the undergraduate
majors in Information Systems and Computer
Science, with Sloan Foundation support
Continuing: “From Virtual Classroom to
Virtual University” 1997-2000
Expand the innovation to other schools
and departments and graduate degree
programs
9. Theories for Studying Online Courses
Pedagogical Theories: Objectivist (passive) vs.
Constructivist (active, collaborative) learning
Media Effect theories (e.g., Media Richness,
Media Synchronicity)
Group Interaction Theories (e.g., Adaptive
Structuration; Poole & DeSanctis)
…LN’s are a social technology through which a
group may choose to faithfully or unfaithfully
appropriate the structures provided by the
technology, heuristic, environment, etc…
10. Premises of the NJIT studies:
Online courses provide unique
opportunities to support
collaborative (group based) learning
Collaborative learning is crucial to
the effectiveness of online learning
environments
11. Seminar: Students as Teachers
Peer Writing Groups (Constructive
Criticism)
Group Projects
Case study discussions
Web “treasure hunts,” compilations
Debates
Construct an exam
Networked classes
Some VARIETIES OF COLLABORATIVE
LEARNING
12. The Research Process
… a series of interlocking choices in which
we try simultaneously to maximize
several conflicting choices…
Key choices
generalizability with respect to populations
realism for the participants
precision in control and measurement of
variables McGrath, 1982
13. Methodological Tradeoffs
Large sample surveys can maximize generalizability
Laboratory Experiments can maximize precision of
control
Field experiments can maximize realism
Qualitative Methods can maximize depth of
understanding ( the “why”)
“Triangulation” or multiple methods maximize overall
validity of results
Replication of results in different settings provides
generalizability
14. 14
Questionnaire Data (1993- 1996
completed project)
Total Responses (sometimes partial):
Virtual Classroom + video: 698
VC + FtF: 463
No VC: 268
Question Form: Please compare online
classes to your previous experiences with
“face to face” college - level courses. To
what extent do you agree with the
following statements...
15. 15
Taking Online Classes is More
Convenient (73% agree)
37 36
17
7
4
0
5
10
15
20
25
30
35
40
Percentage
(%)
Strongly A. Agree Neutral Disagree Strongly D.
N = 624
16. 16
Having the computerized conferencing system available
provided better access to the professor(s). (65% agree)
29
42
17
8
4
0
5
10
15
20
25
30
35
40
45
Percentage
(%)
Strongly A. Agree Neutral Disagree Strongly D.
N = 622, Mean= 2.2
17. 17
Did use of the system increase the quality of
your education? (58% agree; +22% Same)
12
19
27
22
7
5
7
0
5
10
15
20
25
30
Percentage
(%)
1 2 3 Unsure 5 6 7
Definitely Yes = 1 N = 617
Definitely Not = 7
18. 18
“Virtual Classroom Overall”
Index Items included:
1. VC Increases Quality
2. Better learning
3. Learned More
4. Would not take another VC
5. Would have gotten more from a
traditional class
6. VC increases efficiency of learning
Chronbach’s Alpha = .85
19. 19
Correlations with VC Overall
(Pearson’s)
All significant at least at .01 level;
n= about 523
Collaboration Index .30
Better Access to Professor .46
Felt more involved in taking active
part .54
20. 20
Conclusions based on field trials
LNs DO tend to increase access to educational
opportunities, efficiency (speed of completion)
of the degree, and quality of educational
experience as subjectively reported
This is conditional on a number of factors, including active
participation by the student, and communicating/
collaborating with classmates
No Significant Differences in Course Grades
between traditional and LN sections
21. Replication: Similar results from
U. of Illinois, Drexel, SUNY, U. of Central
Florida, etc.
E.g. Drexel: 95% felt they had better access to
Prof
Vast majority of students report that LN
courses are “better” in terms of access and
learning
Student evaluations strongly correlated with
amount on online interaction with professor
and with other students
22. BUT:
In terms of the relationship between
Collaborative Learning and the effectiveness
of LNs, “correlation is not causation”
Triangulation: One course in the project,
Computers and Society, was selected for a field
experiment
Effects of Computer-Mediated Communication on
Learning, Performance and Satisfaction: A
Comparison of Groups and Individuals Solving
Ethical Scenarios, A thesis by: Raquel Benbunan
23. 23
TASK SELECTION
Discussion and solution of computer
ethics scenarios
Decision-making task, completed by writing a
report on analysis and recommendations for
action
Key knowledge in the course and on the final
exam
RESEARCH METHODS
25. 25
Mean Final exam scores on ethics,
(GPA as covariant)
Manual On-line
Individuals 71.83 64.64 68.23
Groups 73.97 70.19 72.08
72.90 67.42
26. Length of the Reports
Mean Words by
Condition
Manual Online
Individuals 380 462 421
Groups 390 756 573
386 609 481
Model F = 8.98 p = .0001 ***
Teamwork Effect F = 11.60 p = .0009 ***
Online Effect F = 21.10 p = .0001 ***
Interaction Effect F = 10.02 p = .002 **
** = Significant at p < .01; *** = Significant at p<.001
27. Triangulation: Faculty Views
Semi-Structured Interviews with 20 NJIT Faculty
Those faculty who utilized group/collaborative
learning and obtained active participation in
discussions online tend to perceive that students
learn more online than in the traditional
classroom; those who did not make online
discussions and group work a central course
activity did not
Replication: SUNY faculty- 47% felt online
students learned more; 46% saw no difference;
only a few thought the classroom performed
better
28. Faculty Workload and Satisfaction- 100
SUNY faculty (1999)
Preparation time: 43% said “much
more” and 41% said “more” than
traditional classroom
Teaching time: 25% said “much more”
and 39% “more”
Teacher-student interaction: 52% more
or much more; 31% the same
Would they do it again: 99 yes
29. Summary and Conclusions:
Each of the three NJIT studies has its
methodological weaknesses, but taken together
they solidly support the conclusions that
1.L N courses can be as effective or more effective
than traditional classrooms, in terms of access and
learning outcomes.
2. Collaborative learning designs are more
effective for online learning than individuals
working alone with materials posted online.
Data from other universities replicate the findings.
What’s the Difference? It’s the pedagogy,
stupid!!
30. A Challenge We Share
Answer the critics with data: Replicate,
Triangulate, INNOVATE
For More Info:
Roxanne’s home page:
http://eies.njit.edu/~hiltz
WWW.ALN.ORG