How to achieve a productive online discussion using Moodle, without teacher marking!

1. Brant Knutzen
Lingnan University
The Teaching and Learning Centre
Lingnan University

2.  Social Constructivism
 Transactivity
 Study #1
 Assessment methods
 Participation marking scheme
 Study #2

3.  CMC : Computer-mediated Conferencing
 Synchronous : “chat”, or instant messaging
 Asychronous : “discussion forum”

5.  The effective construction of knowledge is a
product of the collaborative group
 Efficacy has been found to be linked to the
process that learners utilize in working on the
task together (Fischer et al 2002)
 Process: social negotiation of arguments and
argument sequences (Leitão 2000; Voss & Dyke 2001)

6.  John Biggs captured the educational value of
discussion when he stated:
"Good dialogue elicits those activities that
shape, elaborate, and deepen understanding“
(Biggs 1999 p. 5)

7.  Transactivity: the method by which students
build on the contributions of their fellow
learners
(Berkowitz & Gibbs 1983)

8.  Transactive communication:
 Participants respond to and build on each other’s
contributions
 Peer exchange of information and ideas
 Social negotiation of knowledge
 Each participant brings their own experiences to
apply to a common educational goal

9.  A key theoretical construct for measuring collaboration
 How can we describe it in easily grasped ways?
 Quantitative
 Qualitative
 How can we formulate the instructional design
conditions which consistently result in more productive
and transactive learning activities?

10.  Case study of one course (Knutzen)
▪ International school in Hong Kong – secondary level
▪ 1-to-1 laptop blended learning environment
▪ Introduction to Psychology course
▪ Sample size = 24
 Investigation of instructional design conditions to
achieve a highly productive online discussion
 At start of study, average student production in
online discussions = 1 post

11.  Four conditions to achieve productive online
discussions:
 Teacher facilitated social formation of small groups
 Class time to initiate online discussion interaction
 Setting open-ended, challenging topic questions that
encourage discussion and debate
 Assessment system that reinforces production and peer
interaction
 At end of study, average student
production: over 10 posts per discussion!

12.  Over the following three teaching years:
 Extensive use of the online discussion design
 Full-time instruction of secondary students
▪ 1-to-1 laptop environment
▪ IT classes
▪ Psychology
 Part-time instruction of post-graduate students
 Literally hundreds of online discussions
 Design continues to result in good production

13.  Traditional – teacher-assessed subjective marking
 Review contributions by each student
 Award mark based on:
▪ Participation – any contribution to discussion
▪ Interaction - responding and seeking feedback
▪ Transaction – sharing / exchanging useful information and resources
▪ Transformation - ideas and understanding clearly develop as a function
of interaction and transaction
 Best method for summative assessment

14.  A highly productive discussion can easily
produce over 200 posts!
 A teacher can become a victim of their own
success
 How much time can they devote to quantitative
marking?
 How much time remains for qualitative
feedback?

15.  Desired graduate attributes:
 Critical thinking skills
 Excellent cooperative skills
▪ Integrity
▪ Personal responsibility
 Subjective peer-assessment can directly
address the development of these attributes
 Requires student training
 Requires review and evaluation by teacher

16.  Possible problems:
 “Revenge grading” or 報復
▪ you gave me a low grade, I will give you a low grade
 “Back-scratching” or 賄賂
▪ If you give me a high grade, I will give you a high grade

17.  One solution: objective peer-rating based on
participation
 No judgment, just rating using a clear system:
 Moodle can automatically average these grades!

18.  Moodle averages the peer-awarded marks
 Grades produced by participation:
▪ One post = 6 -> D-
▪ Two posts = 8 -> B-
▪ Three posts = 8.6 -> B
▪ Four posts = 9 -> A-
▪ Five posts = 9.2 -> A-
▪ Six posts = 9.33 -> A
▪ Seven posts = 9.42 -> A
▪ Eight posts = 9.5 etc
 More participation = higher grade

19.  Students cannot mark own work, only others
 Awareness of contributions by other students
 Team-building incentive
 Teacher has plenty of time to:
 Monitor progress
 Provide qualitative feedback
 Name: the “6 / 10 / 10 / 10” peer-rated
participation-based marking scheme

20.  Knutzen & Kennedy – study conducted in 2009
 Two versions of same course at HKU
 Sample size: n = 53
 Same lecture content, same 5 discussion topics
 Different instructional design of discussions:
 Teacher facilitated social formation of small groups
 In-class time given to begin discussions
 Open-ended topic questions which encourage discussion and debate
 Peer-rated participation marking scheme (6 / 10 / 10 / 10)

21.  Quantitative:
▪ Production = Total number of discussion posts / n
▪ Interactivity = Total number of feedback posts / n
▪ Group Activity = Total number of discussion posts / # topics
▪ Transactivity = Production × Interactivity
 Qualitative: a new type of graphical
representation – the “BushGraph”

22.  a non-transactive Moodle discussion forum:

23.  The BushGraph of Class #1 – discussion #1
 Each student creates their own discussion topic and posts once, but rarely
reads or gives feedback to any other posts – a “lawn”

24.  The same Moodle discussion, with improved
instructional design:

25.  The BushGraph of Class #2 - discussion #1

26.  The BushGraph of Class #1 - discussion #2

28.  Class #1 – Discussion #3
 The one discussion in Class #1 worth any points!
 5% of total grade, teacher assessed

30.  The BushGraph of Class #1 - discussion #4

32.  The BushGraph of Class #1 - discussion #5

35.  Participation marking has high reliability

36.  Participation marking has very high validity

37.  Could there be a further relationship between
the discussion grades <-> project grades and
the descriptive statistics, such as interactivity?
Class #2 Discussion
Grades
Correlation
With
Final Project
Discussion # Production Interactivity Group Activity Transactivity Grades
1 3.4 1.75 11.3 5.95 0.608
2 5 2.8 20 14.00 0.757
3 5.56 4.15 22.6 23.07 0.919
4 6.15 3.45 24.6 21.22 0.675
5 6.25 3.55 25 22.19 0.885

38.  As interaction in a discussion goes up, the learning activity
becomes more accurate in assessing student performance

39.  We found evidence that the redesign of the
instructional design variables to meet the
conditions developed in study #1 have a significant
positive effect on:
 the production in the resulting online discussions
 several measures of transactivity:
▪ Interactivity
▪ Group activity
▪ Transactivity (Interactivity × Group Activity)

40.  Peer-marked participation-based scoring
method was found to have:
 High reliability
▪ Good consistency between discussion grades
 Very high validity
▪ Strong correlations to teacher-assessed learning
activities and the course total grade

41.  Sample sizes were small
 Almost exclusively Asian students
 Replication and extension of this study with a larger sample
size in other cultural settings would provide additional
perspectives with greater predictive validity
 Possible additional factor affecting results:
 differences in instructor teaching style
 emphasis on the importance of online discussions

42.  Determine a causal direction:
 is the highly interactive discussion environment
conducive to the creation and demonstration of
higher levels of understanding?
 Or
 are students with higher levels of understanding
more capable and willing to create interactive
discourse?

43.  To achieve highly productive and transactive online
discussions in a blended 1-to-1 teaching
environment, instructors should incorporate a
specific set of four design conditions:
 Teacher facilitation of social grouping of students
 Class time to initiate online discussion interaction
 Setting open-ended, challenging topic questions that
encourage discussion and debate
 Assessment system that reinforces production and peer
interaction

44.  When these instructional design conditions
are successfully incorporated, the potential
benefits of social constructivism as an
instructional design paradigm can be realized
within a blended educational environment.

45.  Berkowitz, M. and Gibbs, J. (1983) "Measuring the developmental features of
moral discussion", Merrill-Palmer Quarterly, Vol. 29, pp 399-410.
 Biggs, J. (1999) "What the Student Does: teaching for enhanced learning", Higher
Education Research & Development, Vol. 18, No. 1, pp 57.
 Fischer, F., Bruhn, J., Gräsel, C., and Mandl, H. (2002) "Fostering collaborative
knowledge construction with visualization tools", Learning and Instruction, Vol.
12, pp 213-232.
 Leitão, S. (2000) "The potential of argument in knowledge building" Human
Development, Vol. 43, pp 332-360.
 Voss, J.F. and Van Dyke, J.A. (2001), "Argumentation in Psychology", Discourse
Processes, Vol. 32, No. 2/3, pp 89-111.

46.  Questions?
 Send me an email to get
a copy of this paper:
 BKnutzen@LN.edu.hk
 Thank you for coming!