This document discusses instructional design strategies for achieving highly productive and transactive online discussions. It summarizes two studies conducted by Brant Knutzen that tested the effects of incorporating specific design conditions into online discussion forums. The conditions included facilitating social grouping, in-class initiation time, open-ended discussion questions, and a peer-assessment marking scheme. The studies found that these conditions led to significantly more interaction and transactivity among students in the online discussions.

1. Brant Knutzen
Lingnan University
The Teaching and Learning Centre
Lingnan University

2.  Social Constructivism
 Transactivity
 Study #1
 Assessment methods
 Participation marking scheme
 Study #2

3.  CMC : Computer-mediated Conferencing
 Synchronous : “chat”, or instant messaging
 Asychronous : “discussion forum”

5.  The effective construction of knowledge is a
product of the collaborative group
 Efficacy has been found to be linked to the
process that learners utilize in working on the
task together (Fischer et al 2002)
 Process: social negotiation of arguments and
argument sequences (Leitão 2000; Voss & Dyke 2001)

6.  John Biggs captured the educational value of
discussion when he stated:
"Good dialogue elicits those activities that
shape, elaborate, and deepen understanding“
(Biggs 1999 p. 5)

7.  Transactivity: the method by which students
build on the contributions of their fellow
learners
(Berkowitz & Gibbs 1983)

8.  Transactive communication:
 Participants respond to and build on each other’s
contributions
 Peer exchange of information and ideas
 Social negotiation of knowledge
 Each participant brings their own experiences to
apply to a common educational goal

9.  A key theoretical construct for measuring collaboration
 How can we describe it in easily grasped ways?
 Quantitative
 Qualitative
 How can we formulate the instructional design
conditions which consistently result in more productive
and transactive learning activities?

10.  Case study of one course (Knutzen)
▪ International school in Hong Kong – secondary level
▪ 1-to-1 laptop blended learning environment
▪ Introduction to Psychology course
▪ Sample size = 24
 Investigation of instructional design conditions to
achieve a highly productive online discussion
 At start of study, average student production in
online discussions = 1 post

11.  Four conditions to achieve productive online
discussions:
 Teacher facilitated social formation of small groups
 Class time to initiate online discussion interaction
 Setting open-ended, challenging topic questions that
encourage discussion and debate
 Assessment system that reinforces production and peer
interaction
 At end of study, average student
production: over 10 posts per discussion!

12.  Over the following three teaching years:
 Extensive use of the online discussion design
 Full-time instruction of secondary students
▪ 1-to-1 laptop environment
▪ IT classes
▪ Psychology
 Part-time instruction of post-graduate students
 Literally hundreds of online discussions
 Design continues to result in good production

13.  Traditional – teacher-assessed subjective marking
 Review contributions by each student
 Award mark based on:
▪ Participation – any contribution to discussion
▪ Interaction - responding and seeking feedback
▪ Transaction – sharing / exchanging useful information and resources
▪ Transformation - ideas and understanding clearly develop as a function
of interaction and transaction
 Best method for summative assessment

14.  A highly productive discussion can easily
produce over 200 posts!
 A teacher can become a victim of their own
success
 How much time can they devote to quantitative
marking?
 How much time remains for qualitative
feedback?

15.  Desired graduate attributes:
 Critical thinking skills
 Excellent cooperative skills
▪ Integrity
▪ Personal responsibility
 Subjective peer-assessment can directly
address the development of these attributes
 Requires student training
 Requires review and evaluation by teacher

16.  Possible problems:
 “Revenge grading” or 報復
▪ you gave me a low grade, I will give you a low grade
 “Back-scratching” or 賄賂
▪ If you give me a high grade, I will give you a high grade

17.  One solution: objective peer-rating based on
participation
 No judgment, just rating using a clear system:
 Moodle can automatically average these grades!

18.  Moodle averages the peer-awarded marks
 Grades produced by participation:
▪ One post = 6 -> D-
▪ Two posts = 8 -> B-
▪ Three posts = 8.6 -> B
▪ Four posts = 9 -> A-
▪ Five posts = 9.2 -> A-
▪ Six posts = 9.33 -> A
▪ Seven posts = 9.42 -> A
▪ Eight posts = 9.5 etc
 More participation = higher grade

19.  Students cannot mark own work, only others
 Awareness of contributions by other students
 Team-building incentive
 Teacher has plenty of time to:
 Monitor progress
 Provide qualitative feedback
 Name: the “6 / 10 / 10 / 10” peer-rated
participation-based marking scheme

20.  Knutzen & Kennedy – study conducted in 2009
 Two versions of same course at HKU
 Sample size: n = 53
 Same lecture content, same 5 discussion topics
 Different instructional design of discussions:
 Teacher facilitated social formation of small groups
 In-class time given to begin discussions
 Open-ended topic questions which encourage discussion and debate
 Peer-rated participation marking scheme (6 / 10 / 10 / 10)

21.  Quantitative:
▪ Production = Total number of discussion posts / n
▪ Interactivity = Total number of feedback posts / n
▪ Group Activity = Total number of discussion posts / # topics
▪ Transactivity = Production × Interactivity
 Qualitative: a new type of graphical
representation – the “BushGraph”

22.  a non-transactive Moodle discussion forum:

23.  The BushGraph of Class #1 – discussion #1
 Each student creates their own discussion topic and posts once, but rarely
reads or gives feedback to any other posts – a “lawn”

24.  The same Moodle discussion, with improved
instructional design:

25.  The BushGraph of Class #2 - discussion #1

26.  The BushGraph of Class #1 - discussion #2

28.  Class #1 – Discussion #3
 The one discussion in Class #1 worth any points!
 5% of total grade, teacher assessed

30.  The BushGraph of Class #1 - discussion #4

32.  The BushGraph of Class #1 - discussion #5

35.  Participation marking has high reliability

36.  Participation marking has very high validity

37.  Could there be a further relationship between
the discussion grades <-> project grades and
the descriptive statistics, such as interactivity?
Class #2 Discussion
Grades
Correlation
With
Final Project
Discussion # Production Interactivity Group Activity Transactivity Grades
1 3.4 1.75 11.3 5.95 0.608
2 5 2.8 20 14.00 0.757
3 5.56 4.15 22.6 23.07 0.919
4 6.15 3.45 24.6 21.22 0.675
5 6.25 3.55 25 22.19 0.885

38.  As interaction in a discussion goes up, the learning activity
becomes more accurate in assessing student performance

39.  We found evidence that the redesign of the
instructional design variables to meet the
conditions developed in study #1 have a significant
positive effect on:
 the production in the resulting online discussions
 several measures of transactivity:
▪ Interactivity
▪ Group activity
▪ Transactivity (Interactivity × Group Activity)

40.  Peer-marked participation-based scoring
method was found to have:
 High reliability
▪ Good consistency between discussion grades
 Very high validity
▪ Strong correlations to teacher-assessed learning
activities and the course total grade

41.  Sample sizes were small
 Almost exclusively Asian students
 Replication and extension of this study with a larger sample
size in other cultural settings would provide additional
perspectives with greater predictive validity
 Possible additional factor affecting results:
 differences in instructor teaching style
 emphasis on the importance of online discussions

42.  Determine a causal direction:
 is the highly interactive discussion environment
conducive to the creation and demonstration of
higher levels of understanding?
 Or
 are students with higher levels of understanding
more capable and willing to create interactive
discourse?

43.  To achieve highly productive and transactive online
discussions in a blended 1-to-1 teaching
environment, instructors should incorporate a
specific set of four design conditions:
 Teacher facilitation of social grouping of students
 Class time to initiate online discussion interaction
 Setting open-ended, challenging topic questions that
encourage discussion and debate
 Assessment system that reinforces production and peer
interaction

44.  When these instructional design conditions
are successfully incorporated, the potential
benefits of social constructivism as an
instructional design paradigm can be realized
within a blended educational environment.

45.  Berkowitz, M. and Gibbs, J. (1983) "Measuring the developmental features of
moral discussion", Merrill-Palmer Quarterly, Vol. 29, pp 399-410.
 Biggs, J. (1999) "What the Student Does: teaching for enhanced learning", Higher
Education Research & Development, Vol. 18, No. 1, pp 57.
 Fischer, F., Bruhn, J., Gräsel, C., and Mandl, H. (2002) "Fostering collaborative
knowledge construction with visualization tools", Learning and Instruction, Vol.
12, pp 213-232.
 Leitão, S. (2000) "The potential of argument in knowledge building" Human
Development, Vol. 43, pp 332-360.
 Voss, J.F. and Van Dyke, J.A. (2001), "Argumentation in Psychology", Discourse
Processes, Vol. 32, No. 2/3, pp 89-111.

46.  Questions?
 Send me an email to get
a copy of this paper:
 BKnutzen@LN.edu.hk
 Thank you for coming!