"Engaging Students in Distance Learning." Presentation given at 7th Drexel e-Learning Conference, March 26th, 2009, by Dr Jim Waters & Dr Susan Gasson.

1. Engaging Students in Distance Learning Jim Waters Susan Gasson The iSchool at Drexel

2. If students have already paid us, why should we care if they are engaged ? Altruism – the long view? Pragmatism – they can always leave taking their tuition money with them Student feedback and tenure decisions? Word gets out! – social networks Students can vote with their virtual feet and Their very real wallets

3. Agenda How do I know if students are engaged? What is the effect of: Question design? Course scaffolding? Instructor moderation? Being an entertaining instructor? So what?

4. Research Study Analyzed course interactions via discussion board on Blackboard learning system. 12 online MS courses (info. systems./info. Science) 313 Students, 11,497 messages Posts to discussion board + small group discussions Analyzed Thread depth, thread length, participants Cognitive content of message Interactive intent of message Patterns of message sequences Examined student outcomes related to interaction Pre and Post questionnaires Demographics and Attitudinal data

5. Three modes of learning engagement Individual Participation Active Course Involvement Iterative Social Engagement Fluid: students can move between modes reacting to drivers

6. Mode 1: Individual Participation The semi-transparent participant Interacts with materials Internalizes knowledge Contractual obligation postings Broadcast messages Superficial learning Hermit!

7. Mode 2: Active Course Involvement Demonstrates (some) genuine interest Interacts with peers (after a fashion) Translates community knowledge Relates posts to own experience or knowledge Internalizes community knowledge Ego-centric approach Small group or clique interactions

8. Mode 3: Interactive Social Engagement Motivated for interactive learning Committed to greater group learning Interacts freely with peers Looped learning cycles Iterative internalizations/externalizations Social construction of knowledge

9. How can we tell what is going on?

10. Mode 1: Course Participation How Do We Know A Project Is On Track? Instructor 10/21/07 2:36 PM RE:How Do We Know A Project Is On Track? S18 10/25/07 12:09 PM RE:How Do We Know A Project Is On Track? S10 10/25/07 6:29 PM RE:How Do We Know A Project Is On Track? S21 10/25/07 8:30 PM RE:How Do We Know A Project Is On Track? S17 10/26/07 7:38 PM RE:How Do We Know A Project Is On Track? S22 10/28/07 6:19 PM RE:How Do We Know A Project Is On Track? S2 10/28/07 7:04 PM RE:How Do We Know A Project Is On Track? S4 10/28/07 10:24 PM RE:How Do We Know A Project Is On Track? S10 10/28/07 10:26 PM RE:How Do We Know A Project Is On Track? S7 10/28/07 10:46 PM RE:How Do We Know A Project Is On Track? S8 10/29/07 12:59 AM RE:How Do We Know A Project Is On Track? S11 12/14/07 11:34 AM

11. Mode 1: Course Participation

12. Mode 2: Active Course Involvement Unanswered questions Instructor 10/5/07 3:23 PM RE:Unanswered questions S1 10/5/07 6:59 PM RE:Unanswered questions S17 10/6/07 3:41 AM RE:RE:Unanswered questions S19 10/8/07 12:33 AM RE:RE:RE:Unanswered questions S20 10/8/07 10:52 AM RE:Unanswered questions S13 10/6/07 10:52 AM RE:RE:Unanswered questions S18 10/7/07 4:14 PM RE:RE:RE:Unanswered questions S6 10/9/07 9:45 PM RE:Unanswered questions S12 10/6/07 11:04 AM RE:RE:Unanswered questions S20 10/7/07 10:34 AM RE:Unanswered questions S9 10/7/07 6:49 AM RE:Unanswered questions S21 10/7/07 4:36 PM RE:Unanswered questions S10 10/7/07 5:31 PM RE:Unanswered questions S4 10/7/07 10:59 PM RE:Unanswered questions S19 10/8/07 12:07 AM RE:Unanswered questions S8 10/8/07 2:21 PM BOK as a communications/marketing tool S12 10/9/07 1:18 PM RE:Unanswered questions S8 10/8/07 2:48 PM RE:Unanswered questions S14 10/12/07 2:33 PM RE:RE:Unanswered questions S12 10/12/07 3:22 PM RE:Unanswered questions S15 10/12/07 3:26

13. Mode 2: Active Course Involvement

14. Mode 3: Interactive Social Engagement

15. Mode 3: Interactive Social Engagement

16. What makes students engage at higher levels in a distance-learning course?

17. Question Frequency vs. response

18. Question Design I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [Fact-finding] Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or slow] I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [Cooking up a new project]

19. And the Winner is I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [Cooking up a new project] I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [Fact-finding] Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or slow]

21. Deliberately pitched as a cooperative task

22. Concrete (well-defined) task

24. Far less cooperative inter-student activity

25. Not pitched as a cooperative activity

27. Five questions in one: 1 was open-ended 4 bounded

28. Very little cooperative inter-student activity

30. Bad questions tended to be Following a highly-interactive question or later in the course Later questions much less interactive and constructive across courses than earlier questions Cognitively complex Containing multiple parts that needed to be considered in turn, or Overly abstract, so students could not draw on their personal experience. Socially isolating Fewer opportunities for interpretation and collaboration in answering the question.

31. How does course scaffolding affect student engagement?

32. Scaffolding Something to hang construction on Solid foundation for task Materials Discussion Support for knowledgeable peers to contribute Task requires an extension of prior abilities Provides a structure on which students can build knowledge Task must be just beyond current unaided skills but doable with help

33. Poor Scaffolding You've been asked to read the ALA Code of Ethics plus two other codes of ethics of your choice. What did you learn from this process? Did any common themes or concerns tend to emerge? What did you relate to in the ALA Code of Ethics? Were there things that seemed problematic, or that you disagreed with? Codes of Ethics Instructor 1/28/08 3:15 PM RE: Codes of Ethics S5 1/31/08 7:00 AM RE: Codes of Ethics S15 1/31/08 5:49 PM RE: Codes of Ethics S13 1/31/08 9:17 PM RE: Codes of Ethics S14 2/1/08 12:05 AM RE: Codes of Ethics S16 2/1/08 12:54 PM RE: Codes of Ethics S9 2/1/08 1:08 PM RE: Codes of Ethics S17 2/3/08 1:20 PM RE: Codes of Ethics S11 2/2/08 3:26 PM RE: Codes of Ethics S18 2/3/08 4:23 PM RE: Codes of Ethics S19 2/4/08 5:33 PM RE: Codes of Ethics S6 2/5/08 6:24 PM RE: Codes of Ethics S1 2/5/08 11:03 PM

34. Better Scaffolding Can ethical behavior really be codified by a professional organization? Can ethical behavior be enforced? How?

35. Unscaffolded Learning materials A long list of codes of ethics web-pages Three abstract ethics articles A body of solid material but this did not directly relate to the posted question or give a framework for answering the question

36. Scaffolded Learning materials A description of ethical models Worksheet for ethical decision making Actions and consequences Responsibilities and obligations A theoretical and pragmatic platform from which discourse could be built Three sparse pages of bullet-points Materials contained less information but provided a structure for thinking.

37. How does instructor moderation affect student engagement?

38. Moderated or leave-alone ? Two sections of an IS course delivered at the same time – same basic syllabus ~Same number of students (23/24) Selected six “identical” questions on each section Different Instructor approach Heavy moderation vs. lightweight moderation

39. Questions and Approach (messages)

41. Instructor interaction and student posts

42. Are students more engaged when the Professor is entertaining?

43. Deconstructing the Entertaining Professor Highly knowledgeable industry professional Very Popular Instructor High level of interaction with students Regular internet chat presence Projects personality into discussion High percentage of social interactions Voluntarily discusses hobbies, weather, music, Disneyworld, cooking, children, Dickens, vintage cars, pets, gardening, insects, Star Wars, birds, Nintendo, Scrabble, foreign films, beer …. Injects lots of jokes

44. Student participation

45. Never mind the width, feel the quality Threads show evaluation and analysis Some hypothesis formation Fairly advanced cognitive activity Some student-student interaction But,some chaff among the wheat 5% Un-focused Anecdotes 32% fluff posts: “LOL,” “Awesome [dead rock star] story!” “OMG,” “Pictures of gardening implements,” “Lawyer Jokes” 50% contribute to learning 33% student-Instructor messages Still, pretty successful overall High student satisfaction Grades were comparable with Prof. Serious

46. Introducing Professor Serious Same course – Same Syllabus Highly knowledgeable industry professional Very skilled Instructor Low level of direct interaction with students Strong Topic-focus Little social interaction

47. Student participation

48. Quality ? More collaborativelearning Messages longer and more detailed Fairly advanced cognitive activity Much stronger student-student interaction Stronger awareness of value of peer interactions So what? High task-Focus 2.5% fluff posts 80% substantive knowledge-building posts 15% student-Instructor messages Successful overall Moderate student satisfaction Grades were comparable with Prof. Entertaining

49. The tale of the tape Instructor participation of Prof. Entertaining inflates thread depth slightly More posts, but smaller % focus on knowledge-building Student-student posts more productive than student-instructor posts Overall productive (knowledge building) activity was about the same for the two Professors Student satisfaction slightly lower for Prof. Serious than Prof. Entertaining Fewer posts, but greater % focus on knowledge-building But Stronger thought leaders for Prof. Serious Explicit kudos for peers in Prof. Serious

50. Conclusions Question design can be crucial to engagement Discussion needs to be framed as collaborative not competitive Relevance to students helps – war stories, company policies and approaches No payoff for frequency of questions Course scaffolding aids engagement Focus and framing of questions Must support task in concrete manner

51. Conclusions Successful course moderation hinges on quality not quantity or frequency Knowing when to intervene Does not mean “being absent” Being entertaining is not essential for success Tradeoff between popularity with students and peer-engagement by students May shift focus from peers to instructor Excessive interaction is a lot of work

52. Students think they are learning more from peer-interaction this was so helpful because often I was struggling with the same thing so I could learn from their errors and gain new information from the answers to their questions I was moved to comment on how refreshing the lack of competition in the communications for the online classes seemed to me. It was a discussion and a sharing of experiences No question that the on line discussion was critical to getting me through the class. There were mostly questions about how to.. I've never done this before. I felt lost and inexperienced most of the time. I have no real full time work experience and I felt I had nothing much to contribute and compared to the rest of the posts mine would feel really insignificant. Honestly, in the second half of the course, I have felt like I must be a pariah. Apart from the professor, I can't get anyone to respond to my posts- a very lonely feeling. I have posted to the the weekly board with little feedback

53. Related References Waters, J. 'Social Network Behavior, Thought-Leaders and Knowledge Building In An Online Learning Community', Proceedings of Hawaii Intl. Conference on System Sciences (HICSS-41), Knowledge Management Track, Jan. 2008. Gasson, S. and Waters, J. “How (not) to construct ALN course questions that encourage student participation in peer collaboration and knowledge construction,” 40th Hawaii International Conference on System Sciences, Hawaii, January 2007. Waters, J., and Gasson, S. "Social Engagement in an Online Community of Inquiry," 27th International Conference on Information Systems (ICIS), Milwaukee WI, 2006. Waters, J. “Determinants of Engagement in an Online Community of Inquiry,” The 12th Sloan-C International Conference on Online Learning, November 2006, http://www.sloanconsortium.org/conference/proceedings/2006/ppt/1162852287092.pot Waters, J., and Gasson, S. "Strategies Employed By Participants In Virtual Learning Communities," Hawaii Intl. Conference on System Sciences (HICSS-38), Collaboration Systems and Technology track, IEEE Software Society, Hawaii, January 2005, p. 3b. A full list of publications, with full copies of articles, is available at http://www.pages.drexel.edu/~jw65/publications.htm

54. Credits Dr. Susan Gasson MBA, PhD from University of Warwick (Warwick Business School), UK. Prior to that, she worked in systems design , IS management, and IS software architecture consultancy. Researches collaborative processes of design, problem-solving, learning. Associate Professor at Drexel University, Philadelphia (USA). Involved in online education since 2000(!). Dr. Jim Waters BA in Psychology from Warwick University, UK (1979), MSc in Occupational Psychology from University of Hertfordshire, UK (1991), MS in Information Systems (2002) and PhD (2009) from the iSchool at Drexel University. Prior to that, he enjoyed a substantial career in systems design, management and IS consultancy. Doctoral Candidate at the iSchool at Drexel University, graduating Summer 2009. Principal research interests lie in Online Collaborative Knowledge Building, Technology-Supported Learning, Student Role-Behavior in Online Learning Communities and HCI. This material is based upon work supported by the National Science Foundation under Grant No. IIS-0347595. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.