Higher-Order Thinking Content Analysis of Cognitive Presence In Chat Sessions David S. Stein, Constance E. Wanstreet, Cheryl L. Engle, Hilda R. Glazer, Ruth A. Harris, Susan M. Johnston, Mona R. Simons, and Lynn A. Trinko

Introduction This study examines student interactions in a blended learning environment as they progress through the stages of practical inquiry using higher-order thinking skills in a chat learning space.

Terminology Critical thinking… Statements leading to deeper learning and the development of higher-order cognitive skills in adult learners (Anderson & Garrison, 1995). An important element to understanding how adults learn and necessary to elevate higher-order learning in online chat discussions (Garrison, 1991). Cognitive presence… Closely associated with critical thinking and reflects higher-order knowledge acquisition and application. Higher-order thinking… Statements that represent the integration (phase 3) and resolution (phase 4) stages of the practical inquiry model.

Social Presence Cognitive Presence Teaching Presence (Structure/Process) Educational Experience Supporting Discourse Setting Climate Selecting Content Community of Inquiry Model Garrison, Anderson & Archer, 2000

Deliberation (Applicability) Conception (Ideas) Perception (Awareness) Action (Practice) EXPERIENCE Exploration Integration Triggering Events Resolution Practical Inquiry Model Garrison, Anderson & Archer, 2001 1 2 3 4

Purpose of Study To examine how higher-order learning occurs through the chat process in a way that reflects the dynamic relationship between cognitive presence and critical thinking in a community of inquiry.

To examine how higher-order learning occurs through the chat process in a way that reflects the dynamic relationship between cognitive presence and critical thinking in a community of inquiry.

Previous Research Discussion in communities of inquiry contribute to higher-order thinking and helps learners create knowledge (Garrison et al., 2000) Others explored higher-order learning in terms of CP in asynchronous environments (Fabro & Garrison, 1998; Garrison, Anderson, & Archer, 2001; Meyer, 2003; Pawan, Paulus, Yalcin, & Chang, 2003) Higher-Order Thinking

Discussion in communities of inquiry contribute to higher-order thinking and helps learners create knowledge (Garrison et al., 2000)

Others explored higher-order learning in terms of CP in asynchronous environments (Fabro & Garrison, 1998; Garrison, Anderson, & Archer, 2001; Meyer, 2003; Pawan, Paulus, Yalcin, & Chang, 2003)

Previous Research Grounded in critical thinking (McPeck, 1981; Brookfield, 1987; Garrison, 1991) Learners construct meaning through sustained communication (Garrison et al., 2001) Fewer studies examine CP in real-time environments (Vaughn & Garrison, 2005) Limited empirical evidence suggests that text-based communications used in real-time environments can support/encourage the development & practice of higher-order thinking skills (Garrison et al., 2000) Cognitive Presence

Grounded in critical thinking (McPeck, 1981; Brookfield, 1987; Garrison, 1991)

Learners construct meaning through sustained communication (Garrison et al., 2001)

Fewer studies examine CP in real-time environments (Vaughn & Garrison, 2005)

Limited empirical evidence suggests that text-based communications used in real-time environments can support/encourage the development & practice of higher-order thinking skills (Garrison et al., 2000)

Research Questions How is higher-order thinking supported by the practical inquiry model in a community of inquiry? How do individuals progress through this model with the aide of collaborative learning and higher-order thinking?

How is higher-order thinking supported by the practical inquiry model in a community of inquiry?

How do individuals progress through this model with the aide of collaborative learning and higher-order thinking?

Method This ex post facto study used a quantitative content analysis to investigate the development of cognitive presence through the practical inquiry process.

This ex post facto study used a quantitative content analysis to investigate the development of cognitive presence through the practical inquiry process.

Course Background Learners studied the role of adult education in American society Seven groups formed by learners’ affinity or proximity to one another in initial class Five groups chose to work online and two chose to conduct their small group discussions face-to-face

Learners studied the role of adult education in American society

Seven groups formed by learners’ affinity or proximity to one another in initial class

Five groups chose to work online and two chose to conduct their small group discussions face-to-face

Course Activities Three face-to-face sessions: at beginning, middle, and end of course Weekly small-group discussions related to course readings and questions posed by instructor: Groups discussed issues using a chat learning space Group moderator synthesized discussion and posted to discussion board

Three face-to-face sessions: at beginning, middle, and end of course

Weekly small-group discussions related to course readings and questions posed by instructor:

Groups discussed issues using a chat learning space

Group moderator synthesized discussion and posted to discussion board

Study Design Of the groups available to us one was selected at random Time 1 (week 3) and Time 2 (week 7) of the group’s transcripts were analyzed

Of the groups available to us one was selected at random

Time 1 (week 3) and Time 2 (week 7) of the group’s transcripts were analyzed

Units of Meaning Chat Transcripts A complete participant response Example: “I know that this is vague, but we have to start somewhere. end”

Chat Transcripts

A complete participant response

Example: “I know that this is vague, but we have to start somewhere. end”

Coding Triggering Event (Phase 1) Exploration (Phase 2) Integration (Phase 3) Resolution (Phase 4)

Triggering Event (Phase 1)

Exploration (Phase 2)

Integration (Phase 3)

Resolution (Phase 4)

Krippendorff’s Alpha Coefficient For Interrater Reliability .81 .98 Two Coders .83 .89 Three Coders Chat 2 Transcript Chat 1 Transcript Number of Coders

Findings: Frequency of Individual Meaning Units Coded by Presence Type 1 χ 2(1, N = 193) = 4.36, p = .04 110 1 24 56 82 25 41 Total 14 4 11 - - - Jay 21 3 9 21 6 9 Gail 26 7 10 17 3 16 John 32 4 16 29 14 10 Ann 17 6 10 15 2 6 Rob Cognitive Presence Teaching Presence Social Presence Cognitive Presence Teaching Presence Social Presence Participant Name Time 2 – Chat Week 7 Time 1 – Chat Week 3

Findings: Frequency of Individual Meaning Units Coded as CP in Chat 1 3 23 43 13 Total - - - - Jay (absent) 2 8 8 3 Gail 0 4 10 3 John 0 10 15 4 Ann 1 1 10 3 Rob Resolution Integration Exploration Triggering Event Practical Inquiry Phase Chat 1 – Week 3

Findings: Frequency of Individual Meaning Units Coded as CP in Chat 2 1 χ 2(1, N = 126) = 12.70, p < .001 2 16 83 1 9 Total 0 3 10 1 Jay 0 4 15 2 Gail 1 3 22 0 John 1 5 23 3 Ann 0 1 13 3 Rob Resolution Integration Exploration Triggering Event Practical Inquiry Phase Chat 2 – Week 7

Findings: Flow of Social, Teaching, and Cognitive Presence in Chat 1 ExEx

Findings: Flow of Social, Teaching, and Cognitive Presence in Chat 2 Te In ExExExEx Te Ex ExExExExExEx Re Te ExExEx In Te Ex ExExEx Re Ex In Ex ExExExExExEx InInIn In ExEx Ex ExEx

Practical Inquiry Model for Chat Time 1

Practical Inquiry Model for Chat Time 2

Findings Cognitive presence accounted for the highest percentage of individual coded meaning units in chat 1 and chat 2. In both chats, exploration (phase 2) accounted for the highest number of individual meaning units and reflects deeper learning approaches. Social presence (SP) and teaching presence (TP) are necessary to move the conversation and learner’s experience to a higher cognitive level. Teaching presence not only joins SP and CP together, but the instructor’s (or moderator’s) role is crucial in facilitating critical thinking (Fabro & Garrison, 1998) in order to move the discussion to the next level of higher-order thinking.

Cognitive presence accounted for the highest percentage of individual coded meaning units in chat 1 and chat 2.

In both chats, exploration (phase 2) accounted for the highest number of individual meaning units and reflects deeper learning approaches.

Social presence (SP) and teaching presence (TP) are necessary to move the conversation and learner’s experience to a higher cognitive level.

Teaching presence not only joins SP and CP together, but the instructor’s (or moderator’s) role is crucial in facilitating critical thinking (Fabro & Garrison, 1998) in order to move the discussion to the next level of higher-order thinking.

Conclusions Communication leading to higher-order thinking is not cyclical. There is a pattern to how groups reach resolution, and this pattern is consistent across multiple studies. (Garrison et al., 2001; McKlin et al, 2002; Meyer, 2003; Vaughan & Garrison, 2003) Bulk of work in exploration phase, followed by integration

Communication leading to higher-order thinking is not cyclical.

There is a pattern to how groups reach resolution, and this pattern is consistent across multiple studies. (Garrison et al., 2001; McKlin et al, 2002; Meyer, 2003; Vaughan & Garrison, 2003)

Bulk of work in exploration phase, followed by integration

Recommendations Instructional course designers should consider the following when using chat learning spaces: Within the context of a community of inquiry, chat spaces increase higher-order thinking skills. Practical Inquiry Model (Garrison, Anderson & Archer, 2001) Chats culminate through well-defined tasks by joining CP, TP, and SP to assist the group in achieving resolution and collaborative learning. Community of Inquiry Model (Garrison, Anderson & Archer, 2000) Instructor’s / moderator’s role is crucial in facilitating critical thinking and encouraging the group to reach the highest pinnacle , resolution (phase 4).

Instructional course designers should consider the following when using chat learning spaces:

Within the context of a community of inquiry, chat spaces increase higher-order thinking skills.

Practical Inquiry Model (Garrison, Anderson & Archer, 2001)

Chats culminate through well-defined tasks by joining CP, TP, and SP to assist the group in achieving resolution and collaborative learning.

Community of Inquiry Model (Garrison, Anderson & Archer, 2000)

Instructor’s / moderator’s role is crucial in facilitating critical thinking and encouraging the group to reach the highest pinnacle , resolution (phase 4).

References Anderson, T.D. and Garrison, D.R. (1995). Critical thinking in distance education: Developing critical communities in an audio teleconference context. Higher Education, 29(2), 183-199. Anderson, T., Rourke, L., Archer, W., & Garrison, R. (2001). Assessing teaching presence in computer conferencing transcripts. The Journal of Asynchronous Learning Networks, 5( 2). Retrieved December 15, 2005, from http://www.aln.org/publications/jaln/v5n2/v5n2_anderson Brookfield, S.D. (1987). Developing critical thinkers, San Francisco, CA: Jossey-Bass. Brookfield, S. D., & Preskill, S. (2005). Discussion as a way of teaching: Tools and techniques for democratic classrooms, 2nd ed. San Francisco, CA: Jossey-Bass. Fabro, K.R. and Garrison, D.R. (1998). Computer conferencing and higher-order learning. Indian Journal of Open Learning, 7(1), 41-54. Garrison, D.R. (1991). Critical thinking and adult education: A conceptual model for developing critical thinking in adult learners. International Journal of Lifelong Learning, 10(4), 287-303. Garrison, R., Anderson, T., and Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2 (2-3), 87-105. Garrison, R., Anderson, T., and Archer, W. (2001). Critical thinking and computer conferencing: A model and tool to assess cognitive presence. American Journal of Distance Education, 15 (1), 7-23. Krippendorff, K. (2004). Content analysis: An introduction to its methodology , 2nd ed. Beverly Hills, CA: Sage. McPeck, J.E. (1981). Critical thinking and education. Oxford, UK: Martin Robertson. Meyer, K. A. (2003). Face-to-face versus threaded discussions: The role of time and higher-order thinking. Journal of Asynchronous Learning Networks, 7 (3). Retrieved June 16, 2006, from http://www.sloan-c.org/publications/jaln/v7n3/pdf/v7n3_meyer.pdf Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage. Pawan, F., Paulus, T. M., Yalcin, S., & Chang, C-F. (2003). Online learning: Patterns of engagement and interaction among in-service teachers. Language Learning & Technology, 7 (3), 118-140. Strijbos, J., Martens, R. L., Prins, F. J., & Jochems, W. M. (2005). Content analysis: What are they talking about? Computers & Education, 46(2006), 29-48. Retrieved June 16, 2006, from http://www.elsevier.com/locate/compedu Vaughan, N., & Garrison, D. R. (2005). Creating cognitive presence in a blended faculty development community. The Internet and Higher Education , 8, 1-12.

Anderson, T.D. and Garrison, D.R. (1995). Critical thinking in distance education: Developing critical communities in an audio teleconference context. Higher Education, 29(2), 183-199.

Anderson, T., Rourke, L., Archer, W., & Garrison, R. (2001). Assessing teaching presence in computer conferencing transcripts. The Journal of Asynchronous Learning Networks, 5( 2). Retrieved December 15, 2005, from http://www.aln.org/publications/jaln/v5n2/v5n2_anderson

Brookfield, S.D. (1987). Developing critical thinkers, San Francisco, CA: Jossey-Bass.

Brookfield, S. D., & Preskill, S. (2005). Discussion as a way of teaching: Tools and techniques for democratic classrooms, 2nd ed. San Francisco, CA: Jossey-Bass.

Fabro, K.R. and Garrison, D.R. (1998). Computer conferencing and higher-order learning. Indian Journal of Open Learning, 7(1), 41-54.

Garrison, D.R. (1991). Critical thinking and adult education: A conceptual model for developing critical thinking in adult learners. International Journal of Lifelong Learning, 10(4), 287-303.

Garrison, R., Anderson, T., and Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2 (2-3), 87-105.

Garrison, R., Anderson, T., and Archer, W. (2001). Critical thinking and computer conferencing: A model and tool to assess cognitive presence. American Journal of Distance Education, 15 (1), 7-23.

Krippendorff, K. (2004). Content analysis: An introduction to its methodology , 2nd ed. Beverly Hills, CA: Sage.

McPeck, J.E. (1981). Critical thinking and education. Oxford, UK: Martin Robertson.

Meyer, K. A. (2003). Face-to-face versus threaded discussions: The role of time and higher-order thinking. Journal of Asynchronous Learning Networks, 7 (3). Retrieved June 16, 2006, from http://www.sloan-c.org/publications/jaln/v7n3/pdf/v7n3_meyer.pdf

Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage.

Pawan, F., Paulus, T. M., Yalcin, S., & Chang, C-F. (2003). Online learning: Patterns of engagement and interaction among in-service teachers. Language Learning & Technology, 7 (3), 118-140.

Strijbos, J., Martens, R. L., Prins, F. J., & Jochems, W. M. (2005). Content analysis: What are they talking about? Computers & Education, 46(2006), 29-48. Retrieved June 16, 2006, from http://www.elsevier.com/locate/compedu

Vaughan, N., & Garrison, D. R. (2005). Creating cognitive presence in a blended faculty development community. The Internet and Higher Education , 8, 1-12.

Thanks!