Using the CoI
  to Assess
ID Strategies
     and
     New
Technologies
  in Online
   Courses
                Phil Ice, Ed...
Community of Inquiry Framework
 a process model of learning in online and
 blended educational environments
 grounded in...
social presence         cognitive presence




           teaching presence
Social Presence
 the ability of participants in a community of
 inquiry to project themselves socially and
 emotionally -...
Social Presence - Elements
 affective expression (expressing emotion,
 self-projection)
 open communication (learning cl...
Cognitive Presence
 the extent to which learners are able to
 construct and confirm meaning through
 sustained reflection...
Cognitive Presence - Elements
 triggering event (sense of puzzlement)
 exploration (sharing information & ideas)
 integ...
Teaching Presence
 the design, facilitation and direction of
 cognitive and social processes for the purpose
 of realizin...
Teaching Presence - Elements
 design and organization (setting curriculum &
 activities)
  facilitation (shaping constru...
Community of Inquiry Survey
 9 social presence items (3 affective
 expression, 3 open communication, 3 group
 cohesion)
...
CoI Survey Validation
 tested in graduate courses at four institutions
 in the US and Canada
 principal component factor...
Community of Inquiry Survey Instrument (draft v15)
Developed by Ben Arbaugh, Marti Cleveland-Innes, Sebastian Diaz, Randy ...
Social Presence
Affective Expression 
14. Getting to know other course participants gave me a sense of belonging in the co...
Cognitive Presence
Triggering Even
23. Problems posed increased my interest in course issues.
24. Course activities piqued...
Promises
Promises
Flavor’s of the Day
 Cloud computing and virtualized applications
 have tremendous promise
 However – the rise to promin...
New Applications are Good
 New applications that impact learning are
 better
 Higher Education often lags out of an
 abu...
What Adoption Looks Like
Who Adopts
Sometimes a Little is Enough
 The potential for overkill exists
 Just because extremely rich apps exist
 doesn’t mean th...
Audio Feedback
 Pilot Study revealed the following benefits of
 providing asynchronous audio feedback using
 Acrobat Pro:...
Audio Feedback & the CoI
 The following slides compare the findings of
 the multi-institutional CoI sample (n = 1085)
 th...
Teaching Presence
 The instructor was helpful in identifying
 areas of agreement and disagreement on
 course topics that ...
Teaching Presence
 The instructor provided feedback that helped
 me understand my strengths and weaknesses
 relative to t...
Social Presence
 Online or web-based communication is an
 excellent medium for social interaction.
   Summer 2007 / mean...
Cognitive Presence
 I felt motivated to explore content related
 questions.
   Summer 2007 / mean = 4.31
   Audio group...
Collaborative
    Tools
SaaS Word Processors
 Two online, graduate level education courses
 (combined n=78) students were asked to
 complete assi...
Document Analysis
 average use of non-text based resources (e.g.
 hyperlinks, graphics, tables, etc.) was 5.1 for
 Word s...
Interview Data
 Buzzword’s ease of use (as compared to
 Word) cited as primary reason for inclusion of
 more links, graph...
Interview Data
 The concept of a Buzzword being a personal
  reflective space was common among 72% of students
  over 34 ...
Explaining
    CoI
Survey Data
Understanding Why
 The CoI survey and rubrics based off of it can tell you
  what is happening but not why
 Other measur...
Rich Internet Application Study
 Full Sail University – Web Design and Development
  Program
 Study conducted in Deploym...
Grading Rubric Differences

Aesthetics
       Significant Difference – 34.1% increase

Layout
           Significant Differe...
Student Interview Data
 11 Participants
    Level of engagement
    Perceived Learning
    Level of thought required
 ...
Student Interview Data


 Cursory Data Analysis
    All believed Photo Viewer activity to be more
     engaging
    9 b...
Designing for
•  Director of Course Design, Research &
   Development
•  American Public University System


 Meaningful
•...
Socio-Epistemological Orientations

 Social – Group vs. Individual

 Epistemological
    Objectivist – lower order thou...
Setting and Coding

 American Public University System – fully online
institution

 Eight undergraduate and eight gradua...
Sample and Data Collection

 CoI Survey administered for six course terms in
all sections of courses that were coded

 N...
Research Question

 Does epistemological orientation influence factor
loading patterns?

 Are other variables responsibl...
Findings I

 Factor analysis of all courses combined produced
a three factor solution

 Factor analysis of all undergrad...
Findings II

 Age banding 18 - 22, 23 - 27, 28 - 32, 33 - 37,
38 - 42, 43 - 47, 48 - 52, 53 - 57, 58 – 62

 Undergraduat...
Findings III

 Factor analysis by age band
    18 - 22 produce a 2 factor solution regardless
   of epistemological orie...
Observations

 Students between 23 - 37 appear to find ways to
collaborate or view learning as a collaborative
process re...
Future Research I

 How does the perception of learning activities
differ between students 23 – 27 years old and their
pe...
Future Research II

 Multi-institutional data

 Substantial qualitative work

 Hierarchical linear modeling
Thank You!
        •  Director of Course Design, Research &
           Development
Phil Ice, Ed.D.
Director ofAmerican Pub...
Upcoming SlideShare
Loading in...5
×

Phil Ice's: Using the Community of Inquiry Framework to Assess the Impact of Instructional Design Strategies and New Technologies in Online Courses

2,477

Published on

SLN SOLsummit 2010
http://slnsolsummit2010.edublogs.org
February 25, 2010


Phil Ice, Director of Course Design, Research & Development, American Public University System

Using the Community of Inquiry Framework to Assess the Impact of Instructional Design Strategies and New Technologies in Online Courses
This presentation will examine how the efficacy of instructional design components and new online learning technologies can be assessed with indicators of the Community of Inquiry Framework (CoI). The CoI framework has attracted considerable interest and has been used extensively to study and design online educational environments (Garrison & Arbough, 2007). The CoI explains the online learning experience as a function of three overlapping presences – social, cognitive, and teaching. The construct was validated through factor analysis by a multi-institutional team of researchers in 2007 (Swan, Richardson, Ice, Garrison, Cleaveland-Innes & Arbough, 2008), however, many questions remain as to what factors influence the effective projection of each presence. As the model is based on constructivist learning theory, the impact of well designed instruction and pedagogically based application of new technologies should impact the level and quality of interactions probed by the CoI indicators. This session will examine how quantitative and qualitative analysis of course outcomes, using the CoI survey instrument and associated rubrics can be applied to continuous quality improvement from an instructional design perspective. Participants will be provided with instruments, analysis techniques and ideas or application in their own practice.

Published in: Education, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,477
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
70
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Phil Ice's: Using the Community of Inquiry Framework to Assess the Impact of Instructional Design Strategies and New Technologies in Online Courses

  1. 1. Using the CoI to Assess ID Strategies and New Technologies in Online Courses Phil Ice, Ed.D. SLN SOL Summit Syracuse, 2010
  2. 2. Community of Inquiry Framework  a process model of learning in online and blended educational environments  grounded in a collaborative constructivist view of higher education  assumes effective online learning requires the development of a community of learners that supports meaningful inquiry and deep learning
  3. 3. social presence cognitive presence teaching presence
  4. 4. Social Presence  the ability of participants in a community of inquiry to project themselves socially and emotionally -- as ‘real’ people  the degree to which participants in computer mediated communication feel socially and emotionally connected
  5. 5. Social Presence - Elements  affective expression (expressing emotion, self-projection)  open communication (learning climate, risk free expression)  group cohesion (group identity, collaboration)
  6. 6. Cognitive Presence  the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse in a critical community of inquiry
  7. 7. Cognitive Presence - Elements  triggering event (sense of puzzlement)  exploration (sharing information & ideas)  integration (connecting ideas)  resolution (synthesizing & applying new ideas)
  8. 8. Teaching Presence  the design, facilitation and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes
  9. 9. Teaching Presence - Elements  design and organization (setting curriculum & activities)   facilitation (shaping constructive discourse)   direct instruction (focusing & resolving issues)
  10. 10. Community of Inquiry Survey  9 social presence items (3 affective expression, 3 open communication, 3 group cohesion)  12 cognitive presence items (3 triggering, 3 exploration, 3 integration, 3 resolution)  13 teaching presence items (4 design & facilitation, 6 facilitation of discourse, 3 direct instruction)
  11. 11. CoI Survey Validation  tested in graduate courses at four institutions in the US and Canada  principal component factor analysis  three factor model predicted by CoI framework confirmed  Arbaugh, Cleveland-Innes, Diaz, Garrison, Ice, Richardson, Shea & Swan - 2008
  12. 12. Community of Inquiry Survey Instrument (draft v15) Developed by Ben Arbaugh, Marti Cleveland-Innes, Sebastian Diaz, Randy Garrison, Phil Ice, Jennifer Richardson, Peter Shea & Karen Swan Teaching Presence Design & Organization 1. The instructor clearly communicated important course topi 2. The instructor clearly communicated important course goals. 3. The instructor provided clear instructions on how to participate in course learning activities. 4. The instructor clearly communicated important due dates/time frames for learning activities. Facilitation of Discourse 5. The instructor was helpful in identifying areas of agreement and disagreement on course topics that helped me to learn. 6. The instructor was helpful in guiding the class towards understanding course topics in a way that helped me clarify my thinking.  7. The instructor helped to keep course participants engaged and participating in productive dialogue. 8. The instructor helped keep the course participants on task in a way that helped me to learn. 9. The instructor encouraged course participants to explore new concepts in this course. 10. Instructor actions reinforced the development of a sense of community among course participants. Direct Instruction 11. The instructor helped to focus discussion on relevant issues in a way that helped me to learn. 12. The instructor provided feedback that helped me understand my strengths and weaknesses. 13. The instructor provided feedback in a timely fashion.
  13. 13. Social Presence Affective Expression  14. Getting to know other course participants gave me a sense of belonging in the course. 15. I was able to form distinct impressions of some course participants. 16. Online or web-based communication is an excellent medium for social interaction. Open communication 17. I felt comfortable conversing through the online medium.  18. I felt comfortable participating in the course discussions.  19. I felt comfortable interacting with other course participants. Group cohesion 20. I felt comfortable disagreeing with other course participants while still maintaining a sense of trust. 21. I felt that my point of view was acknowledged by other course participants. 22. Online discussions help me to develop a sense of collaboration.
  14. 14. Cognitive Presence Triggering Even 23. Problems posed increased my interest in course issues. 24. Course activities piqued my curiosity. 25. I felt motivated to explore content related questions. Exploration 26. I utilized a variety of information sources to explore problems posed in this course. 27. Brainstorming and finding relevant information helped me resolve content related questions. 28. Discussing course content with my classmates was valuable in helping me appreciate different perspectives. Integration 29. Combining new information helped me answer questions raised in course activities. 30. Learning activities helped me construct explanations/solutions. 31. Reflection on course content and discussions helped me understand fundamental concepts in this class. Resolution 32. I can describe ways to test and apply the knowledge created in this course. 33. I have developed solutions to course problems that can be applied in practice. 34. I can apply the knowledge created in this course to my work or other non-class related activities.
  15. 15. Promises Promises
  16. 16. Flavor’s of the Day  Cloud computing and virtualized applications have tremendous promise  However – the rise to prominence is so rapid that adequate evaluation is difficult  Longevity OR extensibility need to be considered prior to adoption to maximize ROI
  17. 17. New Applications are Good  New applications that impact learning are better  Higher Education often lags out of an abundance of caution  Academics want to see learning outcomes before they are willing to adopt  Give your faculty what they want
  18. 18. What Adoption Looks Like
  19. 19. Who Adopts
  20. 20. Sometimes a Little is Enough  The potential for overkill exists  Just because extremely rich apps exist doesn’t mean they are always needed  Remember that distance learners are isolated from the instructor and classmates physically  This may be by choice  But they still want some contact
  21. 21. Audio Feedback  Pilot Study revealed the following benefits of providing asynchronous audio feedback using Acrobat Pro:  THEME 1 – Ability to understand nuance.  THEME 2 – Feelings of increased involvement.  THEME 3 – Increased content retention.  THEME 4 – Instructor caring.
  22. 22. Audio Feedback & the CoI  The following slides compare the findings of the multi-institutional CoI sample (n = 1085) that received text-based feedback and responses from a multi-institutional sample(n = 1138) that received audio feedback   In the items addressed there was a significant difference (p > .05) in responses
  23. 23. Teaching Presence  The instructor was helpful in identifying areas of agreement and disagreement on course topics that helped me to learn.  Summer 2007 / mean = 4.12  Audio group / mean = 4.43  The instructor encouraged course participants to explore new concepts in this course.  Summer 2007 / mean = 4.44  Audio group / mean = 4.58
  24. 24. Teaching Presence  The instructor provided feedback that helped me understand my strengths and weaknesses relative to the course’s goals and objectives.  Summer 2007 / mean = 4.28  Audio group / mean = 4.57
  25. 25. Social Presence  Online or web-based communication is an excellent medium for social interaction.  Summer 2007 / mean = 3.90  Audio group / mean = 4.27
  26. 26. Cognitive Presence  I felt motivated to explore content related questions.  Summer 2007 / mean = 4.31  Audio group / mean = 4.55  Reflection on course content and discussions helped me understand fundamental concepts in this class.  Summer 2007 / mean = 4.37  Audio group / mean = 4.49
  27. 27. Collaborative Tools
  28. 28. SaaS Word Processors  Two online, graduate level education courses (combined n=78) students were asked to complete assignments (mean page count = 9.75 pages), alternating Word and Buzzword as the creation and submission mechanism.  Buzzword is an online document editor that is a part of Acrobat.com
  29. 29. Document Analysis  average use of non-text based resources (e.g. hyperlinks, graphics, tables, etc.) was 5.1 for Word submissions and 14.3 for Buzzword based submissions.  Cognitive Presence Indicators Exploration Integration Resolution Word 10.8 3.3 2.3 Buzzword 11.2 6.2 5.1
  30. 30. Interview Data  Buzzword’s ease of use (as compared to Word) cited as primary reason for inclusion of more links, graphics, etc.  Majority (n = 12) indicated that it was easier for them to express complex concepts using multimedia.
  31. 31. Interview Data  The concept of a Buzzword being a personal reflective space was common among 72% of students over 34 years old.  As an example these students frequently developed a document on their own and then shared it with classmates.  82% of students 34 and younger recognized it as a collaborative tool and began workflow projects by sharing a common document.  Qualitative analysis revealed that these younger students frequently (68%) likened Buzzword to a Wiki or similar collaborative tool. In contrast, only 27% of older students made this association.
  32. 32. Explaining CoI Survey Data
  33. 33. Understanding Why  The CoI survey and rubrics based off of it can tell you what is happening but not why  Other measures are needed  Grading Rubrics and Student Interviews make great sources of data  Involve your faculty – this is data they may already have
  34. 34. Rich Internet Application Study  Full Sail University – Web Design and Development Program  Study conducted in Deployment of Flash Projects Course  Implement Flash via Multiple Deployment Types  HTML-based Deployments with Flash Content  Full Flash Deployments using FlashPlayer and AIR Runtime  Significant gains on 4 Cognitive Presence items – positive for RIA’s over conventional HTML applications
  35. 35. Grading Rubric Differences Aesthetics Significant Difference – 34.1% increase Layout Significant Difference – 30.5% increase Follow Through No Significant Difference Craftsmanship Significant Difference – 37.7% increase Validation No Significant Difference Architecture Significant Difference – 35.6% increase Functionality Significant Difference – 46.2% increase
  36. 36. Student Interview Data  11 Participants  Level of engagement  Perceived Learning  Level of thought required  Applicability to future coursework and career
  37. 37. Student Interview Data  Cursory Data Analysis  All believed Photo Viewer activity to be more engaging  9 believed they learning more from Photo View activity  All believed Photo View activity required more higher order thought  9 believed Photo Viewer activity would be more relevant to the career ambitions
  38. 38. Designing for •  Director of Course Design, Research & Development •  American Public University System Meaningful •  pice@apus.edu Learning
  39. 39. Socio-Epistemological Orientations  Social – Group vs. Individual  Epistemological  Objectivist – lower order thought processes vis-à-vis Bloom’s Taxonomy  Constructivist – higher order thought processes vis-à-vis Bloom’s Taxonomy  Student satisfaction and perceptions of Community may be impacted by the instructors Socio-Epistemological orientation – as projected in content and interactivity (Arbaugh & Benbunnan-Fich, 2006)
  40. 40. Setting and Coding  American Public University System – fully online institution  Eight undergraduate and eight graduate level courses were coded for epistemological orientations  Lower three levels of Bloom’s coded as objectivist  Higher three levels of Bloom’s coded as constructivist  Coding of all course activities and discussions  Majority of indicators determined classification
  41. 41. Sample and Data Collection  CoI Survey administered for six course terms in all sections of courses that were coded  N = 4397  Undergraduate – 2576  Graduate – 1821  Factor Analysis ran:  Overall  By level  By course  By five year age bands  By clusters – defined by school
  42. 42. Research Question  Does epistemological orientation influence factor loading patterns?  Are other variables responsible for factor loading patterns?  Impetus – despite validation of the CoI in 2008, a few subsequent factor analyses have produced a two factor solution  Anecdotal evidence – two factor solution appeared among groups where the emphasis was on training as opposed to true knowledge acquistion
  43. 43. Findings I  Factor analysis of all courses combined produced a three factor solution  Factor analysis of all undergraduate courses combined produced a three factor solution  Factor analysis of all graduate courses combined produced a three factor solution  Factor analysis of individual courses (n range of 221 - 405) produced a three factor solution  Factor analysis by school produced three factor solutions
  44. 44. Findings II  Age banding 18 - 22, 23 - 27, 28 - 32, 33 - 37, 38 - 42, 43 - 47, 48 - 52, 53 - 57, 58 – 62  Undergraduate maximum age band = 43 – 47  Graduate minimum age band = 23 – 27
  45. 45. Findings III  Factor analysis by age band  18 - 22 produce a 2 factor solution regardless of epistemological orientation or course level  23 - 37 produce 3 factor solution regardless of epistemological orientation or course level  38 - 62 overall produce a 3 factor solution overall  38 - 47 produce a 2 factor solution when the epistemological orientation is objectivist  38 - 47 produce a 3 factor solution when the epistemological orientation is constructivist  48 - 62 produce a 2 factor solution regardless of epistemological orientation or course level
  46. 46. Observations  Students between 23 - 37 appear to find ways to collaborate or view learning as a collaborative process regardless of level  Students 18 - 22 appear to view teaching and cognitive presence as the same construct regardless of course orientation  Students 48 - 62 appear to view teaching and cognitive presence as the same construct  Students 38 - 47 appear to be influenced by the epistemological orientation of course materials and activities
  47. 47. Future Research I  How does the perception of learning activities differ between students 23 – 27 years old and their peers  Why do students 18 - 22 not transfer native social networking and collaboration skills to learning  How can life skills be used to leverage learning for students 48 - 62 years old  Why is epistemological orientation significant for students 38 - 47 and not other age groupings
  48. 48. Future Research II  Multi-institutional data  Substantial qualitative work  Hierarchical linear modeling
  49. 49. Thank You! •  Director of Course Design, Research & Development Phil Ice, Ed.D. Director ofAmerican Public University System •  Course Design, Research & Development American Public University System •  pice@apus.edu pice@apus.edu
  1. ¿Le ha llamado la atención una diapositiva en particular?

    Recortar diapositivas es una manera útil de recopilar información importante para consultarla más tarde.

×