Downloaded 17 times
![How evaluate
cMOOCs acc to
these criteria?
• Don’t measure each aspect of a cMOOC against these
as if a checklist; rather, consider cMOOCs a
“language” and a course as an expression in it
• These criteria should be considered “an aid, used to
assist a person who is already fluent in MOOC
design (or at least in the domain or discipline being
studied) [to] recognize the quality (or lack of quality)
of a MOOC” (Downes, 2013b).](https://image.slidesharecdn.com/opened2013-evaluatingcmoocscopy-131108111219-phpapp01/85/Difficulties-Evaluating-cMOOCS-Open-Education-Conference-2013-14-320.jpg)
Uploaded byChristina Hendricks
Difficulties Evaluating cMOOCS (Open Education Conference 2013)
The presentation discusses the challenges of evaluating connectivist MOOCs (cMOOCs) regarding participant autonomy and course effectiveness. It emphasizes the importance of understanding participant goals and the emergent knowledge produced through interaction, rather than simply measuring against a checklist of design specifications. Suggestions for evaluation include assessing long-term participant impacts and ensuring that cMOOC design aligns with its intended purposes.
More Related Content
Similar to Difficulties Evaluating cMOOCS (Open Education Conference 2013)
More from Christina Hendricks
Difficulties Evaluating cMOOCS (Open Education Conference 2013)
- 1. Difficulties evaluating cMOOCs: Negotiating Autonomyand Participation #DiffCMOOC Christina Hendricks University of British Columbia, Vancouver Open Education Conference, November 2013 Presentation licensed CC-BY
- 2. Connectivist MOOCs Network: Facilitating connections between peopleand information, ideas (not transmitting knowledge from central source) (Siemens 2012 http://is.gd/K5JfXK ) Distributed: Takes place in multiple spaces (blogs, wikis, tweets, discussion boards, webinars, etc.): “A MOOC is a web, not a website” (Downes 2013a http://www.downes.ca/presentation/327 ) #OOE13 Open Online Experience 2013-2014 http://www.ooe13.org
- 3. Connectivist MOOCs when &how to participate; Autonomy: Participants decide create own learning goals, choose own paths through course (McAuley et al. 2010 http://is.gd/6j1X1k; Downes 2009 http://is.gd/AYc84B) Open: free access available to anyone with reliable internet connection; curriculum open to alterations by participants (Downes, 2013b http://is.gd/Downes2013 ) From a video on MOOcs by Dave Cormier & Neal Gillis (licensed CC-BY) http://is.gd/cQwOSP
- 4. Evaluating cMOOC effectiveness Do theyachieve goals? Which goals? • Of designers • Of participants • Connectivism: making connections w/people & information • What sort of entities cMOOCs are & whether fulfill purposes (Downes)
- 5. Goals of cMOOC designers Participantautonomy: •What happens in course depends on what participants do: “learners are expected to actively contribute to the formation of the curriculum through conversations, discussions, and interactions” (Cormier & Siemens, 2010 http://is.gd/nqTED ) Hub & Spoke, flickr photo by Antony_Mayfield, licensed CC-BY •course may be successful (or fail) in ways designers never envisioned
- 6. Goals of participants E.g.,Lane, 2013 http://is.gd/W0360s •participants may have goals that don’t fit course; a problem if not fulfilled? • may not have any goals; just want to see what happens • course may have other benefits not captured in participants’ goals; may miss this if focus on their goals • benefits may take a long time to realize Huma Bird tweet analysis, #whyopen http://is.gd/4h5CFq
- 7. Goals of participants Whatone might do: •Ask participants at the end what they got out of the course, with or without reference to their original goals •Return to them six months or more later to ask again-perhaps see longer-term effects •Consider how to support learners in being self-directed, working to achieve own goals in a cMOOC (e.g., Kop, Fournier and Mak, 2011 http://is.gd/KopEtAl2011 )
- 8. Connectivism: connections among people &info • “Knowledge is defined as a particular pattern of relationships and learning is defined as the creation of new connections and patterns as well as the ability to maneuver around existing networks/patterns." (Siemens 2008 http://www.connectivism.ca/?p=116 ) • "At its heart, connectivism is the thesis that knowledge is distributed across a network of connections, and therefore that learning consists of the ability to construct and traverse those networks." (Downes, 2007 http://is.gd/Downes2007)
- 9. Connectivism: connections among people &info Participation rates: •surveys of participants: Milligan, Littlejohn and Margaryan, 2013 http://is.gd/MilliganEtAl2013 •log data from P2PU platform: Ahn, Weng and Butler, 2013 http://is.gd/AhnEtAl2013 •mixed methods: • Waite, Mackness, Roberts and Lovegrove, 2013 http://is.gd/WaiteEtAl2013 • Kop, 2011 http://is.gd/Kop2011
- 10. Connectivism: connections among people &info Negotiating autonomy & participation •Tension: need at least some active participation, but participants must have autonomy •Lurkers valued? Just b/c may become active participants? #ds106zone May 25-June 6, 2013 http://is.gd/o27mvc
- 11. Purposes of cMOOCs themselves Downes2013b http://is.gd/Downes2013 •look at what sorts of entities cMOOCs are, what purposes they serve, whether designed well for those purposes (rather than how they’re used) •To evaluate a cMOOC, consider: “what a successful MOOC ought to produce as output, without reference to existing (and frankly, very preliminary and very variable) usage.” (Ibid.) • output: “emergent knowledge”
- 12. Emergent knowledge: See somethingor say something: Jakarta, Flickr photo shared by Eric Fischer, licensed CC-BY Blue dots tweets; red dots Flickr, white dots both In a successful cMOOC, “the structure of the interactions produces new knowledge, that is, knowledge that was not present in any of the individual communications, but is produced as a result of the totality of the communications, in such a way that participants can through participation and immersion in this environment develop in their selves new (and typically unexpected) knowledge relevant to the domain.” (Downes, 2013b; emphasis added)
- 13. Networks that tendto produce emergent knowledge 1.Autonomy 2.Diversity 3.Openness 4.Interactivity/Connectedness (Downes, 2013b) http://is.gd/Downes2013 Anek Rang, Ek Sang, Flickr photo shared by Sanjay, licensed CC-BY
- 14. How evaluate cMOOCs accto these criteria? • Don’t measure each aspect of a cMOOC against these as if a checklist; rather, consider cMOOCs a “language” and a course as an expression in it • These criteria should be considered “an aid, used to assist a person who is already fluent in MOOC design (or at least in the domain or discipline being studied) [to] recognize the quality (or lack of quality) of a MOOC” (Downes, 2013b).
- 15. Questions & concerns aboutthis approach • How can we determine if emergent knowledge has been produced? Where would we look? Whom would we ask? • Seemingly exclusive focus on design and purpose of cMOOCs-doesn’t consider the experiences of participants E.g., participant experiences in a cMOOC: Mackness, Mak and Williams, 2010 Crowd, Flickr photo by James Cridland, lic http://is.gd/MacknessEtAl2010 ensed CC-BY (altered)
- 16. Footprints of emergence • Williams, Karousouand Mackness, 2011 http://is.gd/WilliamsEtAl2011 • Emergent and prescriptive learning--need balance Williams, Mackness and Gumtau, 2012 http://is.gd/WilliamsEtAl2012 - Draw “footprints” of courses to map degrees of prescriptive and emergent learning
- 17. Footprint for CCK08 publishedin Williams, Mackness & Gumtau 2012 http://is.gd/WilliamsEtAl2012 (licensed CC-BY) Centre: prescriptive learning Light area: apex of emergent learning Periphery: “edge of chaos” Map points based on 24 factors, in four clusters See wiki for factors, how to draw footprints, and more: http://footprints-of-emergence.wikispac /
- 18. Full circle • Footprintsare not meant to provide evaluations of courses by themselves • Instead, provides way to evaluate if course fits purposes (Footprints of emergence wiki: http://footprints-of-emergence.wikispaces.com/ ) • After have footprint, ask: “Is this appropriate, or fit, for the purpose and context of the course and for you, and/or the particular learners?” Back to the beginning...
- 19.
- 20. THANK YOU! Twitter: @clhendricksbc Blog:http://blogs.ubc.ca/chendricks Slides, video and bibliography at: http://is.gd/HendricksOpenEd2013
Editor's Notes
- #14 autonomy: do people make their own decisions about goals and objectives? Do they choose their own software, their own learning outcomes? -- “members of the network … employ their own goals and objectives, judgments and assessment of success in the process of interaction with others” (Downes, 2013c) diversitydo people speak different languages, come from different cultures, have different point of view, make different software selections, access different resources? If everybody does the same thing, then nothing new is generated by their interacting with each other; but if they are diverse, then their participation in the network produces new knowledge” oppenness: “does communication flow freely within and without the network, is there ease of joining (and leaving) the network? In a community, this means, are people able to communicate with each other, are they easily able to join the community, are they easily able to participate in community activities? Interactivity: “is the knowledge produced in the network produced as a result of the connectedness, as opposed to merely being propagated by the connectedness? If a signal is merely sent from one person to the next to the next, no new knowledge is generated. Rather, in a community that exhibits connectivist dynamics, knowledge is not merely distributed form one person to another, but is rather emergent from the communicative behaviour of the whole. The knowledge produced by the community is unique, it was possessed by no one person prior to the formation or interaction in the community.”
- #15 -- “the act of learning a discipline--a trade, for example, or a science, or a skill--is more like the learning of a language than it is like learning a set of facts. ... the bulk of expertise in a language ... [is] in fluency and recognition, cumulating [sic] in the (almost) intuitive understanding .... This sort of fluency is acquired by immersion in a language-speaking community (of which a MOOC is a characteristic example)” (Downes, 2013c)
- #17 emergent; “learning which arises out of the interaction between a number of people and resources, in which the learners organise and determine both the process and to some extent the learning destinations, both of which are unpredictable. — knowledge “created and distributed largely by the learners themselves” (Ibid., 43) "Emergent learning is likely to occur when many self-organising agents interact frequently and openly, with considerable degrees of freedom, but within specific constraints; no individual can see the whole picture; agents and system co-evolve." (Ibid., 45) presccriptive: hierarchical, transferred from central knowledge source to those who are going to learn, knowledge is pre-determined for learners 2. need balance of openness and constraint: if too open can become chaotic, people get lost and drop out, or people can end up in echo chambers b/c dissenting voices may not be heard; — need to have constraints to ensure inclusive, inviting environment for ppl to share, discuss, so learn from one another, try to ensure dissenting voices don’t get drowned out