Yannis@brisbane cipl research_seminar_20120717


Published on

Orchestrating CSCL, Research seminar, CIPLUniversity of Queensland, Brisbane, July 19, 20122

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Special thanks to Luis Palblo Prieto (doctoral student) and Juan Ignacio Asensio (co-director), Sara Villagrá (doctoral student) and Iván Manuel Jorrín (director) from the Technology and Education “divisions” of GSIC respectively.
  • Complex TEL/CSCL ecosystems Multiple tasks activities, multiple social levels, multiple tools How can we manage them and survive (ina sustainable way!), especially the teachers?
  • Technology-enhanced classrooms or distributed learning environments are becoming increasingly complex ecosystems, which may include VLEs, Web 2.0 tools, 3D worlds, or tangible artifacts, either digital or not. In CSCL environments, multiple social planes are involved in multiple teaching and learning activities that may take place using multiple devices or tools. Orchestration of such collaborative ecosystems is challenging, since it may require creation and flexible deployment of learning designs or real-time class monitoring and management. This seminar presents a view of conceptual and technological tools that may support teachers in orchestrating CSCL classes. Different aspects of orchestration will be discussed focusing on design and atomic enactment patterns, as mediating conceptual artifacts. On the other hand, various technology-based tools will be presented that enable the teachers perform the complete orchestration life cycle. The presentation will draw on data from two years of field studies in a primary classroom, as well as professional development workshops and field studies with university teachers, where the conceptual and technological tools have been employed.
  • Cite main people and proposals, especially Dillenbourg Mention here and in the beginning the work by Luis Pablo and his publications and thesis (and Sara, etc.)
  • Cite main people and proposals, especially Dillenbourg Mention here and in the beginning the work by Luis Pablo and his publications and thesis (and Sara, etc.)
  • Will be used in future: 4.43 (international) – 4.40 (spanish) Provided new insights: 4.17 (international) – 3.36 (spanish)
  • CLFP and assessment patterns
  • Related and structured according to the models, maps or pattern languages
  • Learning Design composed of JIGSAW learning flow pattern And the REPORT REVIEW and OBSERVATIONS OF COLABORATIVE WORK assessment patterns
  • From 2 LD formalizations: one widely used (IMS-LD) and one specialized/higly expressive (LDL) ... ... to 2 different LEs: one widely used (Moodle) and one peculiar/lo-fi (MediaWiki) GLUE!-PS WORKSHOP EVALUATION * 24 teachers, multi-disciplinary, most had experience with CL, patterns, routines... from a previous workshop * Tried both deployment and (some) real-time flexibility (but Moodle is limited in this regard!) * Questionnaire responses indicated overwhelmingly positive feedback from respondents (21/24): all were able to deploy, only 2 could not deploy their own design (because of lack of time) ** Very good ratings for usability (in a 1-8 scale, WebCollage 6.76, GLUEPS 7, Moodle 6.47!), usefulness (7.38), time-efficiency (7.14) ** Good ratings for immediate future use (in a 1-8 scale, 6.19), still some reluctance due to beliefs (do not like CL, or prefer pen/paper), or lack of adequate support * Mini-focus groups (debates) indicated very positive feedback, with a few "buts": ** using CL (independent of CS) would depend very much on the context (subject, number of students, own expertise), sp. since current education/evaluation paradigm is very individualistic ** positive feedback on GLUEPS (intuitive, flexible), but they need more technical support (human resources, manuals, courses...) and reliability of ICT tools ** Some other well-known issues: some teachers still think that ICT is more complex than social/pen&paper, to do basically the same things, students sometimes prefer the individual/IRE mode
  • ATOMIC PATTERNS AND GLUE!-PS MAIN IDEAS * Of the original 169 atomic patterns in the catalogue, 21 were selected for eventual implementation (on the basis of: the AP being implementable through technology, having APs from all orchestration aspects, APs that appear frequently) * From these 21, GLUEPS functionalities have been abstracted, and generated an implementation roadmap * The implementation has shown certain architecture/system issues (e.g. having an API favors flexibility in real-time, synchronizing the VLE and GLUEPS/GLUE is difficult) IMPLEMENTATION * 8 of them were supported "by design" Monitoring the task On -the- fly assessment (Monitoring + Formative Assessment) Use results of a task in a different task / Reuse generated artifacts Teacher as participant in student groups Teacher chooses tools Use Moodle to automate submissions Use wiki to do collaborative writing Use wiki to structure activities * the next 5 most common have been implemented Strategies to group formation / Form groups Reform groups in face of the current attendants (only in MW) Spontaneous use of additional ICT (only in MW) Successive deployment of activities (only in Moodle, for now) Use pre-existing groups (only in Moodle) * 1-2 more are being implemented till the end of the thesis Peer review Adding improvised activities TRIED WITH SARA AND BEA * Authentic experiences, one with Moodle, one with MW * TICs-Sara2 (WebCollage -> MediaWiki, jigsaw pattern) ** Real-time changes in groups (many!) were needed. The teacher chose to make changes manually (it was feasible because it had no external tool), because the changes would have overwritten manual changes that she did after deploying ** Underlying issue to take into account in DLEs: state synchronization among elements, in both ways (VLE <-> GLUEPS, GLUElet Manager...), which is not trivial without modifying the VLE ** The teacher tried the real-time changes afterwards and valued GLUEPS very positively ** Still, a lot of time was needed to make sense of the (large) jigsaw and seeing if all instrinsic constraints were complied with. GLUEPS does not help with that * OyT-Bea (Pattern Collector -> Moodle, non-CLFP role playing) ** Integrate the design into an existing, ongoing Moodle course. Large number of groups/students (16 groups, around 60 students) ** Slight usability problems, resolved as the experiment went on ** Incremental deployments, to help with contingent elements of the design (i.e. materials/resources that could not be known in design-time) ** Teacher valued highly the usability enhancements, but... ** ... instantiation of groups and resources still took a lot of time. More automation/intelligence needed! (e.g. we will implement the peer review atomic pattern, and tell Bea to do it again) *** social coordination is still a desirable option, if you are willing to decrease usability for students a little bit (e.g. put all links to resources in the moodle, and tell socially who reviews what - instead of hardwiring the data flow as GLUEPS does). However, in larger groups social coordination might not be feasible...
  • We begin to bridge the deployment gap in DLEs - time is saved and the process is less error-prone ** ... but the real time savings is in sharing/reusing (e.g. deploy a design in 8 minutes when reusing, vs. in 48 min without GLUEPS) GLUEPS makes possible the deployment of LDs in DLEs which teachers thought were not feasible ** But still...*** ... average teachers (e.g. in our PD workshops) are reluctant to rely so much on ICT (they need support, help, tutorials, time to try...) also a hint for managers/policy-makers *** ... many operations in instantiation are repetitive (e.g. peer reviews con Bea) - there is still room for improvement by automating * More naturalistic evaluation needed (up to now, all cases were "special teachers")! ** Highlight the role of PD workshops to disseminate and get feedback from "more real" people... and "capture" them for experiments ;)
  • From 5+3 ** do we (or do researchers) really want orchestration to be defined? do we want an "orchestration theory"? from our study, not everybody does! ** descriptive frameworks are fine, but we also need a more normative frame, for concrete advice: is it possible to give advice, regardless of context? can atomic patterns be the beginning of an answer? Are we really sustainable? ** Atomic patterns are "expensive" to elicit, and are quite contextual... could we "crowdsource" it to e.g. teachers? (but let us remember the failure of Crea-TIC - what is in it for the (average) teacher?) ** The GSIC technological tools currently put a big orchestration burden/responsibility on the teacher... could we co-orchestrate with students (e.g. teacher defines high-level structure, students choose the tools and other fine-grained stuff) - but still maintaining teacher control somehow (e.g. for monitoring)? * Security, ethical questions... where is the data? is it protected? (e.g. CSCL-EREM tool does not want your data, only links to it) permissions across this complex service ecosystem... (e.g. see the problems GLUE is having with implementing supposedly standard stuff like Oauth) * To which extent our current CSCL focus can be extended to more integrated curriculum (which includes, IBL, even more IRE stuff...)?
  • Yannis@brisbane cipl research_seminar_20120717

    1. 1. University of Queensland, Centre for Innovation in Professional Learning, Lunchtime research seminarOrchestrating Computer Supported Collaborative Learning (CSCL) Prof. Yannis Dimitriadis GSIC/EMIC research group University of Valladolid July 19, 2012 Based on work by L.P. Prieto – S. Villagrá
    2. 2. TEL/CSCL ecosystem 2
    3. 3. Outlines The two hats of this seminars Hat 1: Conceptual tools s A framework on orchestration s Multi-level patterns as mediating toolss Hat 2: Technological tools for s Learning design s Flexible deployment 3
    4. 4. The two hats of this seminars Personal s Engineering background + Interest in educations Audience ? s Researchers in TEL, e-learning designerss TEL research s sound pedagogy + efficient engineerings Quest for sustainable innovation s good research for understanding, abstraction and evidence s effective support to actors – teachers, learners or even service and technology providers 4
    5. 5. The two hats of GSIC/EMIC s “Dark and bright side” in a group since 1994 s Focus on CSCL: complex and innovative s Support real-world actors and own action-research s Inquiry cycle (design-enactment-evaluation) for s teaching/learning and technological support 5
    6. 6. Hat 1: Conceptual toolss Analyze and understand the field of orchestration in complex TEL ecosystems s The “5+3 aspects” framework for orchestrations Collect and use knowledge in terms of good practices during the CSCL cycle s Multi-level patterns 6
    7. 7. Orchestration: Moving targets Increasing presence in literature, but what is it about? Is there a new focus?s Efforts to define the field/concept/metaphor: s Stellar NoE deliverable, CSCL 2011 and ICLS 2012 workshops, collective paper s New proposal for “5+3” orchestration framework and validation through expert panel 7
    8. 8. Focus of orchestration on …s Acknowledge complex integrated ecosystems for innovative TEL/CSCL environmentss Support teachers as orchestrators (and think of how to minimize-distribute the orchestration load on teachers, learners and systems)s Build on pragmatism and minimalism as keys for sustainabilitys Explicit emphasis on how to flexibly manage (learning time) enactments Design for adequate orchestrating and orchestrable technologies (and conceptual tools) 8
    9. 9. The “5+3” frameworkA diagramatic viewhttp://prezi.com/aa2vighak7hh/orchestration-in-tel-cscl-as-easy-as-53/ 9
    10. 10. The “5+3” frameworkA process view 10
    11. 11. Evaluating the frameworkThe setups Research questions on s Completeness of common understanding s Usefulness for researcherss Two panels (March – May 2012) s Pilot phase: 22 Spanish researchers of low- medium expertise (2.58) and more technological profile (16T-7E-1M) s International panel: 24 researchers of medium- high expertise (4,46), high research experience in TEL (15,8 years), more balanced profile (6T-11E-7M) 11
    12. 12. Completeness of frameworks Quantitative and qualitative datas Non-normative and inclusives Need to clarify role of actors, technology, theorys More than 4,5 (Lickert scale of 1-6) for all itemss Logical: 4,78 –Comprehensive: 4,52 –Relevant: 5,13 12
    13. 13. Usefulness of frameworks To be referenced in papers, checklist and didactic tool for young researcherss But need for normative advice and real-world exampless Will be used in future (4.43)s Provided new insights (4.17) 13
    14. 14. Multi-level patternss But the “5+3” Orchestration framework is a conceptual tool mainly for researcherss Tools (conceptual or technological) for orchestration should also support “teachers” s Exploiting articulated (design) knowledges (Pedagogical) patterns as mediating tools regarding various orchestration aspects s Design and Management of learning flow, Assessment, Adaptation …s Related fields of s Learning Design, Educational Modeling, Scripting 14
    15. 15. Collaborative Learning Flowand assessment patternss Learning Flows Assessment 15
    16. 16. Aligning both sets of patterns 16
    17. 17. Usefulness of patternss Are these patterns useful for practitioners? s Evidence for learning flow patterns for OER repurposing to collaborative activities s Appropriation in University (Higher Education – HE) teachers (Professional Development - PD) workshopss But not always s If not included in appropriate technological tools s Too complex (abstract) for (Primary Education – PE) practitioners s Do not take into account enactment issues 17
    18. 18. The need for atomic patterns s Two independent studies by GSIC and SRI using GroupScribbles in Primary and Secondary Education schools (2008-2011) s Studying enactment and disciplined improvisation s Showed that other small-scale, informal, contextual, actionable patterns emerged 18
    19. 19. A learning design in actionin a Primary Education classroom 19
    20. 20. Using atomic patternsin a HE PD workshop 20
    21. 21. Atomic patternsas mediating conceptual toolss And we need to “fill the gap”, “flesh the bones” with appropriate mediating artifacts (conceptual tools) 21
    22. 22. Evaluation of atomic patternsin PD workshopss Four (2 in PE, 2 in HE) PD workshops (2010-2011)s WS4 data quantitative data (scale 1 to 7): s Enactment patterns useful (5,31) s Feasibility (5,22) and nearness (6,06) s 19 out of 22 teachers enriched designs Useful but not revolutionarys Especially important for non-experienced or pre- service teacherss Need to condense and classify long list of APs What about inclusion in technological tools? 22
    23. 23. Approach for multi-level patternss And the way to elicit and use multi-level patterns 23
    24. 24. Hat 1: Wrap-ups Knowledge regarding orchestration for researchers: s “3+5 Aspects” framework s Approach for multi-level patterns s Atomic pattern catalogue and LD representations Support teachers too with conceptual tools: s Classroom norms for participation, discourse moves for discussion, decision rules for contingent teaching (SRI) s Multi-level patterns s Use in PD workshops Role playing and learning design enrichment using cards (GSIC) 24
    25. 25. Hat 2: Technological toolss But TEL is not only about s Conceptual tools (patterns, moves, …)s It is also about s enacting learning in TEL complex ecosystems s especially in HE distributed learning environments where VLE/PLE are combined with third-party toolss And also support teachers (and learners) with technological tools when s orchestrating TEL/CSCL ecosystems (less orchestration load for teachers – more for tools) 25
    26. 26. Learning Design toolss Articulate/exploit the knowledge of patterns and support the learning design processs Create a model or representation of a learning design (unit of learning, script, scenario, etc.) s That allows understanding and reflecting on the pedagogy, making the learning design explicit, sharing it with otherss Through the Web Collage tools Or other Learning tools by the Stellar Learning Design Grid Theme Team http://www.ld-grid.org/home 26
    27. 27. Web Collage:Learning & Assessment flows Students Teacher Select and work Support individual individually in a work subproblem Write activity input for… Review report report Work in expert Work expert groups of students groups students provide Support expert with the same with same feedback… group work subproblem subproblem Form Jigsaw groups and explain each Support jigsaw other the different group work subproblems 27 http://pandora.tel.uva.es/wic2
    28. 28. A learning design(HE teacher in a PD workshop) 28
    29. 29. Web Collage:Evolution and evaluations Evolution of the Collage tool since 2005 s Towards a more stable and user-friendly Web- based tool that includes learning and assessment patterns, as well as support for instantiations Extensive use and evaluation using mixed methods s Multi-case studies in authentic settings and PD workshops with HE teachers s Wide acceptance of the pattern-based design process 29
    30. 30. The deployment gaps But teachers wanted to complete the cycle s deploy their design in their TEL environment, especially their institutional VLE (e.g. Moodle) 30
    31. 31. Alternatives for deployments Have the learning design “in mind” and then go through the VLE/PLE interface and struggle to “translate” the design to a course/lesson plans Use a single system (LAMS) to do both design and deployment in the same environments Use an LD Tool (LDSE) to create the interpretable script and then deploy it to one environments But there are many LD tools and VLEs: s Provide intermediary to connect them: Glue!-PS 31
    32. 32. Glue!-PSAn architectural view 32
    33. 33. Glue!-PSAn architectural view - II 33
    34. 34. Glue!-PS:A functional view 34
    35. 35. Glue!-PSValidations Proof of concept validations using the Planet Game (ICALT 2006) scenario s Two LD languages: LDL + IMS-LD s Two Learning Environments: Moodle + MediaWikis Design and deployment s by GSIC researchers of 36 learning designs created using pen and paper at two HE PD workshops (Sep-Nov 2011) s by 24 teachers in a follow-up PD workshop (Feb 2012) 35
    36. 36. Glue!-PS and atomic patternss Atomic patterns are especially important for flexible orchestrations University teachers suggested embedding atomic patterns in technological toolss Initial evidence of use in Glue!-PS s Implementation of a subset of atomic patterns without architectural changes as e.g. s AP: Spontaneous use of additional IT s Actions: Add and synchronize in real-time resources in an activity s Feasibility evidence in 2 cases (Mar-Apr 2012) 36
    37. 37. Hat 2: Wrap-ups Technological tools to support orchestrations Patterns (that reflect good educational practices) guide a flexible design and deployment processs Glue!-PS allows to bridge the deployment gap in distributed learning environments s time-efficient and less error-prone s allows sustainable migration to new VLE and connection to various LD toolss More naturalistic evaluation is necessarys Important role of PD workshops for innovation and feedback to research 37
    38. 38. Hat 2 (cont.): More tools to …s Regulate through monitoring and the detection of critical aspects of the learning designs (GLUE!-CAS)s Find and select appropriate tools and include them in learning designs (SEEK)s Integrate third-party tools in learning designs directly through the VLE interface (GLUE!)s Design evaluation of CSCL case studies and generate multimedia reports (CSCL-EREM)s …. 38
    39. 39. Hats 1 and 2: wrap-ups Support bits and pieces but also the complete cycle of orchestration s flexibly and sustainablys Design for orchestration to show added value of TEL s innovative pedagogies using old and new technologiess Integrate conceptual and technological tools for s researchers, teachers and students 39
    40. 40. And many open questionss Do researchers need an orchestration theory and framework and should it be normative?s Is the proposed approach for pattern elicitation sustainable? (crowd-sourcing ….)s How can we support co-orchestration with students and integrate efficiently PLE?s How various (Web, 3D worlds, augmented physical) spaces can be orchestrated?s Is there a way to solve the ethical and security problems that are involved?s … 40
    41. 41. Some linkss Publications s http://www.gsic.uva.es/public.php?lang=en&list_pus GSIC/EMIC channel at YouTube: s http://www.youtube.com/user/gsicemic? feature=watchs WS on orchestration at ICLS2012 (July 3) s https://sites.google.com/site/iclsorchestration2012/ 41