Orchestrating CSCL, Research seminar, CIPLUniversity of Queensland, Brisbane, July 19, 20122

1. University of Queensland,
Centre for Innovation in Professional
Learning,
Lunchtime research seminar
Orchestrating Computer Supported
Collaborative Learning (CSCL)
Prof. Yannis Dimitriadis
GSIC/EMIC research group
University of Valladolid
July 19, 2012
Based on work by L.P. Prieto – S. Villagrá

2. TEL/CSCL ecosystem
2

3. Outline
s The two hats of this seminar
s Hat 1: Conceptual tools
s A framework on orchestration
s Multi-level patterns as mediating tools
s Hat 2: Technological tools for
s Learning design
s Flexible deployment
3

4. The two hats of this seminar
s Personal
s Engineering background + Interest in education
s Audience ?
s Researchers in TEL, e-learning designers
s TEL research
s sound pedagogy + efficient engineering
s Quest for sustainable innovation
s good research for understanding, abstraction and
evidence
s effective support to actors – teachers, learners or
even service and technology providers 4

5. The two hats of GSIC/EMIC
s “Dark and bright side” in a group since 1994
s Focus on CSCL: complex and innovative
s Support real-world actors and own action-research
s Inquiry cycle (design-enactment-evaluation) for
s teaching/learning and technological support
5

6. Hat 1: Conceptual tools
s Analyze and understand the field of orchestration in
complex TEL ecosystems
s The “5+3 aspects” framework for orchestration
s Collect and use knowledge in terms of good
practices during the CSCL cycle
s Multi-level patterns
6

7. Orchestration: Moving target
s Increasing presence in literature, but what is it
about? Is there a new focus?
s Efforts to define the field/concept/metaphor:
s Stellar NoE deliverable, CSCL 2011 and ICLS
2012 workshops, collective paper
s New proposal for “5+3” orchestration framework
and validation through expert panel 7

8. Focus of orchestration on …
s Acknowledge complex integrated ecosystems
for innovative TEL/CSCL environments
s Support teachers as orchestrators (and think of
how to minimize-distribute the orchestration load on
teachers, learners and systems)
s Build on pragmatism and minimalism as keys
for sustainability
s Explicit emphasis on how to flexibly manage
(learning time) enactment
s Design for adequate orchestrating and
orchestrable technologies (and conceptual tools)
8

9. The “5+3” framework
A diagramatic view
http://prezi.com/aa2vighak7hh/orchestration-in-tel-cscl-as-easy-as-53/
9

10. The “5+3” framework
A process view
10

11. Evaluating the framework
The setup
s Research questions on
s Completeness of common understanding
s Usefulness for researchers
s Two panels (March – May 2012)
s Pilot phase: 22 Spanish researchers of low-
medium expertise (2.58) and more technological
profile (16T-7E-1M)
s International panel: 24 researchers of medium-
high expertise (4,46), high research experience
in TEL (15,8 years), more balanced profile
(6T-11E-7M)
11

12. Completeness of framework
s Quantitative and qualitative data
s Non-normative and inclusive
s Need to clarify role of actors, technology, theory
s More than 4,5 (Lickert scale of 1-6) for all items
s Logical: 4,78 –Comprehensive: 4,52 –Relevant: 5,13
12

13. Usefulness of framework
s To be referenced in papers, checklist and didactic
tool for young researchers
s But need for normative advice and real-world
examples
s Will be used in future (4.43)
s Provided new insights (4.17)
13

14. Multi-level patterns
s But the “5+3” Orchestration framework is a
conceptual tool mainly for researchers
s Tools (conceptual or technological) for orchestration
should also support “teachers”
s Exploiting articulated (design) knowledge
s (Pedagogical) patterns as mediating tools
regarding various orchestration aspects
s Design and Management of learning flow,
Assessment, Adaptation …
s Related fields of
s Learning Design, Educational Modeling, Scripting
14

15. Collaborative Learning Flow
and assessment patterns
s Learning Flow
s Assessment
15

16. Aligning both sets of patterns
16

17. Usefulness of patterns
s Are these patterns useful for practitioners?
s Evidence for learning flow patterns for OER
repurposing to collaborative activities
s Appropriation in University (Higher Education –
HE) teachers (Professional Development - PD)
workshops
s But not always
s If not included in appropriate technological tools
s Too complex (abstract) for (Primary Education –
PE) practitioners
s Do not take into account enactment issues
17

18. The need for atomic patterns
s Two independent studies by GSIC and SRI using
GroupScribbles in Primary and Secondary Education
schools (2008-2011)
s Studying enactment and disciplined improvisation
s Showed that other small-scale, informal, contextual,
actionable patterns emerged
18

19. A learning design in action
in a Primary Education classroom
19

20. Using atomic patterns
in a HE PD workshop
20

21. Atomic patterns
as mediating conceptual tools
s And we need to “fill the gap”, “flesh the bones” with
appropriate mediating artifacts (conceptual tools)
21

22. Evaluation of atomic patterns
in PD workshops
s Four (2 in PE, 2 in HE) PD workshops (2010-2011)
s WS4 data quantitative data (scale 1 to 7):
s Enactment patterns useful (5,31)
s Feasibility (5,22) and nearness (6,06)
s 19 out of 22 teachers enriched design
s Useful but not revolutionary
s Especially important for non-experienced or pre-
service teachers
s Need to condense and classify long list of AP
s What about inclusion in technological tools?
22

23. Approach for multi-level patterns
s And the way to elicit and use multi-level patterns
23

24. Hat 1: Wrap-up
s Knowledge regarding orchestration for researchers:
s “3+5 Aspects” framework
s Approach for multi-level patterns
s Atomic pattern catalogue and LD representation
s Support teachers too with conceptual tools:
s Classroom norms for participation, discourse
moves for discussion, decision rules for
contingent teaching (SRI)
s Multi-level patterns
s Use in PD workshops Role playing and learning
design enrichment using cards (GSIC) 24

25. Hat 2: Technological tools
s But TEL is not only about
s Conceptual tools (patterns, moves, …)
s It is also about
s enacting learning in TEL complex ecosystems
s especially in HE distributed learning
environments where VLE/PLE are combined with
third-party tools
s And also support teachers (and learners) with
technological tools when
s orchestrating TEL/CSCL ecosystems (less
orchestration load for teachers – more for tools)
25

26. Learning Design tools
s Articulate/exploit the knowledge of patterns and
support the learning design process
s Create a model or representation of a learning
design (unit of learning, script, scenario, etc.)
s That allows understanding and reflecting on the
pedagogy, making the learning design explicit,
sharing it with others
s Through the Web Collage tool
s Or other Learning tools by the Stellar Learning
Design Grid Theme Team
http://www.ld-grid.org/home
26

27. Web Collage:
Learning & Assessment flows
Students Teacher
Select and work
Support individual
individually in a
work
subproblem
Write activity input for… Review
report report
Work in expert
Work expert
groups of students
groups students provide Support expert
with the same
with same feedback… group work
subproblem
subproblem
Form Jigsaw groups
and explain each Support jigsaw
other the different group work
subproblems
27
http://pandora.tel.uva.es/wic2

28. A learning design
(HE teacher in a PD workshop)
28

29. Web Collage:
Evolution and evaluation
s Evolution of the Collage tool since 2005
s Towards a more stable and user-friendly Web-
based tool that includes learning and assessment
patterns, as well as support for instantiation
s Extensive use and evaluation using mixed methods
s Multi-case studies in authentic settings and PD
workshops with HE teachers
s Wide acceptance of the pattern-based design
process
29

30. The deployment gap
s But teachers wanted to complete the cycle
s deploy their design in their TEL environment,
especially their institutional VLE (e.g. Moodle)
30

31. Alternatives for deployment
s Have the learning design “in mind” and then go
through the VLE/PLE interface and struggle to
“translate” the design to a course/lesson plan
s Use a single system (LAMS) to do both design and
deployment in the same environment
s Use an LD Tool (LDSE) to create the interpretable
script and then deploy it to one environment
s But there are many LD tools and VLEs:
s Provide intermediary to connect them: Glue!-PS
31

32. Glue!-PS
An architectural view
32

33. Glue!-PS
An architectural view - II
33

34. Glue!-PS:
A functional view
34

35. Glue!-PS
Validation
s Proof of concept validations using the Planet Game
(ICALT 2006) scenario
s Two LD languages: LDL + IMS-LD
s Two Learning Environments: Moodle +
MediaWiki
s Design and deployment
s by GSIC researchers of 36 learning designs
created using pen and paper at two HE PD
workshops (Sep-Nov 2011)
s by 24 teachers in a follow-up PD workshop (Feb
2012)
35

36. Glue!-PS and atomic patterns
s Atomic patterns are especially important for flexible
orchestration
s University teachers suggested embedding atomic
patterns in technological tools
s Initial evidence of use in Glue!-PS
s Implementation of a subset of atomic patterns
without architectural changes as e.g.
s AP: Spontaneous use of additional IT
s Actions: Add and synchronize in real-time
resources in an activity
s Feasibility evidence in 2 cases (Mar-Apr 2012)
36

37. Hat 2: Wrap-up
s Technological tools to support orchestration
s Patterns (that reflect good educational practices)
guide a flexible design and deployment process
s Glue!-PS allows to bridge the deployment gap in
distributed learning environments
s time-efficient and less error-prone
s allows sustainable migration to new VLE and
connection to various LD tools
s More naturalistic evaluation is necessary
s Important role of PD workshops for innovation and
feedback to research 37

38. Hat 2 (cont.): More tools to …
s Regulate through monitoring and the detection of
critical aspects of the learning designs (GLUE!-CAS)
s Find and select appropriate tools and include them
in learning designs (SEEK)
s Integrate third-party tools in learning designs
directly through the VLE interface (GLUE!)
s Design evaluation of CSCL case studies and
generate multimedia reports (CSCL-EREM)
s ….
38

39. Hats 1 and 2: wrap-up
s Support bits and pieces but also the complete cycle
of orchestration
s flexibly and sustainably
s Design for orchestration to show added value of
TEL
s innovative pedagogies using old and new
technologies
s Integrate conceptual and technological tools for
s researchers, teachers and students
39

40. And many open questions
s Do researchers need an orchestration theory and
framework and should it be normative?
s Is the proposed approach for pattern elicitation
sustainable? (crowd-sourcing ….)
s How can we support co-orchestration with students
and integrate efficiently PLE?
s How various (Web, 3D worlds, augmented physical)
spaces can be orchestrated?
s Is there a way to solve the ethical and security
problems that are involved?
s …
40

41. Some links
s Publications
s http://www.gsic.uva.es/public.php?lang=en&list_pu
s GSIC/EMIC channel at YouTube:
s http://www.youtube.com/user/gsicemic?
feature=watch
s WS on orchestration at ICLS2012 (July 3)
s https://sites.google.com/site/iclsorchestration2012/
41