Metis project deliverable D3.2: Draft of pilot workshop

Project Number: 531262-LLP-2012-ES-KA3-KA3MP
METIS - Meeting teachers co-design needs by means of
Integrated Learning Environments
D3.2: Draft of pilot workshop
WP3: Workshop Design
WPLeader: OU
Author(s)/Editor(s): Andrew Brasher,
Chris Walsh, Patrick McAndew, Yishay Mor
(OU)

Project information
Project acronym: METIS
Project title: Meeting teachers' co-design needs by means of
Integrated Learning Environments
Project number: 531262-LLP-1-2012-1-ES-KA3-KA3MP
Sub-programme or KA: KA3 Multilateral projects
Project website: http://www.metis-project.org
Reporting period: From 31/1/13
To 31/5/13
Report version: 0.1
Date of preparation: 10/5/13
Beneficiary organisation: University of Valladolid (UVa), Spain
Project coordinator: Prof. Yannis Dimitriadis
Project coordinator organisation: University of Valladolid (UVa), Spain
Project coordinator telephone number: +34 983 423696
Project coordinator email address: info@metis-project.org
WP Leader: Yishay Mor (OU)
WP Leader email address: Yishay.Mor@open.ac.uk
Document history
Date Version Author(s) Description
10/5/13 01 Andrew Brasher, Chris
Walsh, Patrick
McAndrew, Yishay Mor
Final draft for formal internal review
16/5/13 0.1.1 Gráinne Conole Review
20/5/13 0.1.2 Michael Derntl Review
29/5/13 02 Andrew Brasher Edited in response to reviewers’
comments.
6/6/13 03 Andrew Brasher, Yishay
Mor
Final version
This work is licensed under a Creative Commons Attribution-NonCommercial-
ShareAlike 3.0 Unported License.
This project has been funded with support from the European Commission. This
publication reflects the views only of the author(s), and the Commission cannot be
held responsible for any use which may be made of the information contained
therein.

Executive Summary
Several decades of research in Technology-Enhanced Learning (TEL) have clearly
demonstrated the potential of digital technology to transform education. Yet the
impact of TEL research on daily teaching-learning practices is still far from fulfilling
this potential (Mellar, Oliver, & Hadjithoma-Garstka, 2009). Teachers lack the
necessary digital literacy skills (Jenkins, 2009) to harness the potential of new
technologies. Arguably, this is a gap in the capacity for learning design1
: educators
need the tools and competencies which would allow them to identify educational
challenges, describe the context in which they arise, identify the opportunities
afforded by technology, project the insights derived from research, and devise new
learning experiences. To address this gap, educators need tools and practices. Tools
that would support them through the cycle of learning design – from conception to
deployment and evaluation of techno-educational innovations. Professional
practices that use such tools to ensure the robustness and effectiveness of their
innovations and make learning design a daily habit and part of their professional
identity. The METIS project (http://metis-project.org/) aims to contribute to this aim,
by providing educators with an Integrated Learning Design Environment (ILDE) and a
workshop package for training educators in using the ILDE to support effective
learning design.
Work Package 3, led by the OU (UK), is concerned with the design and development
of the workshop package.
Deliverable D3.2 represents the analysis of best practices and workshop design from
the first cycle of the METIS project methodology. Alongside this report a prototype is
provided to allow access to the package of resources representing a workshop
structure developed from the preliminary analysis of best practices in teacher
training reported in Deliverable D3.1. (This prototype is included in appendix 1).
Section 2 provides an account of the review of best practices – the methodology
used for this review, its current status and outcomes, and plans for the future.
Section 3 provides an account of the design of the METIS workshop – the process,
current status and outcomes, and plans for the future. It also lists risks and
challenges and implications to and from WP 2 and 4.
The METIS partners responsible for designing the workshops have extensive
experience of running and evaluating learning design workshops. This includes the
1
. Typically “Learning Design” (with capital letters) is used to refer to specific tools or projects e.g. ‘
IMS Learning Design’, and the ‘Integrated Learning Design Environment’. The phrase “learning design”
(all lower case) is used to refer to general practices and instantiations e.g. ‘enactment of innovative
learning designs’ .

work undertaken by the OULDI project at the OU, the Carpe Diem workshops run by
the University of Leicester, the 7Cs of Learning Design framework (developed by
University of Leicester), the recent OLDS MOOC (led by the OU) and the JISC-funded
SPEED project.
The workshop structure presented here is a “meta-design”, which needs to be
customized and specified for each user group. The structure provides a flexible basis
for developing the ready-to-run workshops, and is the input for subsequent tasks in
WP3/WP4 in the second cycle. The partner methodologies (provided in Appendices)
show some of the activities envisioned that can be placed within the structure to
develop the different workshops.
This report consists of an account of the workshop design process and critical
decisions, an overview of the design principles and outline of the workshop
structure, and a review of the pedagogical framework and best practices informing
the design. The current version of the design prototype, as well as some of the best
practices data, are includes as appendices.

METIS PROJECT
D3.2: Draft of pilot workshop Page 5 of 64
Contents
1 Introduction .............................................................................................................................7
2 Review of best practices ..........................................................................................................9
2.1 Method..............................................................................................................................9
2.2 Results .............................................................................................................................11
3 Outline of draft design of the METIS Workshops ..................................................................16
4 Next steps and implications for other work packages...........................................................17
4.1 Risks.................................................................................................................................17
4.2 Implications for and from WP 2 ......................................................................................18
5 Conclusions ............................................................................................................................19
6 Appendices.............................................................................................................................20
7 References..............................................................................................................................20
Appendix 1. Workshop design..................................................................................................22
Appendix 2. Methodology descriptions ...................................................................................34
a. Learning Design Workshop Methodologies from the ICOPER Best Practice Network..........34
b. 7Cs of learning design framework .........................................................................................35
c. Design-Practice.......................................................................................................................37
d. Learning Design Studio...........................................................................................................39
e. Collage / LdShake workshops.................................................................................................41
f. Participatory Pattern Methodology.......................................................................................42
g. Design Challenge....................................................................................................................44
h. OULDI Learning Design Training Module...............................................................................45
Appendix 3. Methodology design narratives and design patterns ..........................................48
a. “Design Narratives” task presented to partners....................................................................48
b. Design narrative template .....................................................................................................49
Appendix 4. Sample of Methodology Design Narratives..........................................................51
a. Meta-Pyramid design.............................................................................................................51
b. Role-playing on problematic situations and technology .......................................................56
c. Mini-focus-groups..................................................................................................................59
Appendix 5. Personas template................................................................................................63
Appendix 6. Survey of user groups...........................................................................................64

METIS PROJECT

METIS PROJECT
1 Introduction
The last few decades have seen tremendous progress in the use of ICT for education and
training across Europe. However, effective integration of ICT should go beyond replacing,
streamlining or accelerating current practices. It must also support “pedagogical and
organisational innovation” (EACEA, 2009). The current gap between research and practice in
Technology-Enhanced Learning (TEL) highlights the need for a shift in attention from the
development of specific tools and resources to the support for their integration, e.g. in teacher
practice for the design of ICT-based learning activities. This integration into practice needs
support for the whole design and implementation life-cycle, from (co-)design to enactment
(Kelly, Lesh, & Baek, 2008; Laurillard, 2008).
The METIS project (http://metis-project.org/) aims to provide this kind of support, synthesising
the achievements of the design paradigm and making them available to a broad circle of
practitioners across multiple educational sectors such as adult education, vocational training
and higher education. This is to be achieved mainly through a practitioner-centred approach,
which combines a) technological support for the whole learning design life-cycle (in the Learning
Design Environment, or ILDE); b) professional development support in the form of ready-to-use
workshop packages; and c) the dissemination of these project outcomes to a wide community
of practitioners.
Work Package 3 of the METIS project is concerned with the design of the professional
development workshops. This work package is led by the OU (UK), in collaboration with other
partners. We will work closely with the projects’ user groups to ensure that the workshop
design addresses their constraints and concerns, and with the evaluation partners to ensure
that the design is robust and effective. The workshop design will be evaluated by the user group
partners (task T3.4), and eventually offered as an open educational resource on the project
website (as part of deliverable D3.4, task T3.9).
Deliverable D3.2 represents the analysis of best practices and workshop design from the first
cycle of the METIS project methodology. Alongside this report a prototype is provided to allow
access to the package of resources representing a workshop structure developed from the
preliminary analysis of best practices in teacher training reported in Deliverable D3.1.
The workshop structure presented here is a “meta-design”, which needs to be customized and
specified for each user group. The structure provides a flexible basis for developing the ready-
to-run workshops, and is the input for subsequent tasks in WP3/WP4 in the second cycle. The
partner methodologies (provided in Appendices) show some of the activities envisioned that
can be placed within the structure to develop the different workshops.
This report consists of an account of the best practices review process and its outcomes (section
2), an overview of the design principles (section 2.2.1), an outline of the workshop structure and
its rationale (section 3), and a discussion of the next steps in the process (section 4) and the
perceived risks (section 4.1) and implications (section 4.2). The current version of the design

METIS PROJECT
prototype, as well as some of the best practices data, are includes as appendices (indexed in
section 6).

METIS PROJECT
2 Review of best practices
2.1Method
D3.1 (Brasher & Mor, 2013) identified a set of workshop aims for participants, a selection of
successful approaches and a set of design principles. These were derived from a survey of
learning design experts, a series of 4 semi-structured interviews of representatives of user
groups and a workshop held on 9th
January 2013 in which representatives of user groups and
other project partners were guided in describing their context and characterising their
prospective workshop participants. These aims, approaches and principles have guided the
development of the draft design of METIS workshops below. In parallel to working on the
design, we have continued the process of reviewing past and current practices, which informs
our design work and will continue to feed into it as we proceed into greater detail.
Our aim is to capture these best practices in the form of transferable design knowledge, which
can then be applied to the core generic workshop design, used in the process of customising this
design to the needs of the user groups, and could also be shared with anyone who wishes to
design their own learning design workshops.
To achieve this aim, we choose to represent this design knowledge using a combination of
overview descriptions, design narratives, design principles and design patterns. This approach is
based on the SNaP! Methodology (Mor, 2013)2
and the PPW methodology (Mor, Warburton and
Winters, 2012)3
.
A design narrative is an “account of critical events in a design experiment from a personal,
phenomenographic perspective”4
. A design pattern “describes a recurring problem, or design
challenge, the characteristics of the context in which it occurs, and a possible method of
solution”5
. A design principle is “…an intermediate step between scientific findings, which must
be generalized and replicable, and local experiences or examples that come up in practice.” (Bell
et al, 2004, p. 83, in Kali, 2009)6
. Together with the overview descriptions, these cover a full arc
from the pedagogical framework, through the high-level design, and down to specific activities.
The design principles will be declared upfront in our workshop design documentation and serve
as a pedagogical contract between the workshop designers, facilitators and participants:
defining a set of mutual expectations regarding the roles and interactions between them. We
2
Mor, Yishay (2013). SNaP! Re-using, sharing and communicating designs and design knowledge using scenarios, narratives and
patterns. In: Luckin, Rosemary; Puntambekar, Sadhana; Goodyear, Peter; Grabowski, Barbara L.; Underwood,
Joshua and Winters, Niall eds. Handbook of Design in Educational Technology. London, UK: Routledge, (In press).
3
Mor, Yishay; Warburton, Steven and Winters, Niall (2012). Participatory pattern workshops: a methodology for open
learning design inquiry. Research in Learning Technology, 20
4
http://www.ld-grid.org/resources/representations-and-languages/design-narratives
5
http://www.ld-grid.org/resources/representations-and-languages/design-patterns
6
In: http://www.ld-grid.org/resources/representations-and-languages/design-principles

METIS PROJECT
will review the general flow of the workshop as well as the individual activities to ensure that
they are compliant with these principles.
The design patterns will be used to mold specific activities, or even elements of activities. They
will also be offered as a resource for workshop facilitators as an aid in customizing the workshop
design, and as a resource for participants as an illustration of an exemplar learning design
process.
The end product of this process with be deliverable D3.4 “Final workshops packages: workshops
for different educational levels and education contexts”. This will contain instructions for the
trainers as to how to run the workshop; a sequence of activities for the trainer and the trainees;
a description of the rationale and pedagogical methodology on which the workshop is based;
and attached learning resources to be used in the workshop.
The review process consists of the following phases:
Phase 1. Collating overviews of the workshop methodologies, with example resources and
links and references to detailed descriptions (completed, see Appendix 2)
Phase 2. Collation of design narrative of successful workshop practices (completed, see
Appendix 3)
Phase 3. Identifying design principles in the design principles database and in the overviews
(completed, see section Results’).
Phase 4. Extraction of initial design patterns from the design narratives, to be achieved by
analysis and cross-comparison of narratives.
Phase 5. Refinement, elaboration and substantiation of the design patterns and design
principles
Phase 6. Identifying the implications of the design patterns and principles for WP 2 (ILDE),
WP 4 (enactment) and WP 5 (evaluation).
Phase 7. Applying the patterns and principles to the workshop design.
Originally, it was planned to complete this process before elaborating the workshop design.
However, our discussions with WP2 and WP5 suggested that they need an early draft of the
design in order to align their work with it. Furthermore, it has become evident that the user
groups need to be engaged early on with the general framework of the workshop design to
verify that they can commit to its structure. Consequently, we are proceeding with the
methodology review and the workshop design in parallel rather than sequentially.
Initial results from phases 1 to 4 of the review process are presented in section Results’. The
next steps required to support this process through phases 5-7 are:
Step 1. WP3: Provide guidance to partners to embark on a more robust and critical of the
new design patterns using the additional guidance provided (completed June 4,
2013);

METIS PROJECT
Step 2. ALL: Elicit/evoke additional and refined design narratives;
Step 3. All: Elicit/evoke additional and more refined design patterns into the ILDE by
refining/building on the design patterns that have already been generated. We will
do this by providing high quality and relevant examples from workshops conducted
by the OU to work as a catalyst to evoke the additional examples;
Step 4. All: Migrate the rest of the original design narratives into the ILDE;
Step 5. WP3: Author a design principles document drawing on the work already completed
and the expected work to be completed in 1-5 above;
Step 6. WP3 and User groups: Use the design principles, patterns and narratives to inform
future versions of the generic as well as the user-group specific workshop designs;
Step 7. WP3 and WP2: review the patterns and principles, and consider their implications
for the ILDE design and implementation;
Step 8. OU and WP4: review the patterns and principles, and consider their implications for
the workshop enactment (WP4);
Step 9. OU and WP5: review the patterns and principles, and consider their implications for
the Metis evaluation strategy;
2.2 Results
Appendix 2 lists the methodology descriptions. Appendix 3 presents the procedure for collating
the design narratives (including our design narrative template). Appendix 4 offers a sample of
design narratives of successful workshop activities. In section 2.2.1 we describe the design
principles which have been extracted so far.
We initiated the process using the project internal collaborative work space
(http://internal.metis-project.org/workpackages/wp3-workshop-design/methodology-cross-
review/). We collected 12 design narratives there (
Figure 1).

METIS PROJECT
Figure 1: screen shot of design narrative repository on the project internal work space.
In parallel, ILDE was launched for internal beta, and we decided to shift our activity there. We
began migrating the design narratives to ILDE, and in parallel began extracting patterns in ILDE.
Currently, we have 10 design narratives in ILDE (Figure 2).
Figure 2: Design narratives in ILDE
So far, we have identified 6 design patterns (Figure 3)

METIS PROJECT
Figure 3: Design patterns in ILDE
Figure 4 shows an example of a design pattern in ILDE. This pattern is not yet available for public
viewing. Another pattern, which has been published and is available for public viewing, can be
found at: http://ilde.upf.edu/v/bjd

METIS PROJECT
Figure 4: example design pattern in ILDE
2.2.1 Design principles
Table shows a first set of design principles extracted during phase 3. This table shows the source
of each principle, the principle itself, and some remarks on how it has or could be applied in the
workshop design presented in Appendix 1.
Source Principle Application
Principles from
the Educational
principles database
that UVA have
found useful in
their workshops
and can be applied
to the METIS
workshops
1. Build on student ideas
(http://www.edu-design-
principles.org/dp/viewPrincipleDeta
il.php?prKey=166)
2. Reuse student artifacts as resource
for learning (http://www.edu-
design-
il.php?prKey=371)
3. Integrate online with offline
activities (http://www.edu-design-
 METIS workshops will be on a
specific theme relevant to
participants’ context, and will
focus on participants own
design problems.
 During the workshop
participants will be encourage
to share resources produced
during a workshop both
through the ILDE and face-to-
face. This principle is
embodied in activity 9

METIS PROJECT
il.php?prKey=330)
4. Connect to personally relevant
contexts (http://www.edu-design-
il.php?prKey=171)
‘Evaluate’ in which participants
will evaluate and learn from
others’ designs.
 The workshop design includes
activities which use both
online and offline tools.
design problems.
Additional
principles from
the Educational
principles
database that UPF
have found useful
in their workshops
and can be applied
to the METIS
workshops
5. Encourage learners to learn from
others (http://www.edu-design-
il.php?prKey=224)
6. Employ multiple social activity
structures (http://www.edu-design-
il.php?prKey=238)
7.
 This principle is used explicitly
in activities 9, 10, and underlies
all the other activities as they
are collaborative in nature.
 The workshop design includes
multiple social activity
structures. The emphasis is on
working in small groups, with
individual activity and whole
class activity occurring
occasionally (e.g. activity 2,
activity 4).
Principles from
reviewing the LDS
methodology:
8. Continued work on a
challenge/design project: "the main
activity of a course is the students'
continued work on design
challenges in a defined domain of
practice" (akin to project-based
learning)
9. Public review of group artifacts:
"classroom sessions are mostly
dedicated to group work and public
review of design artefacts"
10. Iterate!: implicit in "continued work
on design challenges in a defined
domain of practice"
design problems.
 This occurs in activities 4 and 9.
 Iteration is implicit in the
workshop design (see figure 2),
but time limitations may limit
the iteration that is possible
during the workshop itself.
Some principles
from CARDET's
Design-Practice
methodology:
11. Include an early "how to ruin X"
activity to get people started
thinking about the topic X
12. Break down a (part of a) learning
design to a full detail level -- this is
an important activity to get
teachers started thinking in detail
about design decisions and
implications of those decisions (see
also this paper)
13. Pitching results -- Important to let
people pitch their results and have
peers discuss those. In the CARDET
 Included as activity 2.
 Creating a fully detailed design
occurs in activity 8 ‘Prototype’.
 This occurs in activity 9,
‘Evaluate’.

METIS PROJECT
Table 1: design principles, their source, and comments on their application to the workshop design
3 Outline of draft design of the METIS Workshops
Based on the aims, approaches and principles outlined in D3.1 (section 3, Brasher & Mor, 2013,
pp. 8-11), we have proceeded to iteratively develop the design for the METIS workshops. Early
sketches of this design have been presented to partners and user groups for their review and
feedback. The current state of the draft design is presented in Appendix 1.
This is a generic, or “template” design, in the sense that it needs to be customized for each of
the user groups to ensure it meets local needs and contexts. The process of customization will
involve further discussions and negotiations with each user group so as implement context
specific versions of the activities described in the template during WP3’s work towards D3.3 (the
three ready to run workshop structures). The design decisions taken in this process will be
recorded by WP3 and used to produce guidance for customisations for application of the
template to other contexts. However, it sets a common framework which we see as necessary
for an effective workshop:
A METIS workshop consists of a sequence of collaborative activities, spanning 6-8 hours. These
can be conducted as one full day event or two half day events. The workshop requires a space
which is set up for group work. It is optimized for 4-5 groups of 3-6 participants each, overall 15-
25 participants. Ideally, it would need a facilitator per every 5 participants. To be effective, a
workshop will require participants to invest 1-2 hours pre-event and 1-2 hours post event.
methodology it sounds
asynchronous, but also very
relevant in a synchronous/f2f
setting.
Some principles
from ICOPER
methodologies:
14. Solving a proposed task (e.g.,
providing a narrative example to
solve) -- helpful when training
specific skills associated to
improving proficiency around the
elements of a design modelling
language / tool.
15. Improving participants' previous
designs -- participants
reconceptualise their actual designs
(courses) using theoretical input
provided gradually.
 Examples are provided in
activity 5 ‘Evidence and
examples of ‘X’’, and also
through patterns in activity 7.
However, decisions need to be
taken about how these
patterns and examples are
instantiated and present to the
different user groups.
 Iteration is implicit in the
workshop design (see figure 2),
but time limitations may limit
the iteration that is possible
during the workshop itself.
Theoretical input is provided
gradually (e.g. activities 5 and
7).

METIS PROJECT
The current draft of the workshop design is presented as a word document (Appendix 1) and a
CompendiumLD7
archive. This format is easy to share, manipulate and edit.
4 Next steps and implications for other work packages
The next phases of the WP 3 design and development process will be:
1. Discussing this draft with partners and user groups, and reviewing their feedback
(This work is part of part of Task T3.4, face to face discussion to occur at the METIS
meeting in Barcelona, July 2013; preliminary online discussion already underway).
2. Work with tool providers for CADMOS, OpenGLM, and WebCollage on integrating
detailed and meaningful activities using these tools into the current workshop structure.
3. Specifically, consider ways of incorporating the collaborative learning flow patterns
(Hernández-Leo, Asensio-Pérez, Dimitriadis, & Villasclaras, 2010) both into the workshop
design and as meaningful resources for workshop participants.
4. Incorporate the outcomes of the “best practices” cross-review process, described in
section 2.
5. Based on the feedback and the “best practices” review, deploying a revised design to a
shared Virtual Learning Environment (VLE). (This work is part of Task T3.5, to occur in
project months 8 – 10, June –August 2013).
6. Run an open survey of potential participants to assess their interest in the topics and
structure which emerged so far (an initial survey of the 3 user partners representatives
within METS highlighted the topics of formative assessment, collaborative learning and
project based learning). (This work is part of Task T3.4, the survey will be launched in
June 2013, project month 8).
7. Work with user groups to customize this design per their specific needs, concern and
constraints (coordinated by WP4). The workshop design draft may be modified in
response to the requirements for customisations, so that it remains a generic template
for the customisations that are produced. The customised instances of this meta design
will be delivered as METIS Deliverable D3.3 (month 11). (This work is part of Task T3.5, to
occur in project months 8 – 10, June –August 2013).
8. Work with WP5 on evaluation procedures for the design. On-going communication with
and reporting to WP5 about the evolution of the workshop design.
4.1Risks
In the course of the work on the workshop design we have identified several risks which need to
be mitigated. We are collaborating with WP 4 (‘Report on the pilot workshops and LD
enactment’) on addressing these:
7
CompendiumLD is a a software tool for designing learning activities using a flexible visual interface that can be
downloaded from http://compendiumld.open.ac.uk/. This site also provides a ‘getting started’ guide for new users.

METIS PROJECT
1. User groups need to commit to one full day / two half day workshops. Initial
discussions with user groups suggest this may be difficult for some of them. However,
we are convinced that it would be ineffective to run shorter workshops.
2. User groups need to recruit sufficient participants and facilitators for each workshop.
In order to properly evaluate and revise the design between the two cycles, we will need
3 workshops (1 per user group) of 10-20 participants in the first and second rounds.
These workshops will require 2-3 facilitators each. In addition, we note that after the
workshops least one participant from each workshop must use the ILDE in their own
teaching practice, applying techniques they have learnt through the workshops.
3. Input from user groups required to customize and add content to their instance of the
workshop design. Some of the activities are common, some are specified in a generic
manner, and need to be elaborated per the specific topics, context and concerns of
every workshop. Furthermore, user groups may want to translate the workshops
materials to their local language.
4. User groups need to accommodate diverse technical skills. The workshop design
assumes a basic level of computer and internet proficiency. The workshops are
supported by a VLE, the ILDE and a variety of computational tools. Some of the
participants may not have the required skills, and the user groups will need to
accommodate them by prior training or by grouping them with other participants who
can assist them.
4.2 Implications for and from WP 2
Table 10 in D2.1 (Hernández-Leo, Asensio, Chacón, & Prieto, 2013) describes the WP2 revised
use cases and requirements stressed by the user groups. These revised use cases are listed
below, along with a comment indicating how the workshop structure reported herein relates to
the use case. We also make a note of any specific requirements for WP2 to bear in mind. Some
of the activities require specific resources to be provided in the ILDE. We will coordinate these
with WP2 after the next round of reviews and edits.
(1) Choose a learning design tool between several available. The skill to select an
appropriate design tool will be developed through several of the activities that occur
within the workshop structure. For example in Activity 7 participants chose one or more
design patterns appropriate to the requirements of their design vision. In this activity
they are also introduced to the ILDE tools with the appropriate affordances to produce
runnable designs from these patterns.
(2) Produce a learning design by both adapting reused designs and starting from scratch.
In the workshop structure we have described, we envisage that workshop participants
will produce many design artefacts during the design process, prior to and during the
creation of the runnable design. These artefacts will be sketches, notes, evaluation
checklists and so on.

METIS PROJECT
(3) Co-produce a learning design. The workshop structure is built around team work and
coproduction.
(4) Share a learning design. The ILDE tools facilities for sharing will be used in Activity 9.
Designs produced during the workshop will be shared amongst all the teams
participating in the workshop and participants will be would encouraged to share them
more widely afterwards.
(5) Instantiate a learning design. Instantiating a learning design occurs during Activity 8
‘Prototype’.
(6) Deploy an instantiated learning design into chosen VLEs. Deploying a learning design
occurs during Activity 8 ‘Prototype’.
(7) Provide feedback and reflections. In Activity 9 participants evaluate each other’s designs
using the checklists developed during earlier activities.
(8) Explore designs, instantiations and feedback. This is the focus of Activities 9 and 10 in
the workshop structure.
5 Conclusions
We have completed the first cycle of the workshop design and development. This deliverable is
the end product of that cycle, and consists of package of resources representing the workshop
structure developed from the best practices identified in Deliverable D3.1 (Brasher & Mor,
2013). This package includes: a sequence of activities for the trainer and the trainees; a
description of the rationale and pedagogical methodology on which the workshop is based; and
attached learning resources to be used in the workshop.
This package provides a meta-design for the Metis workshops. Over the next few months we
will use this as the basis for developing the detailed designs of the actual workshops to be run
by and for the user groups.
In the course of our work towards this deliverable, we identified a need to deepen and broaden
our review of best practices. Due to the high level of interdependencies between the workshop
design and other work packages, we decided to prioritise progress on the workshop design, and
continue the review of best practices in parallel.
Also, during the work on the design and review of best practices, the ILDE had moved into
private beta phase. Once we verified that it is stable and the basic functionality is in place, we
decided to shift our work from the project’s internal workspace to the ILDE. This entailed a
transition cost, but will allow us to develop our expertise in using ILDE, and will provide WP2
with “real world” testing and evaluation.
Our next goals would be to synchronise the workshop design with the ILDE development, and
use the generic design here to develop the structures for the three pilot workshops (D3.3). We
will continue our review of best practices, and these will feed into the design process
continuously.

METIS PROJECT
6 Appendices
Appendix 1: Workshop
designAppendix 1
The sequence of activities, rationale and pedagogical
framework, and associated resources. This will form the basis
for the structure of the three pilot workshops.
Appendix 2: Methodology
descriptions
High-level descriptions of the workshop methodologies
considered in the review of best practices.
Appendix 3: Methodology design
narratives and design patterns
Process and template used for collecting the methodology
design narratives.
Appendix 4: Sample of
Methodology Design Narratives
Example of design narratives collected in the review of best
practices.
Appendix 5: Personas template An example resource used in a workshop activity. Also used by
us as an aid in designing the workshops.
Appendix 6: Survey of user
groups
Summary of data collected to ensure the workshops address
user concerns and constraints.
7 References
Brasher, A., & Mor, Y. (2013). METIS deliverble D3.1: Report 2 on meetings with user groups:
Early feedback on candidate best practices for teacher training on learning design.
Cross, S., Galley, R., Brasher, A., & Weller, M. (2012). OULDI-JISC Project Evaluation Report
Retrieved 3/8/2012, from http://oro.open.ac.uk/34140/1/OULDI_Evaluation_Report_Final.pdf
EACEA. (2009). Lifelong Learning Programme Key Action 3: Information and Communication
Technologies (ICT) Retrieved 22/5/2013, from
http://eacea.ec.europa.eu/llp/ka3/information_communication_technologies_en.php
Galley, R. (2010). Activity: 30 mins: Learning Outcomes view Retrieved 9/5/2013, from
http://cloudworks.ac.uk/cloud/view/4036
Hernández-Leo, D.; Asensio-Pérez, J.I.; Dimitriadis, Y.; & Villasclaras, E.D. Generating CSCL
Scripts: From a Conceptual Model of PAttern Languages to the Design of Real Scripts. In:
Goodyear P.; Retalis, S. (eds.). Technology-Enhanced Learning, Design patterns and pattern
languages, Sense Publishers, Series Technology-Enhanced Learning; 2010. p. 49-64. Appendix
available at http://ulises.tel.uva.es/%7Edherleo/dpbook/appendix-chapter.pdf
Hernández-Leo, D., Asensio, J. I., Chacón, J., & Prieto, L. P. (2013). METIS deliverble D2.1: Report
1 on meeting with stakeholders: early feedback on ILDE requirements.
Jenkins, H. (2009). Confronting the Challenges of Participatory Culture: Media Education for the
21st Century The John D. and Catherine T. MacArthur Foundation Reports on Digital Media and
Learning (pp. 146). Retrieved from
http://mitpress.mit.edu/sites/default/files/titles/free_download/9780262513623_Confronting_
the_Challenges.pdf

METIS PROJECT
Kelly, A. E., Lesh, R. A., & Baek, J. Y. (2008). Handbook of Design Research Methods in Education.
New York: Routledge.
Laurillard, D. (2008). The teacher as action researcher: using technology to capture pedagogic
form. Studies in Higher Education, 33(2), 139-154. doi:
http://dx.doi.org/10.1080/03075070801915908
Mellar, H., Oliver, M., & Hadjithoma-Garstka, C. (2009). The role of research in institutional
transformation'Transforming Higher Education through Technology-Enhanced Learning. York,
UK: Higher Education Academy. Retrieved from
http://www.heacademy.ac.uk/assets/York/documents/ourwork/learningandtech/Transforming.
pdf.

METIS PROJECT
Appendix 1. Workshop design
METIS Workshop Design (version 0.7)
Summary
Despite the abundance of high-quality educational technology, and the wealth of research
demonstrating its value, educators are still struggling to make effective use of these
technologies and the associated pedagogies in their daily practices. Arguably, this is a gap in the
capacity for learning design: educators need the tools and competencies which would allow
them to identify educational challenges, describe the context in which they arise, identify the
opportunities afforded by technology, project the insights derived from research, and devise
new learning experiences. To address this gap, educators need tools and practices. Tools that
would support them through the cycle of learning design – from conception to deployment and
evaluation of techno-educational innovations. Professional practices that use such tools to
ensure the robustness and effectiveness of their innovations and make learning design a daily
habit and part of their professional identity. The METIS project (http://metis-project.org/) aims
to contribute to this aim, by providing educators with an Integrated Learning Design
Environment (ILDE) and a workshop package for training educators in using the ILDE to support
effective learning design.
This document describes the aims, organisational requirements and activity structure for METIS
learning design workshops. The aims are specified in terms of outcomes for participants and the
METIS project. The organisational requirements describe the human and other resources
required to run a workshop. The activity structure describes a reusable structure of activities to
enable participants to reach the learning outcomes specified. It includes a description of the
tools and the information resources for both participants and facilitators for each activity.
Workshop aims and organisation
1 Learning and other outcomes
Our research of user concerns suggests that practitioners have limited interest in training on
learning design in general, but are have much higher interest in learning design for specific
themes, such as collaborative learning, formative assessment or project-based learning (Brasher
& Mor, 2013). Hence, the workshop design presented here is a meta-design that is flexible and
can be customised to a specific theme. In this document, this theme is noted as ‘X’.
The intended learning and other outcomes for a METIS workshop on learning design for ‘X’ are
shown in figure 1. The learning design is intended to be applicable across a range of topics.

METIS PROJECT
Figure 5: Learning and other outcomes of the workshop
2 Workshop organisation
2.1 Duration
The total duration of the activities in the workshop(s) is 7.5 hours (Table 2). The activities need
to run in the order they are presented in the ‘Workshop activity structure’ section, but they can
be split into two sessions (e.g.one 4 hour workshop including activities 1 to 6 followed by one
3.5 hour workshop consisting of activities 7 to 11).
Activity Hours
1 Introduction 0.3
2 How to ruin a course / pedagogical features 0.33
3 Personas 0.5
4 Barriers and challenges 0.66
5 Evidence and examples of X 0.5
6 Initiate, Ideate, Investigate: produce your Vision 1.5
7 Connect: gather tools and resources 0.75
8 Prototype 1.5

METIS PROJECT
9 Evaluate 0.75
10 Reflect 0.5
11 Wrap up 0.25
Total 7.74
Activities 1 to 6 (half day workshop 1: Context and vision,
plus wrap up) 3.79
Activities 7 to 11 (half day workshop 2: Prototype and
evaluate, plus intro and ice breaker) 3.25
Table 2: activity durations
2.2 People: Participants, facilitators and others
Participants work in teams composed of 3 to 6 members. This team size encourages and
facilitates each team member to be fully involved in design discussions throughout the
workshop. A team size of 3 to 6 members is optimal because it allows for a diverse range of
views to be debated. Teams of more than 6 often split up into smaller teams and lose focus; this
should be avoided if possible.
It is highly advisable to allocate a ‘critical friend’ to each team to challenge design thinking and
stimulate focused and informed discussion (Cross, Galley, Brasher, & Weller, 2012, p. 28). This
‘critical friend’ role could be played by the workshop facilitators circulating amongst the teams,
but if there are many more teams than facilitators it is advisable to recruit additional ‘critical
friends’ to ensure that each team can benefit from their input and challenges. A learning design
workshop facilitator must have detailed knowledge of the both the workshop topic and the
Integrated Learning Design Environment (ILDE) so he/she can answer any question that might
arise during the workshop. A distinction needs to be made between a critical friend and a
facilitator. A critical friend is required to have knowledge of the topic in order to prompt and
encourage a focused and robust discussion. Obviously, some awareness of the ILDE will be
useful for critical friends. A learning design workshop facilitator is an expert on both the topic
and the ILDE.
2.3 Resources
Each team needs access to one or more laptop or desktop computers with internet access to
interact with the ILDE and other online resources. Prior to the workshop starting, an empty
learning design for each team will be created in the ILDE, and all team members will be given
editing rights for that learning design. If other resources such as printed material and specific
online resources are required for a particular activity, they are described within the relevant
activity in the activity structure that follows.

METIS PROJECT
Workshop activity structure
A diagram of the structure of the participants’ activities during the workshop is shown in Figure
2. This illustrates the relationships between the activities, the tools to be used, the resources to
be produced.

METIS PROJECT
Figure 6: Participants activities, tools used, and resources produced
1 Introduction (0.30 hour)
Facilitators introduce themselves, describe the structure of workshop, the organisation in terms
of the available human resources, tools and information and the intended learning outcomes.
They also introduce the critical friends and explain their role in workshop.
Facilitators’ resource: slideshow with notes, ILDE.
2 How to ruin a learning experience (0.33 hour)
(Note: this activity reuses the OULDI activity http://cloudworks.ac.uk/cloud/view/2597).
This icebreaker activity will focus on the key issues and strategies that impact on the success (or
otherwise) of learning and teaching within their context. The output of this activity will be a
design checklist which can be used as one of the design evaluation tools in a mid-way design
review and at the end of the workshop.
Resources (all to be available online and in printed form at the workshop)
Participants’ resources:
 Instructions
1. List the 10 best ways to ensure that the learning experience you are designing will
fail! (or: 10 ways technology can ruin the learning experience you are designing)
2. Share these with the others by uploading your lists to the ILDE.
3. What are the key themes?
Output
A first version of a design checklist which can be used as one of the design evaluation tools in a
mid-way design review and at the end of the workshop. The checklist will be uploaded to the
ILDE.
3 Personas (0.5 hour)
“Personas are a tool for sharing our understanding of our expected users, as a starting point for
design” (Mor, 2013). In this activity the participants will create descriptions of 2 or 3 personas
relevant to the context in which they teach. This activity is a first step towards a detailed
specification of the context in which the “X” learning activity will occur.
Resources

METIS PROJECT
 Printed and online versions of the Personas template (appendix x) will be provided for
participants. (The online version will be in the ILDE).
Output
The personas created will be uploaded or linked to the ILDE.
4 Barriers and challenges (0.66 hour)
20 minutes team work, 20 minutes plenary (assuming 3 or 4 teams)
In this activity the participants are asked to describe what they see as the barriers and
challenges with respect to designing and running ‘X’ learning activities in their context.
Facilitators refer to the activity “How to ruin a course”, where they will have already created a
list that is likely to be applicable to any type of course or activity. This activity prompts
participants to relate their current understanding of the workshop topic (X) to the context in
which they teach. It also allows the facilitators and critical friends to gain an understanding of
each team’s context. The participants will use the personas created in the “Personas” activity to
discuss and answer the following questions for a learning activity on ‘X’ (e.g. collaborative
learning)
a) What are the barriers and challenges from a learners’ perspective of ‘X’?
b) What are the barriers and challenges in implementing ‘X’ in your course or topic
area?
This will be a collaborative mapping exercise:
 Each participant writes down 3-4 barriers and 3-4 motivations, each one on a separate
post-it (5 minutes).
 Participants place their post-its on an A1 paper and arrange them in some order or map
(15 minutes).
 Each group presents its map to the whole workshop (10 minutes)
 Each participant thinks about their own view, discusses them with one other member
and works collaboratively towards a team list of at least 5 barriers/challenges for
learners and 5 for barriers/challenges for implementation. (20 minutes)
Each team shares their list with the wider group. These will be shared with the other teams by
displaying them in the workshop and each team will describe one from the learners’ perspective
and on from the implementation perspective in the plenary (one only to limit the time spent on
sharing). Facilitators foster focused discussion of remaining barriers/challenges between
different teams to occur during breaks.
Resources (all to be available online and in printed form at the workshop)

METIS PROJECT
 Facilitators’ resources: slideshow of questions (as above).
 Participants’ resources: slideshow with questions as above with notes indicating themes
that they might discuss (e.g. technical, motivational, temporal challenges). One A1 sheet
and several post-it packs per team(As an alternative to post-its, participants can use a
concept mapping tool of their choice.)
 Instructions
1. As individuals, write down 3-4 barriers and 3-4 motivations, each one on a separate
post-it. Use green post it notes for barriers/challenges from learners’ perspective,
yellow post-it notes for barriers/challenges for implementation (5 minutes).
2. Share these with the others in your team by placing your post-its on an A1 paper and
collaborate to arrange them in some order or map (15 minutes)
3. Each team should present its map to the whole workshop; focus on describing up to
5 barriers/challenges for learners, and up to 5 barriers/challenges to implementation
from the map (20 minutes)
 Critical friends’ resources: document describing suggestions for questions to ask.
Output
Each team should produce a list of up to 5 barriers/challenges for learners, and a list of up to 5
barriers/challenges to implementation. These will be added to the output produced in the first
activity ‘How to ruin a course’ to produce a new version of the team’s evaluation checklist
5 Evidence and examples of ‘X’ (0.5 hour)
The facilitators present a few examples of X that are chosen so as to be relevant to the
participants. (four or five examples should be sufficient). Evidence demonstrating that each
example is effective is also presented.
Each team selects one or two for review, and notes design features that may be transferred to
their context.
Facilitators and critical friends will support the discussion by pointing out particular aspects of
the example designs critical to the success of the example.
Resources
 Slides showing the example designs, descriptions of their use, and evidence of their
success.
 Copies of academic papers will also be provided. The focus of the presentation will vary
depending on the participants, and will vary from workshop to workshop.

METIS PROJECT
Output
Notes on potentially useful design features added to the ILDE.
6 Initiate, Ideate, Investigate: produce your Vision (1.5 hours)
In this activity each team will describe their vision for a ‘X’ activity. This is a first draft, and it may
be modified during the workshop. This vision should focus on describing the effects the activity
is intended to have on the learners (not on how these effects will be achieved which is the focus
of activity 8 ‘Prototype’).
Resources
 Printed templates for the Course Map (see appendix B.5 in Hernández-Leo, Asensio,
Chacón, & Prieto, 2013) need to be provided to participants and they will be able to
complete this within the ILDE.
 For the Learning Outcomes View (Galley, 2010) A3 sheets of paper and CompendiumLD
icon post-it notes need to be provided, along with a guide showing participants how to
create a Learning Outcomes View in CompendiumLD should participants wish to
produce a digital version..
Output
The vision should be described in terms of
 Learning outcomes
Other outcomes (e.g. affective outcomes such as individual motivation, confidence, team
building)
 Leaners’ outputs
An initial description of the evidence required to indicate that the learners have reached
the learning outcomes. Examples include a written piece of work, and an observable
behaviour.
Participants will produce representations such as Course Map and Learning Outcomes View.
The Learning Outcomes View is probably most appropriate as that view is very simple to
produce, and gives a specification that a single activity can be evaluated heuristically against. In
contrast the Course Map is intended to describe complete courses as a whole such as a course
of 100 or more hours of study time. The Course Map could still be applied to show the context
for the ‘X’ learning activity that the participants will design. Other representations of context
could also be introduced (e.g. factors and concerns table) to help focus participants’ attention
on consideration of the context where the design will be run.

METIS PROJECT
7 Connect: gather tools and resources (0.75 hour)
By this stage of the workshop participants will have an idea of the kind of behaviours they want
their learners to demonstrate, as well as the kind of products/competencies they desire learners
to produce/become proficient in during the learning activity they are collaboratively designing.
These need to be described in the learning outcomes view.
In this activity the learners are shown a set of ‘X’ learning patterns. The examples used in the “5
Evidence and examples of ‘X’” activity will provide concrete examples of some or all of the
patterns that are now made available to the participants.
Each team will select one or more patterns which best suit their articulated Vision as defined by
the Course Map, Learning Outcomes view, Personas and the barriers and challenges that they
developed during the preceding activities. If more than one pattern is chosen it could be
because there are alternatives, or because the team thinks that a sequence of ‘X’ activities is
necessary.
They now begin the detailed design. Questions that should be used as prompts for this session
include:
 Which parts of the activity should be synchronous, and which should be asynchronous?
 Which tools have the right affordances for your activity?
This could be carried out further using a Think-Pair-Share activity.
Resources
 A set of design patterns for X. These should be provided both online and in printed
form.
Output
The selected pattern(s), a set of resources that are to be used within the activity and an
annotated diagram of the pattern(s) showing:
 asynchronous and synchronous stages within the design
 where particular tools and resources are used.
8 Prototype (1.5 hour)
Participants work together to produce a prototype of an ‘X’ activity. The prototypes illustrate
tasks the facilitator and learners may choose to undertake. Prototyping would use a suitable
ILDE tool or be carried out using the technology of the team’s choice (including paper).
Resources

METIS PROJECT
Participants’ resources: a guide to the affordances of the ILDE instantiation and deployment
tools which explains which tools may be best suited to instantiate and run patterns of type X.
9 Evaluate (0.75 hour)
An appropriate summary view of each team’s design is selected by the team to be shared and
evaluated. The summary view to be used will depend on ‘X’ (e.g. if ‘X’ is Collaborative Learning
the appropriate summary view will be generated by Web Collage).
Designs are swapped amongst each group and evaluated heuristically (http://www.ld-grid.org/resources/methods-
and-methodologies/heuristic-evaluation).
Facilitators and critical friends advise on how to apply the teams evaluation checklist to the
design in question.
The final step is to share the result of the evaluation back to group as a whole.
Resources
Participants’ resources: the design of another team, along with the same team’s evaluation
checklist (i.e. the output from Activity 4).
10 Reflect (0.5 hour)
In the last activity (‘Evaluate’), each group’s design was evaluated by another group, and the
results of the evaluation shared. Now everyone should have an understanding of all the designs.
Participant groups should now reflect on if and how their activity design could be connected to
other groups’ designs to produce a course.
11 Wrap up (0.25 hour)
The facilitators conclude by describing how participants can find out more about learning design
in general, and the ILDE in particular.
Resources
Facilitators’ resource: slideshow with notes, ILDE.
References
Cross, S., Galley, R., Brasher, A., & Weller, M. (2012). OULDI-JISC Project Evaluation Report.

METIS PROJECT
Retrieved 3/8/2012, from http://oro.open.ac.uk/34140/1/OULDI_Evaluation_Report_Final.pdf
Mor, Y. (2013). Personas - The Learning Design Grid, Retrieved 9/5/2013, from
http://www.ld-grid.org/resources/representations-and-languages/personas

METIS PROJECT

METIS PROJECT
Appendix 2. Methodology descriptions
This appendix lists the high-level descriptions of methodologies which formed the
basis for the review of best practices. For each methodology, we compiled an
overview and an index of key references.
a. Learning Design Workshop Methodologies from the ICOPER Best
Practice Network
Summary
Two methodologies were developed and then delivered several times during the
ICOPER eContentplus project8
by the people in the Instructional Design work
package. ICOPER was about interoperability and the alignment of all phases in the
instructional analysis, design and deployment lifecycle based on intended and
achieved learning outcomes.
(1) The first series of workshops was targeted at propelling the learning design
proficiency (in particular IMS LD) at partner institutions. We gave them a brief
narrative description of a concrete unit of learning. We presented the idea of
learning design and the set of elements that are used in LD to abstractly describe
a teaching/learning process [1]. We then asked them to describe the unit of
learning using the IMS LD level A + B concepts (i.e., role, activity, activity
structure, environment, role-part, property, condition). For results see [2].
(2) The second series of workshops was to give teachers at partner institutions a
hands-on feeling of how to think of their teaching / courses in terms of concepts
that are demanded by sound instructional design (alignment of learning
outcomes, teaching methods, and assessment) as well as specified in the
European Qualification Framework for lifelong learning (EQF). The latter was
used in particular to foster understanding of the concept of learning outcomes
and how to formulate those. The procedure: we gave them some background on
learning outcomes (from EQF perspective), then some theoretical input on how
to align intended outcomes, methods and assessment [3]. Then we had them
"re-conceptualize" one of their actual courses using the provided input.
Both workshops were heavily based on hands-on work based on concrete problems
from the participants' context. While we had workshop procedure (1) in both
software-based and paper-based settings, we conducted workshop procedure (2)
mostly paper based using simple, uniform templates.
8
http://icoper.org/

METIS PROJECT
Key references
[1] M. Derntl (2010). ICOPER IMS LD Learning Design Hands-on Workshop.
Presentation recording: http://distance.ktu.lt/vips/flash/play.php?&rid=5846 and
slides: http://www.slideshare.net/mikederntl/introduction-to-ims-learning-design
[2] M. Derntl, S. Neumann, D. Griffiths, P. Oberhuemer (2012). The Conceptual
Structure of IMS Learning Design Does Not Impede Its Use for Authoring.
Transactions on Learning Technologies, 5(1): 74-86
[3] M. Derntl, S. Kabicher, P. Oberhuemer (2010). Workshop on outcome based
education. Workshop material. Available at METIS internal
site:https://docs.google.com/viewer?a=v&pid=sites&srcid=bWV0aXMtcHJvamVjdC5
vcmd8bWV0aXMtaW50ZXJuYWx8Z3g6NDIxZTY0MDUzYzk1Yjc2Ng
b. 7Cs of learning design framework
Summary
The 7Cs of learning design framework (Figure 1) illustrates the key stages involved in
the design process, from initial conceptualisation of a learning intervention through
to trialling and evaluating it in a real learning context.

METIS PROJECT
Figure 1: The 7Cs of Learning Design Framework
The framework consists of the following stages:
1. Conceptualise: Vision for the course, including:
a. Why, who and what you want to design
b. The key principles and pedagogical approaches
c. The nature of the learners.
2. Capture: Finding and creating interactive materials, including:
a. Undertaking a resource audit of existing OER
b. Planning for creation of additional multimedia such as interactive
materials, podcasts and videos
c. Mechanism for enabling learners to create their own content
3. Communicate: Designing activities that foster communication, such as:
a. Looking at the affordances of the use of different tools to promote
communication
b. Designing for effective online moderating
4. Collaborate: Designing activities that foster collaboration, such as:
a. Looking at the affordances of the use of different tools to promote
collaboration
b. Using CSCL (collaborative) Pedagogical Patterns such as JIGSAW, Pyramid,
etc.
5. Consider: Including three elements:
a. Designing activities that foster reflection
b. Mapping Learning Outcomes (LOs) to assessment
c. Designing assessment activities, including diagnostic, formative,
summative assessment and peer assessment
6. Combine: Combining the learning activities into the following:
a. Course map, providing a holistic overview of the nature of the course
b. Activity profile, showing the amount of time learners are spending on
different types of activities
c. Storyboard, creating a temporal sequence of activities mapped to
resources and tools
d. Learning pathway, providing a temporal sequence of the learning designs
7. Consolidate: Putting the completed design into practice in the following ways:
a. Implementation in the classroom, through a VLE or using a specialised
Learning Design tool
b. Evaluation of the effectiveness of the design
c. Refinement based on the evaluation findings

METIS PROJECT
d. Sharing with peers through social media and specialised sites like
Cloudworks
For each of the seven stages a series of conceptual design tools have been
developed, including the Course Features card, Course Map, Activity Profile,
Storyboard, Resource Audit tool, and E-tivity template.
The framework and its tools and resources have been trailed in the HE context,
including four UK universities and overseas institutions such as SAIDE (South African
Institute of Distance Education). In addition, we have run a series of 7Cs workshops
at a number of international conferences, with participants from a variety of
different educational sectors. In METIS, we think that the 7Cs methodology offers
the potential to be applied to other educational sectors, such as adult learning and
vocational training. The conceptual design tools developed from the 7Cs enable
teachers at all level of education to rethink their design practice and to create more
engaging learning experience for their learners.
Key references
Conole, G. (forthcoming). Innovative approaches to learning design – harnessing new
technologies for learning. In T.D. Bilham (eds.) For the Love of Learning: innovations
from outstanding university teachers. Palgrave MacMillan.
Conole, G. (2013). Current thinking on the 7Cs of learning design (blog post).
http://e4innovation.com/?p=628.
SPEED project. 7Cs toolkit: http://www2.le.ac.uk/departments/beyond-distance-
research-alliance/projects/speed, and SPEED blog:
http://speedprojectblog.wordpress.com/
c. Design-Practice
Summary
Design-Practice project (http:// www.design-practice.org) was aimed at preparing
them in integrating ICT in their teaching and advancing their lifelong learning skills by
building a community of teachers for sharing, discussing, debating, and improving
instructional activities and learning designs. During the project an online portal with
community of teachers and face-to-face and online teacher training modules were
developed.
The workshops were held in Cyprus, Greece and United Kingdom and were based on
the work of the Open University UK (OU) on Learning Design, which was first

METIS PROJECT
developed in the context of the OULDI (Open University Learning Design Initiative)
project. A detailed description of the workshops is available at:
http://www.open.ac.uk/blogs/OULDI/?page_id=987
The workshops included the following 7 activities:
Activity 1 - How to ruin a course: The main goal of this activity was to pinpoint key
issues and strategies that can affect the success (or otherwise) of learning and
teaching within an educator’s context.
Activity 2 - Comparing four web 2.0 tools: This activity was aimed at introducing
participants to four web 2.0 tools and to motivate them to think of various ideas on
how to use these tools in the classroom.
Activity 3 - Affordances: Tools in use: The aim was to explore the ‘affordances’ of one
or more technological tools. The goal was to encourage participants to develop
critical thinking and to make judgements as to which tools are the right ones to use
in their teaching based on their affordances.
Activity 4 - Course Map: The goal of the activity was to propose a way to represent
lessons which can help the design and implementation of a lesson and it can also
facilitate the sharing of designs amongst educators.
Activity 5 - Mapping a design using learning design notation: The goal was to
introduce a method for designing learning activities to the participants.
Activity 6 - Breaking-down a design into the micro-level: This activity was aimed at
further analyze the lesson representation designed in Activity 5.
Activity 7 - Sharing and discussing designs: The goal was to develop an environment
of cooperation and sharing, which can be facilitated with the use of social
networking tools such as Cloudworks.
Key references
Vrasidas, C., Conole, G., Retalis, S. (2010). Usable representations of Learning Design
for Educators & Instructional Designers. Workshop at Online EDUCA Berlin,
December, 2010. (http://www.slideshare.net/pambos/usable-representations-of-
learning-design)
Vrasidas, C., Theodoridou, K., Theodoulou, F., Aravi, C., Pattis, I. (2010). Design
Practice: A Framework for Preparing Teachers to Teach with ICT. International Visual
Literacy Association Conference.

METIS PROJECT
d. Learning Design Studio
Summary
The learning design studio is a collaborative, blended, project based framework for
training educators in effective and evidence-based use of educational technology.
This approach is modelled after the tradition of studio-instruction in arts and design
disciplines (such as architecture). In this model, the main activity of a course is the
students' continued work on design challenges in a defined domain of practice.
Students typically work in groups. They identify an educational challenge, research it,
and devise innovative means of addressing it. The course instructor guides the
students through the process, and classroom sessions are mostly dedicated to group
work and public review of design artefacts.
The Learning Design Studio manifests a model of teaching a Design Inquiry of
Learning. Design, in this context, is the informed creative practice of devising
“courses of action aimed at changing existing situations into desired ones” [0Simon,
1996, p 129]. Inquiry-based learning attempts to shape educational experiences in
the model of scientific investigation. Similarly, an inquiry approach to the training of
educational practitioners should mimic the form of design research in education.
Thus, the learning design studio mimics the structure of a design experiment (Mor &
Winters, 2007), with the exception that students do not have the resources or the
time to conduct several iterations, scaling up from a conceptual prototype to an
extensive deployment.

METIS PROJECT
Key references
Mor, Yishay and Mogilevsky, Orit (2012). A Learning Design Studio in Mobile
Learning. In: The 11th World Conference on Mobile and Contextual Learning (mLearn
2012), 16-18 October, Helsinki.
Mor, Y, & Mogilevsky, O. (submitted to EC TEL) The Learning Design Studio:
Educational Practice as Design Inquiry of Learning
https://docs.google.com/file/d/0B5iNZunquTigUVN1bTdwYi1UVTA/edit
Other references
Anastopoulou, S.; Sharples, M.; Ainsworth, S.; Crook, C.; O'Malley, C. & Wright, M.
(2012), 'Creating Personal Meaning through Technology-Supported Science Inquiry
Learning across Formal and Informal Settings', International Journal of Science
Education 34 (2) , 251-273
Cox, C.; Harrison, S. & Hoadley, C. (2008), 'Applying the "studio model" to learning
technology design',Educating learning technology designers: guiding and inspiring
creators of innovative educational tools, 145
Kali, Y, & Ronen-Fuhrmann, T. (2011). Teaching to design educational technologies.
International Journal of Learning Technology, 6(1), 4–23.

METIS PROJECT
Mor, Yishay and Winters, Niall (2007). Design approaches in technology enhanced
learning.Interactive Learning Environments, 15(1) pp. 61–75.
Mor, Yishay (2013). SNaP! Re-using, sharing and communicating designs and design
knowledge using scenarios, narratives and patterns. In: Luckin,
Rosemary; Puntambekar, Sadhana; Goodyear, Peter; Grabowski, Barbara
L.; Underwood, Joshua and Winters, Niall eds. Handbook of Design in Educational
Technology. London, UK: Routledge, (In press).
Ronen-Fuhrmann, T, & Kali, Y. (2010). The role of concretization in acquiring design
knowledge. Proceedings of the 9th International Conference of the Learning Sciences-
Volume 1 (pp. 468–475).
Ronen-Fuhrmann, T., Kali, Y., & Hoadley, C. (2008). Helping Education Students
Understand Learning Through Designing. Educational Technology, 48, 26–33.
Simon, H. A. (1996), The Sciences of the Artificial - 3rd Edition , The MIT Press ,
Cambridge, MA
e. Collage / LdShake workshops
Summary
At the UVA team (and later, also at UPF as Davinia moved from Valladolid to
Barcelona), we have hosted several teacher workshops related with our research
efforts in the field of Computer-Supported Collaborative Learning and Learning
Design. These workshops were mainly directed at university teachers (although the
approach can be applied to practitioners at other levels – as long as the
technological tools proposed make sense in the actual context). The workshops
aimed to promote participants’ awareness of collaborative learning techniques (e.g.
in the form of collaborative patterns such as the Jigsaw). The main focus was on
providing practical guidance about how to design such learning experiences (both
conceptually by using and combining patterns at different levels, and using
technological support such as the Web Collage authoring tool). In the last editions
there has also been an emphasis on how to implement those collaborative designs
using VLEs (such as Moodle) and external ICT tools (e.g. wikis, Google apps, etc.),
again conceptually and through the use of the GLUE!-PS technological system.
An example of workshop can be found in Hernández-Leo et al. (2011), in which
LdShake (the precursor to METIS’s ILDE) is shown and used by teachers. Only very

METIS PROJECT
recently we have made efforts in reflecting on the workshop approach that has
emerged through these years of research and professional development
interventions. Prieto et al. (accepted) describe one instance of the last round of
workshops that have been held (focusing on design, implementation and
enactment), and tries to extract some of the main guiding principles and patterns of
the approach:
 Focusing on multiple aspects of the activities’ “orchestration” (beyond pure
design, also management, adaptation, pragmatic restrictions, etc.)
 Focusing on multiple phases of the activities’ lifecycle (beyond design,
towards implementation, enactment)
 Use of pedagogical patterns (e.g. design patterns like Jigsaw, but also
deployment/enactment patterns like “on-the-fly monitoring of activities”)
 Providing hands-on practical experience not only in the concepts of design,
but also the use of technologies to design and implement the activities
 Modeling: the workshops themselves are designed and modelled using the
tools and strategies being taught at the workshop
 Authentic/meaningful problems: Participants work on design/enactment
problems that are meaningful for their own concrete courses
Key references
Description of one workshop instance and main characteristics of the approach:
Luis P. Prieto, Yannis Dimitriadis, Juan I. Asensio-Pérez, Sara Villagrá-Sobrino, Iván M.
Jorrín-Abellán (accepted). Fostering CSCL adoption: an approach to professional
development focused on orchestration. Accepted at the International Conference on
Computer-Supported Collaborative Learning (CSCL 2013).
Example of LdShake workshop with teachers: Davinia Hernández-Leo, Lauren
Romeo, Miguel A. Carralero, Jonathan Chacón, Mar Carrió, Pau Moreno, Josep Blat
(2011). LdShake: Learning design solutions sharing and co-edition, Computers &
Education, 57(4), 2011, p. 2249-2260. (see sp. Table 1 and section 3.1)
f. Participatory Pattern Methodology
Summary
The Participatory Methodology for Practical Design Patterns is a process by which
communities of practitioners can collaboratively reflect on the challenges they face
and the methods for addressing them. The outcome of the process is a set of

METIS PROJECT
Design narratives, design patterns and design scenarios situated in a particular
domain of practice. This pattern is an “envelope” for the rest of the patterns in this
paper, and the context described here is the baseline for all the others.
At the heart of the methodology is the PARTICIPATORY PATTERN WORKSHOPS
pattern, which describes the interrelation between three COLLABORATIVE
REFLECTION WORKSHOPS: a DESIGN NARRATIVES WORKSHOP, a DESIGN PATTERNS
WORKSHOP and a DESIGN SCENARIOS WORKSHOP.
Apart from these, the methodology includes a “toolkit” of support patterns, which
address critical points in the process or specific recurring needs.
The methodology is based on two fundamental assumptions: we are all experts, and
we are all designers. This methodology utilises narrative epistemology: practitioners
are prompted to recount their experiences as design narratives, and discuss these
with their peers. The construction and discussion of these narratives are scaffolded
by a set of tools and activities to extract transferable and verifiable elements of
design knowledge in the form of design patterns.
Key references
Mor, Yishay; Warburton, Steven and Winters, Niall (2012). Participatory pattern
workshops: a methodology for open learning design inquiry. Research in Learning
Technology, 20.

METIS PROJECT
Mor, Yishay (2010). Embedding design patterns in a methodology for a design
science of e-Learning.In: Kohls, Christian and Wedekind, Joachim eds. Problems
Investigations of E-Learning Patterns: Context Factors, Problems and
Solutions. Hershey, PA, USA: IGI, pp. 107–134.
http://projects.lkl.ac.uk/ppw
Video - Part 1
Video - Part 2
g. Design Challenge
Summary
This is one example of a series of related workshop designs conceived, delivered and
evaluated by the Open University Learning Design Initiative (OULDI). This one-day
workshop was developed to introduce course teams to using a learning design
approach at the curriculum level as opposed to lower level detailed design of
specific activities. The example below (originally developed in collaboration with the
Open University’s Faculty of Education and Language Studies (FELS) in summer 2009)
uses the OULDI learning tools and activities but tools and activities from other
projects (especially the Viewpoints project) have been used successfully and the
format holds well. For a list of other activities and tools you could use try the
learning design toolbox.
Designed to be fun and engaging, the event gives an awareness of the latest in
thinking innovatively about curriculum design and is designed to be proactive rather
than being composed around an uncontextualised set of one-to-many presentations.
The time-limited challenge enables participants to make a preliminary assessment of
which of the ideas, tools and resources are useful and gives them some feel for what
might be possible in a longer, term real course production process. The challenge is
intended to open avenues for participants to pursue this further.
The OULDI team delivered almost twenty workshops during the project pilots
including fourteen directly associated with the pilots. Post-workshop questionnaires
and later impact surveys reveal a wide range of reaction from participants - even
those present at the same workshop. Overall, feedback has been overwhelmingly
positive (Cross, et al., 2012).
With respect to application to METIS, the following issues should be considered.

METIS PROJECT
(1) The OULDI workshops were designed for Higher Education practitioners. All
of the activities and tools are transferable to other sectors such as those
represented by KEK and Agora, but customisation will be necessary.
(2) In general, the OULDI tools and workshops focus on curriculum level design
whereas METIS is focused on activity design. This means that a selection of
the tools and activities can be usefully applied within METIS (for example, to
specify the educational context of a particular learning activity), they must be
complemented by workshop activities and tools targeted at activity level
design.
Key references
A schedule for the workshop including a description of the activities and tools:
The Open University Learning Design Initiative. (2012). Workshop template: Design
Challenge Retrieved 9/5/2013, from
http://www.open.ac.uk/blogs/OULDI/?page_id=985
Evaluation of the workshops is contained in:
Cross, S., Galley, R., Brasher, A., & Weller, M. (2012). OULDI-JISC Project Evaluation
Report Retrieved 3/8/2012, from
http://oro.open.ac.uk/34140/1/OULDI_Evaluation_Report_Final.pdf
h. OULDI Learning Design Training Module
Summary
As part of the EU Design-Practice project the OU Learning Design Initiative delivered
d a learning design training module to around 40 OU Associate Lecturer participants
in the UK using Cloudworks. This module has a higher theoretical content than the
“Design Practice” workshops (see appendix 2.c; these “Design Practice” workshops
were modelled on the OULDI’s “LD Lite” template which uses more of a practice
based approach).
The learning outcomes of the training module are to improve participants’
knowledge and understanding of:
 The design process for learning activities
 Choices that practitioners make about the ways of applying technologies for
a variety of learners
and to improve participants’ ability to :

METIS PROJECT
 Use and evaluate particular technologies and tools for individual and
collaborative learning
 Share and collaborate learning designs.
The training had three components:
(1) A five hour (approximately) self-paced introduction to learning design
module which was held online. This provided the background to (2) below.
(2) A five hour face-to-face ‘Using Technologies in Teaching’ workshop.
(3) A five hour online self-paced activity, which draws on the work undertaken in
the workshop, and is based on the collaboration and sharing of learning
designs.
The training module provides an overview of the methodology for learning design
developed by the Open University's Institute of Educational Technology. The module
is aimed at participants with an interest in using technology in their teaching practice
and/or are involved in the design of learning activities at all scales.
With respect to application to METIS, the following issues should be considered.
(1) The OULDI workshops were designed for Higher Education practitioners (e.g.
it requires the participant to read academic papers).
(2) The duration of the complete module is 15 hours. However, activities from
within the face-to-face and second online components could be customised
to suit METIS participants.
(3) In general, the OULDI tools and workshops focus on curriculum level design
whereas METIS is focused on activity design. However, activities from within
the face-to-face and second online components of this module are targeted
at activity design and sharing and so could be customised to suit METIS
participants.
Key references
A schedule for the workshop including a description of the activities and tools:
http://cloudworks.ac.uk/cloudscape/view/2294.

METIS PROJECT

METIS PROJECT
Appendix 3. Methodology design narratives and design patterns
a. “Design Narratives” task presented to partners
The following instructions where provided to project partners asking them to
provide design narratives of successful workshop practices:
METIS Methodology Design Narratives: telling a ‘good’ story
Task: Each METIS partner authors 2-3 Methodology Design Narratives
(see http://www.ld-grid.org/resources/representations-and-languages/design-
narratives), based on an activity, from your own methodology (the methodology
descriptions you authored are located here[1]).
The idea behind the methodology design narrative is simple: narrate a ‘good’ story
about a workshop activity (e.g. icebreaker, task, wrap-up, etc.) that was successful.
This can be from a workshop you either led or attended.
The narrative should describe what participants were asked to do “X” (e.g.
participate in an activity and/or complete a task, etc.) and what was the intended
outcome “Y”.
In writing the narrative please describe the “how” in as much detail as possible (e.g.
steps A, B, which led them to the intended outcome Y).
You can do this by providing a “thick description” of the activity and sequence of
events participants (protagonists) were given, the steps (challenge), their choices
and the results (positive ones). If you have a sample of what materials were used in
the activity or photos this is also useful.
Rationale: We require these Methodology Design Narratives for WP3. Once we have
them, we will guide you in extracting patterns from them. Our goal is to extract
applicable patterns from existing Methodology Design Narratives, with a view to
applying some (or all) of the applicable patterns to the METIS workshop design.
LD Grid: Please have a look at The Learning Design Grid’s[2] "Healthy Eating[3]"
as a Design Narrative example. This example is far more extensive than what
we're asking for here, but if anyone wants a broader view it may be useful.
Process: Please download the file below 'metis-methodology design narratives-
template_template.docx' (Sample Methodology Design Narrative Template) and
metis-methodology design narratives_Sample.docx (Sample Methodology Design
Narrative). Then using the template (and sample) author 1 to 3 Methodology

METIS PROJECT
Design Narratives and upload your narratives (each one in a separate file) on this
page by May 15, 2013.
To assist you in this task and for consistency across the project, we are providing the
following template and example in the word document below. The example is of
one activity, out of eight, from a workshop that lasted 2 hours.
[1] See https://sites.google.com/a/metis-project.org/metis-internal/workpackages/wp3-workshop-
design/methodology-cross-review/methodology-descriptions
[2] http://www.ld-grid.org/resources/representations-and-languages/design-narratives
[3] https://sites.google.com/a/ld-grid.org/www/resources/learning-designs/pi-project-healthy-eating-
activity/healthy-eating-design-narrative
b. Design narrative template
Situation
A Tagline for your project or work
Describe the user group and the work context
Describe your technological setup
Task
Describe what you are trying to achieve
Actions (How did you try to address the issue?)
Activity .
Activity goal(s) or intended
outcome (“X”)
Activity title & description Title:
Activity materials

METIS PROJECT
The “how” (steps A, B, C, that
lead to Y or the activity
outcomes)
.
.
.
Activity outcomes (“Y”)
(positive)
The activity was successful because it achieved the
intended outcomes as well as:
Results (What were the results of the actions you took?)
Activity outcomes
(“Y”) (positive)
The activity was successful because it achieved the intended outcomes
as well as:
Reflection
Observation and what you have learned.

METIS PROJECT
Appendix 4. Sample of Methodology Design Narratives
a. Meta-Pyramid design
1. Situation
The GSIC-EMIC research group at the UVA has conducted several workshops in the last two
years, for professional development of academic teaching staff of the university. The main
aim of these workshops was to promote the adoption of computer-supported collaborative
learning in university teachers’ everyday practice, including both the conceptual side of
designing and running collaborative activities, and the technological one of using commonly
available ICTs (Moodle, web apps), but also specific ones (e.g. the Web Collage authoring
tool).
This narrative is extracted from a workshop held in February 2012, which made more
emphasis on the conceptual side of designing collaborative activities, and deciding about ICT
tools to use to support such learning. The workshop included 2 face-to-face sessions and a
few online activities, for a total of about 12 hours of workshop. There were 25 university
teachers from a variety of backgrounds and levels of teaching experience in the workshop.
The technological setup was based around a Moodle course with embedded Google Docs
documents for collaborative work; also, extensive use of pen and paper was also part of the
workshop philosophy.
2. Task
The main aim of the workshop was to provide teachers with a few best practices for
collaborative learning and its implementation using ICTs, at different levels, including high-
level patterns such as the Jigsaw9
, and lower level design and implementation “tricks” (which
we called “routines” or “atomic patterns”10
) extracted from successful collaborative learning
practice of other teachers. The ultimate goal of the workshop was that, after the workshop,
each teacher had created a collaborative scenario for his/her own practice, as a necessary
step for applying such techniques in practice. It is also important to note that the workshop
activities themselves were designed and structured around these best practices, so that
teachers could experience what collaborative learning “felt like” as an student (which not
everyone had done in their own previous education).
3. Actions (How did you try to address the issue?)
9
A very common collaborative strategy for addressing complex problems by dividing them in parts.
See http://pandora.tel.uva.es/wic2/patterns/en/jigsaw/ for a more complete description.
10
See Prieto et al.’s “Recurrent routines: Analyzing and supporting orchestration in technology-
enhanced primary classrooms” at
http://www.sciencedirect.com/science/article/pii/S0360131511000091

METIS PROJECT
Activity Designing a collaborative learning scenario.
Activity
goal(s) or
intended
outcome
(“X”)
 Engage teachers in their first collaborative learning design
 Encourage collaboration among participants
 Make a number of best practices available to teachers (patterns,
routines)
 Show the value of iteration in learning design
Activity
title &
descriptio
n
Title: Design a Pyramid activity using a Pyramid
This was a lengthy activity that encompassed three main phases, of 30-45
minutes each, where they had to address a fictitious but realistic scenario (the
need for a zero-course on bibliographic search and collaborative report writing
for undergraduates) that required them to design (and later, enact) a set of
collaborative learning activities. These three phases introduced progressively
collaborative best practices of different kinds (overall strategies or patterns,
such as the Pyramid11
; more concrete design “routines”; and even more
concrete deployment/implementation “routines”), which were explained briefly
by facilitators and provided to participants in the form of cards (see below,
translated from the original Spanish).
Also, this activity led participants to review their designs and those of their
peers, and iterate over them, going from the initial, rather abstract idea, up
until they had a clearer idea of how they would use ICTs to implement it.
Later in the workshop, this design was taken further, and parts of it were
enacted through a role-playing, to discuss yet another set of enactment
“routines” that might be useful when running such collaborative activities.
Activity
materials
 Large white paper surfaces
 Pens, highlighters, pencils
 A digital camera
 Scenario description (see an example in Spanish here).
11
See http://pandora.tel.uva.es/wic2/patterns/en/pyramid/, Hernández-Leo, D.; Asensio-Pérez, J.I.;
Dimitriadis, Y.; & Villasclaras, E.D. Generating CSCL Scripts: From a Conceptual Model of PAttern
Languages to the Design of Real Scripts. In: Goodyear P.; Retalis, S. (eds.). Technology-Enhanced
Learning, Design patterns and pattern languages, Sense Publishers, Series Technology-Enhanced
Learning; 2010. p. 49-64. Appendix available at:
http://ulises.tel.uva.es/%7Edherleo/dpbook/appendix-chapter.pdf

METIS PROJECT
 Pattern, design routine and deployment routine cards (examples in
Spanish are available here, here, and here, respectively), and an English
translation of the titles of such cards can be found here (see appendix
B)
The
“how”
(steps A,
B, C, that
lead to Y
or the
activity
outcomes
)
This activity took place after a short introduction by the facilitators to the
workshop methodology and goals, and about collaborative learning in general.
A. Explanation of the scenario: Although participant teachers had had to
read the scenario previously, the facilitators remind participants about
the scenario (zero course in bibliographic search and joint report
writing) and the scope of the design (10 hours of blended student
work).
B. First design iteration (with patterns): facilitators present very briefly the
idea of collaborative design patterns, and describe two of them
(Pyramid and Jigsaw). Then, participants are divided in heterogeneous
groups of three people (they had already been seated in such groups at
the beginning of the workshop to minimize hassle) and are provided
with a paper description of several such design patterns. Then, for 30
min, they are asked to design the learning activities, applying one of the
patterns (the Pyramid) to the provided scenario. The design is to be
done in paper, in whatever format they desire (textual, graphical,
diagrammatic, tabular...). Below we can see the result from one of the
groups:
C. Second design iteration (with design routines): Facilitators then briefly
present other set of best practices, of a lower granularity (design
routines), as elements of blended CSCL practice that have been
observed in successful practice by other practitioners, that can be used
to further flesh out the overall structure denoted by the design pattern.
Facilitators give participants a set of these routines (each on a yellow
card) and, in the same groups as in phase B, participants are asked to
re-iterate their design, using such cards to make the activities and
strategies to use more concrete. Again, 30 min are allocated to this

METIS PROJECT
task. Although it was not explicitly said, many teachers at this point
glued the yellow cards over different parts of the previous design, to
mark when a certain design routine would be used. At the end of this
phase, a short break is scheduled, which facilitators use to make photos
of the designs so far, uploading them to the workshop’s Moodle.
D. Cross-review of designs (with implementation routines): Now,
facilitators provide yet another set of routines (“deployment routines”,
represented by green cards), which are also lower-level, and which deal
with implementation and deployment details of a CSCL activity using
the workshop’s concerned ICTs (Moodle, Google Docs, etc.). Then, for
15 min, participants are asked to look at the design of another group
(and vice-versa), and provide three aspects they liked, three problems
they saw in the design, and suggestions of deployment routines that
might be useful in the context of that design. These suggestions are to
be written in Google Docs that have been set up beforehand by
facilitators (one per group), and are accessible through the workshop’s
Moodle.
E. Third design iteration: Participants now are asked to work in 6-person
groups by joining the two groups that have cross-reviewed each other
(thus, following the Pyramid pattern that is being used in the designs).
For another 30 min, participants are asked to produce a complete
design using all the elements (by re-iterating either of the groups’
original designs, or doing an entirely new one that takes advantage of
the strategies of both). Below, one of the final products is reproduced:

METIS PROJECT
Afterwards, other workshop activities were meant for teachers to reflect on the
experience they had just undergone, and to individually apply their recently-
discovered skill (designing collaborative activities) to their own real courses.
Activity
outcomes
(“Y”)
(positive)

4. Results (What were the results of the actions you took?)
Activity
outcomes (“Y”)
(positive)
The activity was successful because it achieved the intended outcomes:
 It exposed participants to a (fairly large and heterogeneous)
number of best practices
 It engaged participants in collaborating with each other,
intensely and for a lengthy period of time
 Teachers had a quite concrete, finished design they had done
themselves
 It provided them with an actual learning experience of the kind
they were designing
 It exposed them to the challenges and benefits of enacting such
collaborative activities using ICT (by seeing facilitators struggle
to follow the workshop’s collaborative design)
5. Reflection
 We use this kind of activity a lot lately in PD actions, especially in those workshops
that try to convey the concepts of collaborative learning to people that may have
little or no
 Despite the fact that teachers tend to like this kind of activity a lot (unless they are
a-priori against collaborative learning), it poses several important challenges (which
are nevertheless intrinsic to CSCL):
o It is quite long, but also very active (participants have to produce something
tangible regularly), and some teachers find it a bit “stressful”
o It requires a good amount of preparation (writing, printing and distributing
the cards, making the participant groups beforehand, etc.). The
technological side also has to be prepared (creation of the Moodle course,
GoogleDocs, etc.), but lately that has been relieved a lot by the usage of
WebCollage + GLUE!-PS (now part of the ILDE)
o Although it seems to be a very long activity, strict timekeeping is essential,
since the creative effort can be easily derailed by discussions, musings,
people just getting to understand each other (take into account that a group
may have people from wildly different disciplines).... Facilitators should
monitor the activities, solve any doubts early, and “push” groups to produce

METIS PROJECT
something (even if only a partial product) on time
o Some teachers often find it difficult to start the design process unless
concrete tips are given about what kind of product is expected – i.e. what
elements should a design have: is it groups and group sizes, timeframes,
materials to use... (but you do not want to make it too restrictive either to
the point that it becomes uncreative). The routine cards seem to help a bit
to make the level of detail needed explicit
 There is a tension between making the scenario (and the participants’ designs)
specific and relevant to only a certain domain (e.g. Engineering), which might make
people from other disciplines uninterested. On the other hand, making the scenario
too generic can be bland and get all participants uninterested. For now, we are
trying to make the scenario to be about a transversal problem (bibliographic search
and report writing), and let each group navigate the problem of how domain-specific
they want their design to be.
b. Role-playing on problematic situations and technology
1. Situation
tool and the GLUE!-PS system, now part of the ILDE).
This narrative is extracted from a workshop held in April 2012, which made more emphasis
on the technological side of designing collaborative activities, implementing such designs
with concrete ICT tools available to the teachers (Moodle, Google Docs and other web apps).
The workshop included 2 face-to-face sessions and a few online activities, for a total of
about 12 hours of workshop. There were 22 university teachers from a variety of
backgrounds and levels of teaching experience in the workshop. The technological setup was
based around a Moodle course with links to the authoring and deployment groups (Web
Collage, GLUE!-PS), and embedded Google Docs documents for collaborative work.
2. Task
The main aim of the workshop was to provide teachers with the necessary technological
skills to implement a collaborative learning and its implementation using ICTs, starting from
a fairly concrete conceptualization of the design (in descriptive form). The ultimate goal of
the workshop was that, after the workshop, each teacher had implemented a collaborative
scenario for his/her own practice, ready to be enacted by the students. The workshop also
emphasized the fact that problematic situations may arise when ICT technologies are used
for collaborative learning, and aimed to show teachers how to use the technology to address

METIS PROJECT
those problematic situations in practice. It is also important to note that the workshop
activities themselves were implemented and structured around the set of best practices and
technologies that participants were learning, so that teachers could experience what ICT-
supported collaborative learning “felt like” as an student (something that almost none of the
participants had done before).
Activity Problem-based technology practice
Activity
goal(s) or
intended
outcome (“X”)
 To be conscious of problems of using ICT for CSCL, especially when
using learning design approaches (e.g., due to a-priori instantiation)
 Review some of the “enactment routines” (run-time best practices
in CSCL) from previous workshops
 Show teachers practically how to address very common run-time
problems using the technology being taught at the workshop
(GLUE!-PS)
Activity title &
description
Title: Solving run-time problems with GLUE!-PS
This was a 1-hour activity that took place once teachers had learned how to
do a learning design with Web Collage and to deploy it using GLUE!-PS into
a Moodle or MediaWiki platform (first by following a walkthrough of the
tools, then by using them to do a design for their own practice and
deploying it into a VLE of their choice).
In this activity teachers are presented with three common situations that
pose problems when ICTs (sp. those addressed by the workshop: Web
Collage, GLUE!-PS, Moodle) are used to support CSCL. These situations
include the formation of groups at the beginning of an activity, and their re-
formation later on, in the face of unexpected occurrences; also, the need
for changing the supporting ICT tools on-the-fly during enactment (e.g. due
to a failure in the service provider – think, what happens if Google Docs
goes down?). Participants are prompted to solve one of the three situations
by freely exploring the GLUE!-PS interface in dyads, and then share the
solution to all situations in 6-people groups.
This activity occurred towards the end of the workshop, but still there was
time set for participant reflection on the challenges and advantages of
using this kind of ICTs to implement CSCL in everyday teaching practice (see
the mini-focus-group design narrative).
Activity
materials
 Documents describing the problematic situations and prompting
participants to action (see example in Spanish here).
 Documents describing step-by-step the solution to each
problematic situation (to be distributed after the workshop for
reference purposes) – see example here.
 Laptop computers (one per two participants), with Firefox/Chrome
browser
 A GLUE!-PS server and Moodle/MediaWiki servers, where each

METIS PROJECT
participant/dyad has an account and an example
design/deployment/course to experiment with
The “how”
(steps A, B, C,
that lead to Y
or the activity
outcomes)
F. General intro: Facilitators introduced briefly and in a general
manner the fact that ICTs can be useful in supporting CSCL and in
designing CSCL... but they may also fail and introduce problems,
especially during the enactment (see slide 12 here, in Spanish).
G. Situation description: Facilitators briefly describe each of the three
problematic situations, which are also given as short documents
(see example in Spanish here).
H. Participants, in dyads, are asked to enter the GLUE!-PS system
(linked from the workshop’s Moodle course) and try to solve one of
the problems using GLUE!-PS.
I. The dyads are joined in groups of 6 people (each dyad had solved a
different situation) and are asked to explain to their partners the
situation they solved and how they did it (by performing the change
live in the system).
J. Once the activity has ended, participants are given documents
detailing, step by step, the solution to each of the three
problematic situations (so that they can use them later in their
practice, if needed). These walkthroughs include both text and
images (see below):
Activity
outcomes (“Y”)
(positive)
as well as:
 Participants collaborated and helped each other to find a
solution
 Participants internalized the solutions by explaining them to
fellow participants
 Participants explored freely the GLUE!-PS user interface and
found intuitively solutions to the presented problems
 All this was achieved in an unexpectedly short amount of time

METIS PROJECT
5. Reflection
One of the main values of this part of the workshop (something that participants also
acknowledged later in questionnaires) was that it reckoned the possibility of failures when
using ICTs, and showed ways of tacking them, something that is seldom portrayed in PD
actions (sp. shorter ones). Other reflections:
 This activity’s progress was surprisingly smooth, and was performed by all groups in
an even shorter time than allocated. This may indicate that teachers understood
rapidly the internal logic of the GLUE!-PS system, or that we mis-assessed the effort
necessary by two people to tackle it.
 Related to this, the work in dyads at the beginning of the activity (dyads that were
pre-defined by facilitators, on the basis of being heterogeneous in tech-savviness)
seemed to help a lot in making progress along the different activities (sp. in the
beginning stages of a workshop, where participants still do not know the
technology). Of course, this runs the risk of one member of the dyad “doing all the
techie work” – but we tried to offset this by encouraging participants to change roles
in each new activity.
 Although in the end we could not do it because of lack of time, this activity could be
pre ceded by a short warm-up role-playing exercise, in which e.g. a 6-people group
enacts the classroom situation, posing as teacher and students.
c. Mini-focus-groups
1. Situation
tool and the GLUE!-PS system, now part of the ILDE).
This narrative is extracted from a workshop held in April 2012, which made more emphasis
on the technological side of designing collaborative activities, implementing such designs
with concrete ICT tools available to the teachers (Moodle, Google Docs and other web apps).
The workshop included 2 face-to-face sessions and a few online activities, for a total of
about 12 hours of workshop. There were 22 university teachers from a variety of
backgrounds and levels of teaching experience in the workshop. The technological setup was
based around a Moodle course with links to the authoring and deployment groups (Web
Collage, GLUE!-PS), and embedded Google Docs documents for collaborative work.

METIS PROJECT
2. Task
The main aim of the workshop was to provide teachers with the necessary technological
skills to implement a collaborative learning and its implementation using ICTs, starting from
a fairly concrete conceptualization of the design (in descriptive form). The ultimate goal of
the workshop was that, after the workshop, each teacher had implemented a collaborative
scenario for his/her own practice, ready to be enacted by the students. The workshop also
emphasized the fact that problematic situations may arise when ICT technologies are used
for collaborative learning, and aimed to show teachers how to use the technology to address
those problematic situations in practice. It is also important to note that the workshop
activities themselves were implemented and structured around the set of best practices and
technologies that participants were learning, so that teachers could experience what ICT-
supported collaborative learning “felt like” as an student (something that almost none of the
participants had done before).
Of course, there was a flip side to these pedagogical objectives, which was the fact that we
were doing research on the support that the workshop and the tools employed in it gave
teachers to apply CSCL in their everyday practice. This secondary goal of the workshop
enters into play prominently in this activity (since it may be shared by the METIS workshops
also).
Activity Small-group discussion
Activity goal(s) or
intended
outcome (“X”)
 For participants to reflect upon the applicability of the practices
and tools presented in the workshop, to their everyday practice.
 For researchers to gather participants’ opinions on the
usefulness of the proposed tools and practices (later to be
triangulated with other data sources)
Activity title &
description
Title: Problems and issues in real use of ICTs in (teaching) practice
This activity was to happen towards the end of the workshop, after
participants had designed, deployed, and solved run-time problems in
CSCL scenarios using ICT tools such as Web Collage and GLUE!-PS. The
idea is to have all participants reflect
Participants were divided in 6-people groups and three main “reflection
prompts” were provided (regarding the applicability of collaborative
learning to their practice, the applicability of the ICT tools presented,
and about the run-time problems presented and elicitation of further
problematic situations). Each of these small groups had one workshop
facilitator to moderate the debate, audio record it and keep participants
on track, and other roles are distributed among the members of the
group: timekeeper, scribe, spokesman. After 30 minutes of discussion,
the spokesmen highlight the one most important idea that emerged
from the discussion of each of the three prompts.
Once this activity was finished, the facilitators did a short wrap-up with
parting reflections and recommendations, before the workshop finishes.

METIS PROJECT
The Google Doc documents with the rest of the debate’s ideas for each
group are shared among all the participants through the workshop’s
Moodle.
Activity materials  Slides presenting the debate dynamics and the reflection
prompts for the group discussion (see slide 19 here, in Spanish)
 Large paper surfaces, post-its, pens and markers
 A Google Doc document for each small group
 One audio recorder for each small group (a laptop with an audio
recording program will also do)
The “how” (steps
A, B, C, that lead
to Y or the
activity
outcomes)
K. Facilitators introduce briefly the dynamic of the debate (see
slide 18 here, in Spanish), the issues to be discussed, and the
roles to be taken: moderator/recorder (one of the facilitators),
scribe (writing the discussion’s ideas in a Google Docs),
timekeeper (ensuring that each issue is dedicated 10 min at
most), spokesman (voicing the main group conclusion on each
issue after the debate)
L. Participants are divided into 6-person groups, and given 30
minutes to discuss about the three issues. They are encouraged
to draw and write their ideas in paper (so that they are not
forgotten), while one of the members tries to write them down
in a Google Docs that was previously created by facilitators).
The facilitator in each group keeps a background role, just to
spark up discussion if it seems to dwindle.
M. Once the time is up, facilitators remind about each of the issues,
and ask the spokespeople to provide a short, 1-2 minute
conclusion about that issue by their small group. The Google
Docs documents are left in the workshop’s Moodle for everyone
in the participant community to access.
Activity
outcomes (“Y”)
(positive)
as well as:
 Allowed some (limited) time for teachers to collectively reflect
on the usefulness (or not) of the presented tools/practices
 Gives researchers first insights into the ecological validity of the
proposed tools/practices
 Captures all participants’ opinions in multiple forms (audio,
written), even if there is not time to share all of them with the
whole group
 Serves to elicit new problems, issues, use cases for the tools,
which researchers might not have thought of
5. Reflection

METIS PROJECT
One of the main values of this activity is as an additional data source for evaluation of the
workshops, but also as the seed to begin building a community of practitioners interested (in
this case) in CSCL, which previously were quite isolated (since they often report being the
“rara avis” in their respective departments). Also, it gives the adequate feeling of “closure”
needed at the end of the workshop.

METIS PROJECT
Appendix 5. Personas template
This work by the OU H817 module ““Openness and innovation in elearning”
(http://www3.open.ac.uk/study/postgraduate/course/h817.htm) is licensed under a
Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK: England & Wales
License.
See http://www.ld-grid.org/resources/representations-and-languages/personas for
a short introduction to personas. To use this template, open it at
http://goo.gl/m1Fp6, make a copy and edit.
Name:
Gender:
Age:
Lives in ... with ... Likes ...
Education and
experience
Role and
responsibilities
Technical
skills
Subject
domain skills
and knowledge
Motivation and
desires
Goals and
expectations
Obstacles to
their success
Unique assets

METIS PROJECT
Appendix 6. Survey of user groups
In March 2013 a survey of representatives of the three METIS user groups was
carried out. The full results of the survey are available to project partners at
https://sites.google.com/a/metis-project.org/metis-internal/workpackages/wp3-
workshop-design/task-3-3-development-of-a-draft-structure-for-pilot-workshops.
Findings from the survey include
 That 2 of the 3 user groups expect that their target audience will be prepared
to do work related to the workshop before the workshop occurs, but the
other group will not.
 Representatives of 2 of the 3 user groups think that their target audience
would prefer no more than 2 half-day workshops in the first round
(September 2013 – February 2014). The other thinks that 1 half-day
workshop is the maximum that their target audience will be able to attend.
 All 3 groups can be expected to do some follow up work afterwards, provided
it is focused or linked to certification.
 All 3 representatives believe a blend of face-to-face workshops and online
activities before or after the workshops will best suit their user group.
 One of the representatives stated that certification would be very important
for their target audience; it was less important for the other two groups.
 The representatives thought that a variety of tangible outputs would be
useful for their groups. The range includes electronic and printed handbooks,
draft of learning design as a Word or image file and in a printed form.

Metis project deliverable D3.2: Draft of pilot workshop

More Related Content

Similar to Metis project deliverable D3.2: Draft of pilot workshop

More from Yishay Mor

Recently uploaded

Metis project deliverable D3.2: Draft of pilot workshop