The European Commission’s
science and knowledge service
Joint Research Centre
Exploratory study:
Is eTwinning a PD programme
that proves successful for
schools and learners?
Malta Oct 27 2017
Dr. Riina Vuorikari
DG JRC – Directorate Innovation and Growth
Unit B4 Human Capital and Employment
Focus on the priorities of the
European Commission:
creating research evidence
to support policy-making
Policy neutral and
Independent:
no private, commercial
or national interests
The Joint Research Centre (JRC)
Directorate
Growth &
Innovation
Seville
Who am I?
 Riina Vuorikari, from Finland - I now work in Seville, Spain!
 Research fellow in the JRC since 2013
 2013-2000 in European Schoolnet
as Senior Research Analyst and
Project Manager
 Background: Teacher education in Finland,
Hypemedia studies in Paris,
Doctoral in 2009
 Slideshare: https://www.slideshare.net/vuorikari
 Twitter: https://twitter.com/vuorikari
 https://ec.europa.eu/jrc/en/person/riina-vuorikari
Outline:
1. Part: Context
2. Part: What do we want to study?
3. Part: How to study what we
want to study
4. End discussion
Part 1. Context
Facilitating Innovation in Teaching (17-18)
 Part of JRC’s work to support DG EAC
 This is a small exploratory workpackage (4 person months)
 Goal: to explore if the digital trace data from the eTwinning
platform could be used for furthering the above-mentioned
research goal
 Outcome: set of research ideas and methodological
descriptions on how to use the digital trace data to create
research-based evidence that can support EAC and MS
digital trace data
”records of activity undertaken through
an online information system”
Main idea
Use these digital records as evidence of
activity
and turn them into measures of
theoretically valid and interesting
constructs
TELLNET project
December 1 2009
November 2012
Tellnet
Key Activity 4
'Dissemination and
Exploitation of
Results'
“measures of theoretically
valid and interesting
constructs”,
e.g.
Social Network Analysis;
Social capital
Channels through which
information, ideas and innovation
flow

Part 2. What do we want to study?
Is eTwinning a PD programme
that proves successful for
schools and learners?
Students’
learning outcomes and
skills for life
Teacher professionalism;
New classroom practices,
Self-efficacy, Job
satisfaction
Enrolment to
eT platform
School
Whole school benefiting,
e.g. better learning
environment
Content browsers
…
…
…
Teacher professionalism;
New classroom practices,
Self-efficacy, Job
satisfaction
New
enrollments
How to influence,
nudge and
incentivise?
e.g. training offers,
ambassadors,
workshops,
PD points, labels
Things that
influencing the
participation: e.g.
Complimentary or
substitution?
Curriculum,Socio-
cultural context;
school culture,..
Enrolment to
eT platform
Collaboration
No action
School
Whole school benefiting,
e.g. better learning
environment
Students’
learning outcomes and
skills for life
Part 3. How to study what we want
to study
Step 1: Getting the base-line of what
happens on the platform
sort of
eTwinning Analytics
Question: what theoretical constructs
would be useful?
Content browsers
…
…
…
New
enrollments
Enrolment to
eT platform
Collaboration
No action
School
Learning pathways: specific activities (online, onsite) and learning experiences that
individual eTwinners complete as they progress in their "eTwinning journey".
Question 1: What kind of activities lead to teacher collaboration? Hint, see
TALIS
Question 2: What evidence of other key factors such as content focus; activities with
possibilities for active learning; modelling of effective practices; coaching and expert
support; activities linked to feedback and reflection; sustained duration?
See TALIS; Darling-Hammond et al., 2017
Question 3: How to support engagement and participation?
What kind
of indicators
could be build
to monitor
and measure?
Step 2: To understand the power
of interventions in eTwinning
Question: can we find out what
causes an effect?
Content browsers
…
…
…
New
enrollments
How to influence,
nudge and
incentivise?
e.g. training offers,
ambassadors,
workshops,
PD points, labels
Enrolment to
eT platform
Collaboration
No action
School
Scenario
 CSS and NSS organise many onsite events which are
great for those who participate but also a financial
burden. How can we evaluate their impact on desired
outcomes?
 First: Define what is the “impact” and what is the “desired
outcome”?
 Method: Impact evaluation using counterfactual analysis
to answer cause-and-effect questions
Step 3: To understand if eTwinning
proves successful for
schools and learners
Question: Can we link eTwinning
to learning outcomes?
Content browsers
…
…
…
Teacher professionalism;
New classroom practices,
Self-efficacy, Job
satisfaction
New
enrollments
Enrolment to
eT platform
Collaboration
No action
School
Whole school benefiting,
e.g. better learning
environment
Students’
learning outcomes and
skills for life
Things that
influencing the
participation: e.g.
Complimentary or
substitution? Socio-
cultural context;
curriculum, school
culture,..
Link with learning outcomes?
 What kind of learning outcomes are measured
 In eTwinning
 Outside of eTwinning
 For whom?
 Teachers
 Students
 Whole school
 The design of such study is more complex, however not
impossible 
 Depends on country specific context and availability of
data,
e.g. Lithuanian example earlier today
 It’s a design challenge!
Focus on the priorities of the
European Commission:
creating research evidence
to support policy-making
Policy neutral and
Independent:
no private, commercial
or national interests
The Joint Research Centre (JRC)
Help me to help you!
 Would you see the line of such studies helpful? Where else to
focus?
 Is one of the areas aligned with your goals? Are you already
doing or planning to do something similar?
 I’m interested in any similar studies, contacts and names!
 Can we work together?
 JRC has the methodological expertise and experience, but
 you might have some data and other relevant knowledge and
understanding of the context
Comments, discussion, follow-up?
• Area 1: To getting the base-line of what happens on the
platform (eTwinning Analytics)
• Area 2: To understand the power of interventions in
eTwinning
• Area 3: To understand if eTwinning proves successful for
schools and learners (e.g. link with learning outcomes)

Exploratory study: Is eTwinning a PD programme that proves successful for schools and learners?

  • 1.
    The European Commission’s scienceand knowledge service Joint Research Centre Exploratory study: Is eTwinning a PD programme that proves successful for schools and learners? Malta Oct 27 2017 Dr. Riina Vuorikari DG JRC – Directorate Innovation and Growth Unit B4 Human Capital and Employment
  • 2.
    Focus on thepriorities of the European Commission: creating research evidence to support policy-making Policy neutral and Independent: no private, commercial or national interests The Joint Research Centre (JRC) Directorate Growth & Innovation Seville
  • 3.
    Who am I? Riina Vuorikari, from Finland - I now work in Seville, Spain!  Research fellow in the JRC since 2013  2013-2000 in European Schoolnet as Senior Research Analyst and Project Manager  Background: Teacher education in Finland, Hypemedia studies in Paris, Doctoral in 2009  Slideshare: https://www.slideshare.net/vuorikari  Twitter: https://twitter.com/vuorikari  https://ec.europa.eu/jrc/en/person/riina-vuorikari
  • 4.
    Outline: 1. Part: Context 2.Part: What do we want to study? 3. Part: How to study what we want to study 4. End discussion
  • 5.
  • 6.
    Facilitating Innovation inTeaching (17-18)  Part of JRC’s work to support DG EAC  This is a small exploratory workpackage (4 person months)  Goal: to explore if the digital trace data from the eTwinning platform could be used for furthering the above-mentioned research goal  Outcome: set of research ideas and methodological descriptions on how to use the digital trace data to create research-based evidence that can support EAC and MS
  • 7.
    digital trace data ”recordsof activity undertaken through an online information system” Main idea Use these digital records as evidence of activity and turn them into measures of theoretically valid and interesting constructs
  • 8.
    TELLNET project December 12009 November 2012 Tellnet Key Activity 4 'Dissemination and Exploitation of Results'
  • 12.
    “measures of theoretically validand interesting constructs”, e.g. Social Network Analysis; Social capital
  • 13.
    Channels through which information,ideas and innovation flow 
  • 14.
    Part 2. Whatdo we want to study?
  • 15.
    Is eTwinning aPD programme that proves successful for schools and learners?
  • 16.
    Students’ learning outcomes and skillsfor life Teacher professionalism; New classroom practices, Self-efficacy, Job satisfaction Enrolment to eT platform School Whole school benefiting, e.g. better learning environment
  • 17.
    Content browsers … … … Teacher professionalism; Newclassroom practices, Self-efficacy, Job satisfaction New enrollments How to influence, nudge and incentivise? e.g. training offers, ambassadors, workshops, PD points, labels Things that influencing the participation: e.g. Complimentary or substitution? Curriculum,Socio- cultural context; school culture,.. Enrolment to eT platform Collaboration No action School Whole school benefiting, e.g. better learning environment Students’ learning outcomes and skills for life
  • 18.
    Part 3. Howto study what we want to study
  • 19.
    Step 1: Gettingthe base-line of what happens on the platform sort of eTwinning Analytics Question: what theoretical constructs would be useful?
  • 20.
    Content browsers … … … New enrollments Enrolment to eTplatform Collaboration No action School Learning pathways: specific activities (online, onsite) and learning experiences that individual eTwinners complete as they progress in their "eTwinning journey". Question 1: What kind of activities lead to teacher collaboration? Hint, see TALIS Question 2: What evidence of other key factors such as content focus; activities with possibilities for active learning; modelling of effective practices; coaching and expert support; activities linked to feedback and reflection; sustained duration? See TALIS; Darling-Hammond et al., 2017 Question 3: How to support engagement and participation? What kind of indicators could be build to monitor and measure?
  • 21.
    Step 2: Tounderstand the power of interventions in eTwinning Question: can we find out what causes an effect?
  • 22.
    Content browsers … … … New enrollments How toinfluence, nudge and incentivise? e.g. training offers, ambassadors, workshops, PD points, labels Enrolment to eT platform Collaboration No action School
  • 23.
    Scenario  CSS andNSS organise many onsite events which are great for those who participate but also a financial burden. How can we evaluate their impact on desired outcomes?  First: Define what is the “impact” and what is the “desired outcome”?  Method: Impact evaluation using counterfactual analysis to answer cause-and-effect questions
  • 24.
    Step 3: Tounderstand if eTwinning proves successful for schools and learners Question: Can we link eTwinning to learning outcomes?
  • 25.
    Content browsers … … … Teacher professionalism; Newclassroom practices, Self-efficacy, Job satisfaction New enrollments Enrolment to eT platform Collaboration No action School Whole school benefiting, e.g. better learning environment Students’ learning outcomes and skills for life Things that influencing the participation: e.g. Complimentary or substitution? Socio- cultural context; curriculum, school culture,..
  • 26.
    Link with learningoutcomes?  What kind of learning outcomes are measured  In eTwinning  Outside of eTwinning  For whom?  Teachers  Students  Whole school  The design of such study is more complex, however not impossible   Depends on country specific context and availability of data, e.g. Lithuanian example earlier today  It’s a design challenge!
  • 27.
    Focus on thepriorities of the European Commission: creating research evidence to support policy-making Policy neutral and Independent: no private, commercial or national interests The Joint Research Centre (JRC)
  • 28.
    Help me tohelp you!  Would you see the line of such studies helpful? Where else to focus?  Is one of the areas aligned with your goals? Are you already doing or planning to do something similar?  I’m interested in any similar studies, contacts and names!  Can we work together?  JRC has the methodological expertise and experience, but  you might have some data and other relevant knowledge and understanding of the context
  • 29.
    Comments, discussion, follow-up? •Area 1: To getting the base-line of what happens on the platform (eTwinning Analytics) • Area 2: To understand the power of interventions in eTwinning • Area 3: To understand if eTwinning proves successful for schools and learners (e.g. link with learning outcomes)

Editor's Notes

  • #3 JRC-IPTS: One of the key knowledge providers for DG EAC
  • #13 “A contact of my contact knows a contact of your contact!” By studying the behaviour of social insects such as ants, termites or certain wasps, scientists have elicited three characteristics behind their success in carrying out complex tasks such as building a nest or finding a shortest route to a food source (Bonabeau & Meyer, 2001). These are: !  Self-organisation (activities are neither centrally controlled nor locally supervised); !  Flexibility (the colony can adapt to a changing environment); !  Robustness (even when one or more individuals fail, the group can still perform its tasks). Self-organisation represents the idea that even if individuals follow simple rules, the resulting group behaviour can be surprisingly complex and effective. Self- organisation is explained as “a set of dynamical mechanisms whereby structures appear at the global level of a system from interactions among its lower-level components. The rules specifying these interactions are executed on the basis of purely local information, without reference to the global pattern, which is an emergent property of the system rather than a property imposed upon the system by an external ordering influence.” (Bonabeau, Dorigo & Theraulaz, 1999, p. 9). According to the authors, the four basic ingredients of self-organisation are the following: 1. Positive feedback: simple behavioural “rules of thumb” that promote the creation of structures. An example of this is “recruitment” by ants, i.e. other ants start following a trail to a food source thanks to indirect interactions among insects. 2. Negative feedback is a technique of control and it counterbalances positive feedback. In the example of wayfinding among ants, this can be food source exhaustion, or competition between food sources. 3. Self-organisation (SO) relies on the amplification of fluctuations (e.g. random walks, errors). Randomness is often crucial since it enables the discovery of new solutions. An example of this is an ant that gets lost and finds a new, unexploited food source. 4. Multiple interactions. SO generally requires a minimal density of mutually tolerant individuals who are able to make use of the results of their own activities as well as of others’ activities. E.g. trail networks can self-organise and be used collectively if individuals use others’ pheromone (a chemical substance that can be sensed by other ants).
  • #17 How to influence
  • #18 How to influence
  • #21 How to influence
  • #23 How to influence
  • #26 How to influence
  • #28 JRC-IPTS: One of the key knowledge providers for DG EAC
  • #29 JRC’s aim is to find evidence of how to lever innovation in education so that policy-makers have the right evidence to align separate policy initiatives toward a coherent set of practices to support desired actions.