IMPADA_O3_A1_ Research methodology successful practices for disadvantaged learners

IMPADA
Research methodology to guide IMPADA partners on
collecting successful adult education practices for
the disadvantaged learner
Output type: Intellectual Output
PROMEA
November 2016

2
Project acronym: IMPADA
Project name: Improving the effectiveness of adult education for disadvantaged
groups
Project code: 2015-1-UK01-KA204-013666
Document Information
Document ID name: IMPADA_03A1_good practices collection guide_2016-12-15
Document title: Research methodology to guide IMPADA partners on collecting
successful adult education practices for the disadvantaged learner
Output Type: Intellectual Output
Date of Delivery: 30/11/2016
Activity type: Document
Activity leader: DACES
Dissemination level: Public
Document History
Versions Date Changes Type of change Delivered by
Version 1.0 15/12/2016
Initial
document
- PROMEA
Disclaimer
The EuropeanCommissionsupportforthe production of this publication does not constitute an
endorsementof the contentswhichreflects the views only of the authors, and the Commission
cannot be heldresponsibleforanyuse whichmaybe made of the informationcontainedtherein.
The projectresourcescontainedhereinare publiclyavailable underthe Creative Commonslicense
Attribution-NonCommercial-ShareAlike 4.0 International.

3
Table of Contents
1. Introduction...........................................................................................................................................................4
2. Basic definitions and questions to explore...............................................................................................5
3. Purposes and activities .....................................................................................................................................6
3.1. Why collect the practices?.....................................................................................................................6
3.2. Sharing successful adult education practices for the disadvantaged .................................7
4. Stakeholders ..........................................................................................................................................................7
5. Sampling methodology......................................................................................................................................8
6. Identifying Good Practices.............................................................................................................................10
6.1. Defining evaluation fields ....................................................................................................................11
6.2. Defining evaluation criteria ................................................................................................................11
6.3. Ranking and Classification...................................................................................................................12
7. Collection process, means and tools..........................................................................................................16
7.1. Processes.....................................................................................................................................................16
7.2. Means............................................................................................................................................................17
7.2.1. Questionnaire template...............................................................................................................18
7.2.2. Identity of the organisation where the successfuladult education practice on
the disadvantaged is adopted.......................................................................................................................19
7.2.3. Description of the successful adult education practice on the disadvantaged
learner 20
7.2.4. Impact.................................................................................................................................................22
7.2.5. Evaluations.......................................................................................................................................23
7.2.6. Challenges .........................................................................................................................................24
7.2.7. Scale.....................................................................................................................................................24
7.2.8. Transferability.................................................................................................................................25
7.2.9. Other details of the good practice...........................................................................................26
7.3. Supporting tools.......................................................................................................................................26
7.3.1. Invitation email...............................................................................................................................26
7.3.2. Follow up email...............................................................................................................................27
7.3.3. Thank you email.............................................................................................................................28
8. Quality assurance...............................................................................................................................................28
9. Annexes..................................................................................................................................................................29
9.1. Invitation email template.....................................................................................................................29
9.2. Follow up email template.....................................................................................................................30
9.3. Thank you email template ...................................................................................................................31
9.4. Link to the questionnaire in English ...............................................................................................31
9.5. Template formfor the registration of the invited providers of successful adult
education practices for the disadvantaged learner.................................................................................31

4
1. Introduction
This document describes the methodology used to identify and evaluate successful adult
educationpractices forthe disadvantagedlearnerthatwill be collected within the context of the
IMPADA project and are going to be used for the development of O3A2 “Recommendations for
adult learning providers to improve effectiveness towards the disadvantaged learner”.
The purpose of the research methodology is:
 To guide the practices collection activities of Partners, in their efforts to gather good
practices on successful measures adopted by European adult learning providers, to
improve their effectiveness on the disadvantaged learner.
 To specify the criteria for identifying the best among the collected ones.
Collection of practices will follow a survey format to be promoted by all IMPADA partners to
relevant stakeholders of their network, such as managing and training staff of adult learning
providers, associations of providers, field experts.
The methodology will contribute towards:
 Revealing the objectives for the reporting and the benefits of sharing the practices.
 Illustrating successful practices definitions.
 Pointing out the parameters, metrics and issues on successful adult education practices
for the disadvantaged learner in the European Adult Education Providers.
 Providing IMPADA Partners the necessary guidelines and practical material for the
reporting,monitoringandcoordinationof the collectionof the successfuladulteducation
practices for the disadvantaged learner.
 Identifying the actions that the IMPADA partners should undertake for the collection of
successful adult education practices for the disadvantaged learner.
Under thispremise, successfuladulteducationpractices on the disadvantaged learner should be
reviewed within a specific evaluation scheme in terms of their effectiveness, applicability and
transferability. The proposed methodology also:
a) defines certain evaluation fields and criteria (i.e. impact, previous evaluations, challenges
encountered in implementation, extent of integration throughout the organisation, and
transferability), and
b) sets score thresholds and weights in order to ensure a minimum quality level and results,
graded according to the weighted significance of each evaluation field. (N.B. This evaluation /
ranking process should be used in order to filter case studies that are going to be included in
O3A2 “Recommendations for adult learning providers to improve effectiveness towards
disadvantaged learner”.

5
2. Basic definitions and questions to explore
A practice is defined as a procedure or a technique that has been tried and shown to work
(whether fully or in part but with at least some evidence of effectiveness), and that may have
implications forpractice at anylevel elsewhere.Inthese contexts,three possible levelsof practice
stem:
 Promising practices have been used for a certain period of time by an organization in
orderto betterperformitstasksand to increase its productivity. The promising practices
have not proventheirtransferability to different geographic or thematic areas, nor to be
successful according to any result indicators.
 Demonstrated practices have proven to be successful for the organization where they
have been applied. The success of the practices is confirmed through some result
indicators, but it is not demonstrated how and to which area these practices can be
transferred.
 Replicatedpracticeshave the potential undercertaincircumstances and conditions to be
transferred and successfully adopted to a different geographic or thematic area.
The following set of criteria should indicate whether a practice is a “good practice”:
 Effective and successful:A “good practice”has provenitsstrategic relevance as the most
effective way in achieving a specific objective; it has been successfully adopted and has
had a positive impact on individuals and/or communities.
 Economically and socially sustainable: A “good practice” meets current needs, in
particularthe essential needsof the disadvantaged, without compromising the ability to
address future needs.
 Gendersensitive:A descriptionof the practice mustshow how actors (men and women),
involved in the process, were able to improve their livelihoods.
 Technically feasible: Technical feasibility is the basis of a “good practice”. It is easy to
learn and to implement.
 Inherently participatory: Participatory approaches are essential as they support a joint
sense of ownership of decisions and actions.
 Replicable and adaptable: A “good practice” should have the potential for replication
and should therefore be adaptable to similar objectives in varying situations.
 Reducing disaster/crisis risks, if applicable: A “good practice” contributes to
disaster/crisis risks reduction for resilience.
Withinthe IMPADA project,a goodpractice refersto a successful adulteducation practice for the
disadvantagedlearnerthathasbeentriedbya EuropeanAdultEducationProviderandhasproved
to solve – most effectively – a problem in the organization.
Considering the aforementioned definitions several questions are arising, such as:
 Why should the practices be collected?
 Which are the benefits from sharing the practices?

6
 Who are the stakeholders of successful adult education practices for disadvantaged
learner?
 Whichis the populationtobe investigated?How isthe sample defined and how it will be
reached?
 How the sample will be motivated for the reporting of successful adult education
practicesfor the disadvantagedlearner,andwhichare the benefitsfromthis motivation?
 Whichare the factors and the elementsthat they should be explored and revealed from
the reporting of practices?
 How the requested data will be collected? How will they be analysed?
 What actions and tasks should each IMPADA partner undertake in order to achieve the
desired results?
This document aims to guide the practices’ collection activities of partners, and to specify the
criteria for identifying the best among those collected, by providing the answers to the above
questions and thus, ensuring the collection of the successful adult education practices for the
disadvantaged learner with response to the needs and requirements of the IMPADA project.
3. Purposes and activities
3.1. Why collect the practices?
The aims of the collectionof successful adulteducationpractices forthe disadvantagedlearnerby
adult education providers in the Partnership countries are:
• To identify the success factors and the criteria under which successful adult education
practices for the disadvantaged learner are assessed as good practices.
• Acquiring knowledge on specific relevant issues and assuring homogeneity and
comparabilityof the gatheredpractices for the identification of the key success factors
and the transferability potential of the collected evidence.
• To provide input for the development of IMPADA O3A2 “Recommendations for
improvements on effectiveness of adult education for disadvantaged learner”.
• To contribute to the catalogue of successful adult education practices for the
disadvantaged learner.
• To disseminateestablished successful adult education practices for the disadvantaged
learner.
In this context adult learning providers in need of greater improvements will gain insight on
practices of others withgreatereffectivenesson the disadvantagedlearner,whilehighlyeffective
adult learning providers will benefit from comparing their practices with practices from other
European countries and will be motivated to devise measures to further improve their
effectiveness towards the disadvantaged learner.

7
3.2. Sharing successful adult education practices for the disadvantaged
Identifying and sharing an organization's successful adult education practices for the
disadvantaged learner helps staff members and educators to learn from each other and reuse
provenpractices.Transferringthe practicesfromone adulteducationprovidertoanotherleadsto
the improvement of its effectiveness towards the disadvantaged learner, and to better quality
results and impact on learners.
The immediate benefits from effective sharing are the following:
• Transferring know-how to other adult education providers, which are less experienced in
working with disadvantaged learners
• Avoidance of replication of effort or “reinventing the wheel”
• Replication of successes throughout an organization, and also from one adult education
provider to another
• Self-improvementof adult education providers which have adopted successful practices on
the disadvantagedlearner, through the exchange of experience with other adult education
providers and the education community
Benefits can also be identified on a macro level horizon:
• Increase of the overall quality of services of adult education providers
• Minimization of organizational knowledge loss (both tacit and explicit)
• Decrease of the learning curve of new educators/recruiters
• Promotion of dissemination of the results, stimulation of the communication and the
coordination among the stakeholders, animation of the debate between the interested
parties
• Contributiontothe development of policy concerning inclusive education by the European
and national policy-makers
4. Stakeholders
The identificationof stakeholders isof crucial importance for the quality of the collected data. In
the context of the IMPADA project, the practitioners of successful adult education practices for
the disadvantagedlearnerare those personswhowill transfertheir factual knowledge related to
a particular case of adoption of a successful practice, by reporting the practice of their
organization and disseminating it to other interested EU adult education providers.
For the IMPADA project, stakeholders adopting successful adult education practices for the
disadvantaged learner are:
 Adult Education centres
 Life-long learning centres
 Vocational training centres

8
 FE and skills providers
 Adult Education Associations
 Educational research institutes
 Educational policy making organizations
 Educational and training National/EU authorities
 Associations of disadvantaged groups
The above should be motivated to participate by becoming aware of the importance, goals and
objectives of the IMPADA project. Furthermore, they should know the reasons why they are
chosen to report their practice (e.g. because it is well known that their organization uses a
successful inclusive practice, and thus, as experts, they are requested to share their factual
knowledge for that specific practice). Stakeholders should be also motivated by being well
informed of the benefits from the collection of successful adult education practices on the
disadvantaged (i.e. sharingpracticeswithanyotherinterested party, the foreseen identification
of at least 20 good practices on successful measures to improve adult learning providers’
effectiveness on the disadvantaged learner, and the fact that these good practices will provide
input for the development of the IMPADA O3A2 “Recommendations for improvements on
effectiveness of adult education for the disadvantaged learner”).
The motivation of adult education providers, research bodies and adult education associations
will alsobenefitthe IMPADA project,asrelevantstakeholders adopting or developing successful
adult education practices on the disadvantaged learner will be willing to share their practice,
assuringthe qualityof the producedevidence.Besides,theirinterestand– hopefully- enthusiasm
for the IMPADA projectcan alsoleadto fulfilling furtherproject’sneedsinalatertime,whentheir
contribution will be requested again. Moreover, possible new players in the field of adult
education maybe foundwhocan play a crucial role inthe exchange of experience among the EU-
wide strong supporters of inclusive adult education.
Moreover,disseminating collectedevidence amongstakeholders canleadtothe replicationof the
successful practicesinthe organizationsof the recipients,by adopting already tested and proven
ones. Furthermore, disseminating collected practices could attract stakeholders such as
associations for the disadvantaged learner that may have interest in investing resources for the
improvement of a specific practice. Finally, propagating to policy makers successful adult
education practices for the disadvantaged learner could facilitate up-scaling relevant EU and/or
national policies.
5. Sampling methodology
The suggestedsamplingmethodologyforthe collectionof successful adulteducationpracticesfor
the disadvantaged learner is “quota sampling”, which is the non-probability equivalent of
stratified sampling. Quota sampling emerges as an attractive choice when time is an issue, as
primary data collection can be achieved in a cost and time saving manner. Moreover the lack of
an exhaustive and accurate sample-frame across all consortium countries further justifies this
choice.

9
In quota sampling, a population is first segmented into mutually exclusive clusters, and then
purposive1
(orjudgment) samplingisusedtoselectthe subjectsorunits from each cluster, based
on a specified proportion. For the purposes of this research, Partners have been foreseen to
gatherat least10 practiceseach; therefore itissuggestedthatatleast40 unitsshould be sampled
by each Partner.
The population for the collection of successful adult education practices for the disadvantaged
learner are the executives and managerial staff of adult education providers (including LLL and
VET), educational researchorganisations, andpolicyrelevantbodies of the consortium countries.
The mutually exclusive clusters for the aforementioned are: adult education providers, adult
education associations, adult education research bodies and adult education related policy
making bodies. Partners will target specific clusters according to their profile, capacities and
relevance, and will accordingly judge which units should be respectively sampled from each
cluster. Table 1 below presentsthe type of organization forthe IMPADA consortiumPartners, and
the suggested clustertoretrieve successful sample-level information. (N.B. project Partners are
advisedtoinvite atthe reportingprocessthose stakeholders who are most appropriate and easy
to reach).
IMPADA
partner
Type of organisation Suggested target cluster
DACES
(UK)
AdultEducation
communitycentre
 adulteducation Providersatnational level
 educational researchorganisations
 policyrelevantbodiesatnational level
 associationsof disadvantagedgroups
IBE (PL) National educational
researchinstitute
 policyrelevantbodiesatnational level
UPTER
(IT)
Private AdultEducation
provider
 adultEducationAssociationsat national level
ENAEA
(EE)
AdultEducation
Association
 adulteducation providersatEU level
 adultEducationAssociationsatEU level
PROMEA
(GR)
Private non-profit
researchorganization
 Educational researchorganisations
 Policyrelevantbodiesatnational level
1Purposivesampling is a non-probability sampling methodwhich,nonetheless, is superior tosimpleconveniencesampling,and where
elements selected for thesamplearechosenby thejudgmentoftheresearcher. It is argued that researchers can obtain a
representative sampleby using a sound judgment, which willresult in saving timeand money. Therefore, given theprofiles and
competences of Partners purposivesampling can safelybe employed.

10
Table 1 Suggested target cluster per IMPADA Partner
6. Identifying Good Practices
Thischapter describesthe methodsusedforthe identification,validation and evaluation of good
practices on adult education provision for the DISADVANTAGED LEARNER. It describes the
evaluation fields and criteria used to filter case studies and define the quality levels of a good
practice.The methodologydescribedisstructuredandappliedinthe following, indicative stages:
1. Define evaluation fields.
2. Define evaluation criteria.
3. Set score levels, weights and thresholds.
4. Define ranking groups / classification categories
Figure1 Structure of the research methodology
Define evaluation
fields.
Define evaluation
criteria.
Set score levels,
weights and
thresholds.
Define ranking
groups /
classification
categories

11
6.1. Defining evaluation fields
Implemented adult education
practices for the disadvantaged
are going to be examined and
evaluatedwithinspecific fields in
order for the IMPADA project to
assess their appropriateness as
good practices, and classify them
accordingly. These fields are also
taking into account the
specificities and objectives of the
IMPADA project and address a
series of issues and aspects that
differentiatevariousimplemented
practices and could also be
phrased as questions (see table
2).
By defining certain fields of
investigation, amore focused and
clearly structured evaluation
process is expected to emerge, targeting specific aspects and factors that differentiate adult
education practices for the
disadvantagedlearner,intermsof
effectiveness and transferability.
6.2. Defining evaluation criteria
The evaluation criteria to be used in order to differentiate and rank practices correspond to the
aforementioned evaluation fields and are set and justified as shown below:
1. Impact.
Assess the impact of an implemented practice according to the number of CAF criteria it
addresses.
2. Previous evaluations
The type and level of anypreviousevaluation (e.g. internal, external, preliminary, systematic) is
an importantfactor.Practices that have been already validated or evaluated to some extent are
rather more promising than others.
3. Challenges in implementation
Possible problemsencountered,difficulties met and lessons learned in implementing a practice
are issues that any stakeholder needs to know in order to assess a practice. If encountere d
problems rose to a prohibiting level or too much time was consumed in trouble-shooting, then
the practice should not be considered successful.
4. Scale of practice integration across organisation departments
Fields Questions
Impact Whichof the CAF keyaspectshas the good
practice shown to be most effective at?
Evaluations Has an implemented practice ever been
validated or evaluated, either by internal
or external evaluators?
Challenges To what extent have problems or
difficulties challenged implementation?
Scale The extent to which the practice been
integrated across the organisational
structure.
Transferability Is the practice customisable andadaptable
for use across disadvantaged learners
groups?
Table 2Fields and Questions

12
Identifyingthe extentof adoptionfor a particular practice within an organisation is an important
aspect. Practices involving higher numbers of individuals or adopted within entire sections or
departments within an organisation are more likely to increase their success potential and
transferability value.
5. Transferability potential
A practice’s potential or proven record of being successfully transferred to different
disadvantaged groups and organisational settings is one of the most important criteria for
identifying it as a good or a promising practice. Practices that demonstrate strong evidence of
transferability should be evaluated higher than others.
Evaluation fields and criteria are linked and structured as shown in the table 3.
Good practices evaluation fields and criteria
Evaluation fields Evaluation Criteria Objectives
Impact Numberof criteria Assess how the practice addressesCAF
identifiedcriteria
Evaluations Type of previously
appliedevaluation
Identifythe type and extentof anyprevious
validationandevaluation(e.g. internal,external,
preliminary,systematic)
Challenges Type and extentof
problemsencountered
inimplementation
Assessthe extentof encounteredproblemsand
difficultiesthathave hinderedapractice’s
implementation,andjudge whetherthis
challenges are inherentinthe goodpractice or
not
Scale Involvedindividualsor
organisational
departments
Identifythe extenttowhichthe practice been
integratedacrossthe organisational structure
Transferability Level of transferability Identifythe extent of transferabilityof the
practice across variousdisadvantagedgroups
Table 3 Good Practice evaluation fields and criteria
6.3. Ranking and Classification
Practicescollected throughIMPADA partners’effortsinthe contextof O3A1and alsoO4A1 will be
ranked and classified in the following indicative stages:
1. Practices are examined in relation to the predefined evaluation fields and criteria.
2. Practices are given scores according to the “IMPADA Good Practice Evaluation Chart”.
3. Practices are grouped on an eligible / non eligible basis.
4. Practices are ranked in predefined ranking groups and classified accordingly.

13
Figure 2 the process of practices’ ranking and classification
The IMPADA Good Practice evaluation chart (table 4) is proposed as a tool to facilitate and
standardize practice evaluation and classification. As shown in the evaluation chart below, each
collected practice could get a score ranging from 1 to 5 for each evaluation field. A score weight
and a score thresholdhave beenassignedtoeachevaluationfieldand criterioninorderto ensure
a minimal quality level and a graded output of results.
Fields Criteria/Score
1 2 3 4 5
Impact2
WEIGHT: 1.5
Matches no
CAF criteria
THRESHOLD:
Matches 1
CAF criterion
Matches 2-3
CAF criteria
Matches 4-6
CAF criteria
Matches 6 or
more CAF
criteria
Evaluations 3
WEIGHT: 1
Practicehas
never been
evaluated,
either
internally or
externally.
THRESHOLD:
Practicehas
been
evaluated on
a preliminary /
non
standardised
stage.
Practicehas
been internally
evaluated.
Practicehas
been
externally
evaluated.
Practicehas
been system-
atically evalu-
ated by both
internal and
external
evaluators.
2 Refer to questionnaire section3
Practices are examined
in relationto the
predefined evaluation
fields andcriteria.
Practices are given
scores according to the
“IMPADA Good Practice
EvaluationChart”.
Practices are groupedon
an eligible / non eligible
basis.
Practices are rankedin
predefined ranking
groups and classified
accordingly.

14
Fields Criteria/Score
1 2 3 4 5
Challenges4
WEIGHT: 1.5
[note:
subjective
judgementwill
be usedto
determine
whetherornot
identified
challengesare
inherentand
therefore
couldaffect
implementatio
n]
Had serious
challenges and
difficultiesthat
significantly
impacted its
results
THRESHOLD:
had some
challenges
and difficulties
that hindered
its
implementati
on
Had only minor
challenges and
difficulties.
Had almostno
problems and
difficulties.
Had no prob-
lems or diffi-
culties whatso-
ever.
Scale5
WEIGHT: 1
THRESHOLD:
Practicehas
been only im-
plemented
within a limited
group of
people.
Practicehas
been only im-
plemented
within a small
department.
Practicehas
been imple-
mented in an
organisational
department
involvingthe
entire depart-
ment’s staff.
Practicehas
been imple-
mented in the
entire organi-
sation involv-
ingmost of its
staff.
Practicehas
been imple-
mented in the
entire organi-
sation involving
its entire staff.
Transferability
6
WEIGHT: 1.5
The good
practicecannot
be used with
other learners
THRESHOLD:
The good
practicecan
only be used
with this type
of learner in
similar
contexts.
The good
practicecan be
customised and
adapted so it
can be used
with a larger
number of
learners
The good
practicecan
be customised
and adapted
so it can be
used with
learners with
a wider range
of
disadvantaged
/to address
other issues of
inclusivity
The good
practiceis
universal and
customisable
and can be
used with any
learner
Table 4 IMPADA good practiceevaluation chart
Accordingto the IMPADA Good Practice EvaluationChartpossible score levels for both eligibility
and non-eligibility scenarios are configured as shown in the table 5.
6 S Refer to questionnaire section 7

15
Critical area
0 5 10 15 20 25 30
level 1
level2
level 3
level 1
level2
level 3
level 1level2level 3
range minimum score 20.66.60
range maximum score 32.520.56.5
Score results are calculated using the following formulae:
Practice score for each field=score x fieldweight.Overall practice score = Sum of all field scores.
N.B. To achieve a “good practice status”, a practice should score no less than the threshold set
for the “Impact” and “Challenges” fields.
According to this score grading, 3 ranking levels are defined as shown in the tables 5 & 6.
Ranking
level
Score Evaluation Status Explanation
Level 1 >20.5 – 32.5 Verygood Eligible Highqualitylevel.Practice
demonstratesproven
effectiveness, transferability,and
relevance toCAF.
Level 2 >6.5 – 20.5 Critical Eligible,if
threshold met
for “Impact”
and
“Challenges”
If score ≥12, and practice has met
“Impact” and “Challenges”
thresholds thenpractice can be
consideredeligible.
Level 3 0-6.5 Notgood NonEligible Non-satisfactoryqualitylevel.
Shouldnotbe consideredagood
practice.
Table 5 IMPADA good practiceranking levels
Practices are
ranked according
to a predefined
setof three levels
on a 1-3 scale.
Practicesclassified
in levels 1 (high
quality level) and
2 (good / eligible
quality level) can
be considered
good practices.
Practicesclassified
in level 3 should
be considered non-
eligible. It is
expectedthat this ranking scheme will ensure a certain degree of consistency in evaluating and
classifying practices also allowing for updated inputs and comparative analysis of additional
practices to be collected.
Table 6 Eligibilityand non-eligibility ranges

16
7. Collection process, means and tools
The collection of practices is of crucial importance for the IMPADA project, as the evidence
collected will be edited and analysed with the aim to contribute to the development of the
catalogue withcase studies,whichwill be available for interested parties and stakeholders, and
will also provide input for the development of the recommendations for improvements on
effectiveness of adult education for the disadvantaged learner (O3A2).
This chapter describes the actions that IMPADA partners should undertake for the collection of
the anticipatedevidence,atime planforthe collectionprocess,the meanswithwhichdatacan be
collected and the tools towards that purpose.
7.1. Processes
Collection of practices will follow a survey format to be promoted by all IMPADA partners to
relevantstakeholdersof theirnetwork.The targetwillbe tocollectatleast10 practicesfrom each
of the partners' networks. In addition to partners own efforts, the survey will be promoted
through the IMPADA network to be developed in O4-A1. Interested parties will fill-in the
respective form and submit their responses. After the completion of the survey, PROMEA will
gather and evaluate all responses collected, to select at least 20 good practices on successful
measures to improve adult learning providers’ effectiveness on disadvantaged learner.
More specifically:
1. Each partner will translate in own language the survey developed in this document (see
Tables 8- 15 for development guidelines, and the link to an online form of the
questionnaire)
2. PROMEA will create the template online & offline surveys
3. All Partners should identify the potential practices’ implementers, according to the
information of chapters 4 “Stakeholders”, and 5 “Target population and sample” of this
document.
4. All Partners should register the selected invited stakeholders in the template form of
annex 9.5 “Template formforthe registrationof the invited providers of successful adult
education practices for the disadvantaged learner” and send it to PROMEA.
5. All Partners should send inviting emails according to the instructions given in chapter
7.3.1 “Invitation email” and to the template email of the annex 9.1.
6. All Partnersshouldsendfollowupemailsbasedonthe instructions given in chapter 7.3.2
“Follow up email” and to the template email of the annex 9.2.
7. Stakeholders should report their practices based on the template form, developed in
chapter 7.2.1 “Questionnaire”.
8. All IMPADA Partners should send a thank you email to those who have responded
according to the instructions given in chapter 7.3.3 “Thank you email” and to the
template email of the annex 9.3 ”Thank you email template”.
9. Each IMPADA Partner is responsible to collect at least 10 practices on effective adult
education provision for the disadvantaged learner.
10. DACES assisted by PROMEA has the overall responsibility for the monitoring of the
process of the collection of practices and it should perform the following actions:

17
a. Check if the desired number of practices is reported.
b. Notify Partners which have not promptly provided practices.
11. All IMPADA Partners should have sent by March 3rd
2017 at least 10 practices each to
PROMEA
The following table contains a summative description of the responsibilities of each IMPADA
partner and the deadlines for the collection of the requested evidence.
IMPADA
partner
Action Deadline
PROMEA Release of methodologyandtools By January 30th 2017
PROMEA Developmentof the final online versionof the
questionnaire
By January 30th 2017
All Partners Translationsof the questionnaire By February 7th 2017
All IMPADA
partners
Identificationof relevantstakeholdersand adult
education providersandregistrationof identified
invitees.(Sendthe formwiththe identifiedinviteesto
PROMEA)
By February 7th 2017
All IMADA
partners
Sendinginvitationemail withthe questionnaire tothe
providers
8 – 13 February 2017
All IMPADA
partners
Sendingfollow-upemailtothe providers,whenitis
appropriate
15– 20 February 2017
All IMPADA
partners
Sendto PROMEA collectedpractices 13 -24 February 2017
All IMPADA
partners
Sendingthankyouemail tothe providers 27-28 February 2017
PROMEA Checkingif questionnaires are beingfilledin 8 – 24 February 2017
PROMEA Administeringandmonitoringthe collectionprocess Throughoutthe whole
collectionprocess
Table 5 Timeline of activities for collectingpractices
7.2. Means
There are two means for collecting the practices from the providers: an online survey (Google
form) and a questionnaire to be distributed through emails.
Reporting successful adult education practices for the disadvantaged learner in a Google form
makes the practice available to everyone who is interested in this area and has the link to the
survey, and the content of the survey can be more easily integrated in the IMPADA platform for
the exchange of goodpractices. Furthermore,awebinterface guarantees the confidentiality and
the safeguarding of the information and it does not invoke any intermediary; therefore the

18
contentcannotbe alteredorchanged.Moreover,reportingpracticesinaGoogle formwebisfast,
simple and flexible.
Alternatively, adult education practices for the disadvantaged learner can be sent in a
questionnaire form, as an attachment to the inviting email which would have to be sent back
filled in. This procedure is simple, but it involves intermediaries or additional persons, thus
mitigating the confidentiality of information gathered and it could possibly increase the effort
needed for the collection of practices (e.g. due to tracking emails back and forth).
The Google form should be user friendly, robust and should guarantee the safety and the
confidentialityof the personal data. PROMEA will develop the questionnaire template in English
and all Partners will then develop translated in own languages. The questionnaire needs to
include an explanatory, introductive text so as to provide to the target groups of the
questionnaire, guidelines and tips for the web site login, the uploading of the practice and the
fillinginof the reportingform (thisshouldbe translatedtoall Partners’ languages). Furthermore,
a counterwill track the numberof personsthat have uploadedtheirpractice.Finally, people filling
inthe questionnaire shouldbe prompted -throughamessage- incase theyhave notcompletedall
requested fields.
7.2.1. Questionnaire template
It isproposedtouse the following template forcreatingthe questionnaire form for the reporting
of adult education practices for the disadvantaged learner.
The questionnaire template is designed to consist of the following thematic categories:
1. Personal and organizational information Description of the practice
2. Description of the successful adult education practice on the disadvantaged learner
3. Impact
4. Evaluations
5. Challenges
6. Scale
7. Transferability issues
8. Details of good practice
Thisdivisionfacilitates the filling in of the questionnaire from the provider and the analysis and
editing of the collected data. Each thematic category contains a table, where the questions are
listed in and which consists of the following fields:
1. The 1st field has the indexing code of the question. The questions are indexed so as to
permitthe quickreference andlocalizationincase anyclarificationsare requested by the
providers.
2. The 2nd field contains the question.
3. In the 3rd field there are instructions for the questionnaire respondent. It is suggested
that these instructions could be included in the “description” field of the online
questionnaire
4. In the 4th fieldthere are some suggestionsforthe developerof the online questionnaire,
regarding the html format the questions should have (not to be included as a distinct
column/field in the actual questionnaire form).
The 1st page of the questionnaire template is an introductory page, which describes shortly the
IMPADA project, the aims and the target groups of the questionnaire, the benefits for the
providers of practices and the total time needed for the filling in.

19
The suggested questionnaire template is the following:
7.2.2. Identity of the organisation where the successful adult education practice on
the disadvantaged is adopted
Index Question Help, instructions for the
questionnaire respondent
Specification/suggested
HTML form forthe
developerofthe online
questionnaire
1.1 Name of the
organisation in English
Short answer
1.2 Country:
a) UK
b) Poland
c) Italy
d) Estonia
e) Greece
f) Other
Please select the country or the
countries where the practice is
adopted.
Multiple choice
1.3 Type of organisation:
a) Adult Education
centres
b) Life-long
learningcentres
c) Vocational
trainingcentres
d) FE and Skills
providers
e) Adult Education
Associations
f) Educational
research
institutes
g) Educational
policies making
organizations
h) Educational and
training
National/EU
authorities
i) Associationsof
disadvantaged
learner
j) Other
Please select the type of the
organisation where the practice is
adopted.
Multiple choice
1.4 Name and job title of
person completing the
questionnairein
English
Short answer

20
Index Question Help, instructions for the
HTML form forthe
questionnaire
1.5 Estimated number of
adultlearners within
the organisation:
a) 1 – 50
b) 51 – 100
c) 101 – 500
d) 500 -1000
e) 1000 – 5000
f) 5000+
The answers should bein a drop
down menu format.
1.6 Number of campuses
– local/national
branches:
a) 1
b) 2-5
c) 5-10
d) 10+
The answers should bein a drop
down menu format.
Table 6 organisational info
7.2.3. Description of the successful adult education practice on the disadvantaged
learner
Index Question Help, instruction for the
Specification/ suggested
HTML form for the
developer of the online
questionnaire
2.1 Brief description (and
title if applicable) of the
adopted practice
Required Short answer. 100 words max
2.2.a Does the good practice
address teachingand
learning?
Required Yes/no
2.2.b Which of the following
leadership and
management aspects
does the good practice
address?
a) Inclusivestrategy
development
b) Inclusiveand
targeted planning
and recruitment
c) Quality
improvement
d) Learner voice
e) None
Please identify the area that the
practice addresses. Tick as many as
apply. In case of “other” please
specify.
Check boxes

21
Specification/ suggested
HTML form for the
developer of the online
questionnaire
2.2.c Which the following
learner outcomes
aspects does the good
practiceaddress?
a) Skills and
qualifications
b) Socio-economic
benefits of
learning
c) Destinations
d) None
Please identify the area that the
practice addresses. Tick as many as
apply. In case of “other”, please
specify.
Multiple choice
2.3 Current state of the
good practiceis
a) Under
consideration
b) Under
development
c) In pilotphase
d) Operational
e) Discontinued
partially
Discontinued totally
Multiple choice
2.4 Who/what prompted
your organisation to
develop this good
practice?
a) external
inspection
b) quality
assurance
c) Internal quality
assurance
procedures,
such as
Observation of
Teaching and
Learning
d) Learner
feedback
e) Partner
feedback
f) Stakeholder
feedback
g) Employer
feedback
h) Funding policy
and targets
i) Other
Multiple choice
2.5 When was the good
practiceimplemented?
Please indicate the year when the
practice was first adopted
Short text
Table 7 Descriptionof the practice

22
7.2.4. Impact
Index Question Help,instructionfor the
HTML form forthe
questionnaire
3.1 Which of the following
leadership and
management aspects has
the good practicebeen
shown to have positive
impactupon? (tick as
many as apply)
a) inclusivestrategy
development
b) strategic
partnerships with
stakeholders
c) strategic policy
for disadvantaged
learners
d) inclusive
curriculum
development
e) promotion to
disadvantaged
learners
f) inclusive
recruitment
strategy
g) continuous
professional
development
(CPD)/staff
trainingfor
working with
disadvantaged
learners
h) Monitoringand
improving
teaching, learning
and assessment
i) Usingfeedback to
effect change and
informplanning
j) None
k) Other
Required Check boxes
teaching, learningand
assessmentaspects has
the good practicebeen
shown to have positive
impactupon ? (tick as
many as apply)
a. Inclusiveinitial
assessment
Required check boxes

23
Index Question Help,instructionfor the
HTML form forthe
questionnaire
b. Inclusivediagnostic
assessment
c. Inclusivelearning
outcomes
d. Evidence-based
pedagogy/andragogy
e. Differentiated
teaching, learningand
assessmentto meet
needs
f. Inclusivesummative
assessment
g. None
h. Other
learner outcomes aspects
has the good practice
been shown to have
positiveimpactupon? (tick
as many as apply)
a. Narrowingthe
achievement gap for
disadvantaged
learners
b. Impact of ‘soft skills’
c. Positivedestinations
and progression for
disadvantaged
learners
d. None
e. Other
Required Check boxes
Table 8 Impact
7.2.5. Evaluations
Specification/
suggested HTML
form for the
developer of the
online
questionnaire
4.1 Has this good practicebeen
evaluated?
a) Practicehas never
been evaluated, either
internally or externally
b) Practicehas been
evaluated on a
preliminary / non
standardised stage
Required Multiple choice

24
c) Practicehas been
internally evaluated
d) Practicehas been
externally evaluated
e) Practicehas been
systematically
evaluated by both
internal and external
evaluators
Table 9 Evaluations
7.2.6. Challenges
Specification/
suggested HTML
form for the
developer of the
online
questionnaire
5.1 Practical implementation of this
good practice:
a) had serious challenges and
difficulties thatsignificantly
impacted its results.
b) had some challenges and
difficulties thathindered its
implementation.
c) had only occasional
challenges and difficulties
that have not hindered its
implementation.
d) had only minor challenges
and difficulties.
e) had no notable challenges
or difficulties
Tick as many as apply. Multiple choice
5.2 Pleasedetail the challenges
encountered in implementation (up
to 150 words)
. Short answer
Table 10 Challenges
7.2.7. Scale
Specification/
suggested HTML
form for the
developer of the
online
questionnaire
6.1 To what extent has the practice
been integrated across the
organisational structure?
a) Practice has been
implemented within a
limited group of people.
Select which best applies Multiple choice

25
Specification/
suggested HTML
form for the
developer of the
online
questionnaire
b) Practice has been
implemented within a small
department.
c) Practice has been
implemented in an
organisational department
involving the entire
department’s staff.
d) Practice has been
implemented in the entire
organisation involving most
of its staff.
e) Practice has been
implemented in the entire
organisation involving its
entire staff.
Table 11 Scale
7.2.8. Transferability
Specification/
suggested HTML
form for the
developer of the
online
questionnaire
7.1 Select which is the most accuratefor
this good practice
a) The good practice cannot
be used with other learners
b) The good practice can only
be used with this type of
learner in similar contexts
c) The good practice can be
customised and adapted so
it can be used with a larger
number of learners
d) The good practice can be
customised and adapted so
it can be used with learners
with a wider range of
disadvantage/to address
other issues of inclusivity
e) The good practice is
universal and customisable
and can be used with any
learner
Identify the extent to which this
practice is customisable and
adaptable for use with
disadvantaged learner
Multiple choice
Table 12 Transferability

26
7.2.9. Other details of the good practice
Specification/
suggested HTML
form for the
developer of the
online
questionnaire
8.1 Please provide a description of the
good practice (max 250 words)
Paragraph
8.2 Specify some conclusions or main
lessons learnt (up to 3) from the
adoption of the practice (max 250
words)
Paragraph
8.3 URL for downloading any additional
documents, resources or other
helpful data
Please provide also a short text
description to describe what is
provided with URL
Short text
Table 13 Other details
7.3. Supporting tools
7.3.1. Invitation email
An explanatory text along with the inviting email helps increasing the credibility of the
questionnaire by granting legitimacy and strength to it, gaining at the same time the invitee’s
interest and trust. Therefore, the inviting email should include:
1. A brief description of the purpose of the questionnaire.
A brief butmeaningfultextisnecessarytointroduce the purpose the questionnaire serves. Say a
fewthingsaboutthat purpose,e.g. the IMPADA ProjectisaEuropeanco-funded projectthataims
at improvingthe systematicevaluationof the effectiveness of adult education provision towards
the disadvantaged learner with a view to reduce disparities in and enhance access to adult
learning.
2. The reason why the person receiving the email was chosen.
In the context of gaining the recipients’ trust it is advised to explain them the reason they were
chosento participate.Use the available informationaboutthe potential provider of the practice,
their relevance to adult education, type of organisation and so on to develop a convincing
argument about their selection. (e.g. Dear [salutation, surname], we would like to invite you to
report an adult education practice for the disadvantaged adopted in. Your experience and
involvement in adult education are very much aligned with the target group expected
characteristics for the reporting of successful adult education practices for the disadvantaged
learner, and we would thus highly appreciate your valuable contribution in our research.)
2. The importance of the collection and sharing of practices/ the reason(s) why the
recipient should complete the questionnaire.
Motivate the recipientsandmake themunderstandhow andwhytheirparticipationwill promote
science,knowledgeoranyothernoble purpose.Staytothe facts and do nottry to make it heavier

27
than itis.Not all recipientsare experts but they do have some knowledge about this issue. (i.e.
“the collectionof successful adulteducationpracticesonthe disadvantagedlearner will facilitate
the developmentof the recommendationsforimprovement on effectiveness of adult education
for disadvantaged learner”, is an explanation that people will understand and accept. “The
collection of practices will help us bring a new era in the field of adult education” is somewhat
farfetched and could be judged negatively.)
4. Explanation of how the reporting of practices will be used.
The participantswill wanttoknowhowthe informationtheyreveal will be treated, regardless of
privacy and consent statements. Spend a few lines explaining how the practices will help
providingwithguidance anduseful instructions tothe interested stakeholders.If the practices will
become publicly available let them know it and promise to keep them updated about the
outcomes: the IMPADA network of adopters and repository of results.
5. The link to the questionnaire
“Our questionnaire canbe foundinthe projects’officialpage” is not the right indication. Provide
the recipient with the exact link where the questionnaire can be found, so that they will be
immediately directed to the questionnaire’s first page. You are already asking for somebody’s
time;donot make themspend extra time searching for the questionnaire; chances are they will
not. Do not forget to include an attached file with the questionnaire.
6. A request for sending feedback on the practice within the specified deadline
The participantswill have toknow until when they can report their practices. Ask them kindly to
report the practices by 18th
February 2017 so as to collect all the requested data within the time
limits specified by the IMPADA project.
7. A contact person’s details
Apart fromthe obviousreasonof feedback,the contactdetailsof a person grant reliability to the
email perse and the requestingeneral.The recipientswouldfeel more comfortable knowingthat
this is a responsibly signed email and that there is an actual person making this request.
8. An appreciation concluding line
Within the context of appreciation to the recipient a thank you line should conclude the email.
(i.e. We would like to thank you in advance for your participation in the reporting of successful
adult education practices on the disadvantaged.”)
7.3.2. Follow up email
Whateverthe reason might be, some people will not report the practice of own organisation by
completing the questionnaire. In these cases a follow up email should be sent after a rational
period of time (in our case one week after the inviting email was sent). It is important that this
email will not be a copy of the original so as to show that some effort has been put in it.
What is more, it should strictly be a reminder of the questionnaire. It should include the initial
invite rephrased,the linktothe questionnaire andakindconcludingphrase.Thishasto be altered
depending on whether it is a targeted follow up or not. If it is a targeted one, a line like “Thank

28
you for your time. Your participation is much appreciated/ important” is fine. Otherwise, the
concluding line should distinct between the intended and not intended recipients of this
reminder. A line like “if you have already answered the questionnaire, we apologise for this
reminder/please ignore this reminder” is a good idea. Do not send a second reminder.
7.3.3. Thank you email
It is polite to send a thank you reply to people that spent some time to complete the
questionnaire. It shows appreciation and acknowledgment.
The thank you email should be brief and to the point. It could also contain some information
about the outcome of the survey and the results to be produced (e.g. the date that the
recommendations for improvement on effectiveness of adult education for the disadvantaged
learner is assumed to be published and the available link).
8. Quality assurance
Qualityassurance isthe processof verifyingordeterminingwhetherproductsor services meet or
exceed expectations. Quality assurance is a process-driven approach with specific steps to help
define and attain goals. This process considers design, development, production, and service.
PROMEA should check progress on the reported practices and on the on-line questionnaire and
notifythe Partnersincase practicesare not reported yetorcontainincomplete questionsor data.
Thiswill guarantee the qualityof the expected data and the collection of the foreseen practices.
Furthermore,itisrecommendedtotest the questionnaire before it is uploaded so as to identify
any problemsordifficulties for the providers of adult education practices for the disadvantaged
learner. It is also suggested, in case of availability, to circulate the questionnaire to external
experts and ask for their advice or any suggestion regarding the design or development of the
questionnaire.
Parametersforevaluating,categorizing,andprioritizingriskstypicallyinclude risk likelihood (i.e.,
the probability of risk occurrence), risk consequence (i.e., the impact and severity of risk
occurrence), and thresholds to trigger management activities. Risk parameters are used to
provide common and consistent criteria for comparing the various risks to be managed. For the
upcoming collecting of practices the following risks have been identified:
Id Risk Likelihood
(Low / Medium/
High)
Impact
(Low / Medium
/ High)
R1 Insufficientpracticesgathered Medium High
R2 Heterogeneousand/orincompletedata
gathered
Low High
R3 Delaysinthe reportingof practices Medium Medium
Table 14 Risk management: identifying risks
A critical component of a risk mitigation plan is to develop alternative courses of action,
workarounds,andfallback positions, with a recommended course of action for each critical risk.

29
Risksare watchedwhenthere isanobjectivelydefined,verifiable, and documented threshold of
performance, time, or risk exposure (the combination of likelihood and consequence) that will
trigger risk mitigation planning or invoke a contingency plan if it is needed. In many cases, risks
will be accepted. Risk acceptance is usually done when the risk is judged too low for formal
mitigation,orwhenthere appearstobe noviable wayto reduce the risk. The identified risksform
a baseline to initiate risk management activities as presented in table 17 below:
Id Risk Threshold ProposedAction
Preventive Contingency
R1 Insufficient
practices
gathered
Lessthan 50
practices
Identifyalternativeproviders
of adulteducation practices
for the disadvantaged
learner
Contac the
alternative
providers and
engage ina
telephone
interview tocollect
data on their
practice
R2 Heterogeneous
and/or
incomplete data
gathered
More than 20% of
the questionsare
not filled in.
Warningmessagestothe
providers duringfillinginand
before the publicationof the
practice
Obligatory
responsesforthose
questionsthat
correspondtothe
respective
evaluationfields
R3 Delaysinthe
reportingof
practices
The providers
have not
deliveredthe
reportswithinthe
specified
deadlines
PROMEA has to monitorthe
total numberof practices
uploadedatthe website
PROMEA should
notifythe
respective partners
to follow upand
pursue responses
Table 15 Risk management: correctivemeasures
9. Annexes
9.1. Invitation email template
Dear [salutation, surname]
I represent the consortium of the European co-funded project IMPADA 2015-1-UK01-KA204-
013666 Improving the effectiveness of adult education for disadvantaged learner) of the
Erasmus+ programme,whichaims atimprovingthe systematicevaluation of the effectiveness of
adulteducationprovisiontowardsthe disadvantagedlearner with a view to reduce disparities in
and enhance access to adult learning
Knowing of your organization’s long and significant involvement in the field of Adult Education
(you can mention specific eventsor previousprojectsorexperiences) we wouldlike toinviteyouto

30
the reportingof a successful adulteducationpractice forthe disadvantagedlearnerthat has been
implemented by your organization. We trust that you are the appropriate person to offer your
valuable insight by reporting the practice adopted or developed by your organisation.
To this end, we have prepared one questionnaire so that you can provide us with all the useful
informationanddetaileddescriptionof the practice adopted. You are kindly requested to report
your adult education practice for the disadvantaged learner following this link and according to
the instructions attached in the respective file “instructions for log in at the website” with this
email. Alternativelyyou can also fill in the attached file and kindly return it to us at your earliest
possible convenience, and by 17th
February 2017. We understand that in some of the topics
tackledyou may have questionsonwhattofill in,andin that case please consult the instructions
found in the introductory section of the questionnaire. If the instructions are proven not to be
helpful to you, please do not hesitate to contact us for further instructions.
In the contextof the IMPADA project, we intend (givenyourpermissionof course) toreferbothto
the name of your organization and to your practice with the aim to include them in the
recommendations on successful adult education practices which is going to be developed from
the IMPADA project.Furthermore, the reported practice could alsobe transferredandadoptedby
any other interested party, so would be grateful if you could provide us any additional
information, documentation and files relevant to the implementation of the practice that your
organisationhasimplemented.Once the aforementioneddeliverablesare finishedandpublished,
youcan findthematthe followinglink www.impada.euatthe tab “Outputs”,andI would also be
happy to send you a copy of them.
The deadline for the reporting of practices is by 17th
February 2017, so please fill in the
questionnaire before that date.
I wouldalsolike to kindly askyoutonotifyme whetheryouagree thatwe listyourname as one of
the experts that assisted us and also on whether you agree that we can cite your views in our
report.
On behalf of the IMPADA consortium I would like to thank in advance for your time and effort.
Kind regards,
Signature,
Contact details
9.2. Follow up email template
Dear Sir/Madame
We would like to kindly remind you that you have been requested to report a successful adult
education practice on the disadvantaged learner of your organization in the context of the
European Project IMPADA.

31
We noticedthatwe have not yetreceivedanyresponse from your side, so we took the liberty to
remindyouthat yourknowledge and contribution are very important to the achievement of the
goals of the IMPADA project.
Your personal data will be treated as confidential.
Here is the link to the questionnaire
We wouldlike to thank you in advance for your attention and actions, and we trust that you will
find the time to fill the questionnaire.
Please ignore this reminder if you have already answered the questionnaire. (optional)
Signature
Contact details
9.3. Thank you email template
Dear Sir/Madame
We wouldlike tothankyouforyour time reportingthe successful adulteducationpractice on the
disadvantaged learner. Your contribution is much appreciated.
We would be pleased to keep you updated once the results are published.
Yours sincerely,
Signature,
Contact details
9.4. Link to the questionnaire in English
https://docs.google.com/forms/d/e/1FAIpQLSfTZVLyPPiV010mxJL9qrKMFcW4wDsmG-
aDynXOqVnXjAKjKg/viewform
9.5. Template form for the registration of the invited providers of successful
adult education practices for the disadvantaged learner
Index Name &
surname
Profession Name of
organisatio
n
Email
address
Contact details7 Brief
description
of practice
Invitation
date
R/
NR8
7 Phone number, mobilephone, address
8 Report/ Not report yet

IMPADA_O3_A1_ Research methodology successful practices for disadvantaged learners

Recommended

Recommended

More Related Content

What's hot

What's hot (12)

Viewers also liked

Viewers also liked (19)

Similar to IMPADA_O3_A1_ Research methodology successful practices for disadvantaged learners

Similar to IMPADA_O3_A1_ Research methodology successful practices for disadvantaged learners (20)

More from Andrea Ciantar

More from Andrea Ciantar (20)

Recently uploaded

Recently uploaded (20)

IMPADA_O3_A1_ Research methodology successful practices for disadvantaged learners