SlideShare a Scribd company logo
1 of 27
Download to read offline
Copyright © Paul Talbot 2015. All Rights Reserved 1
Achieving Sustainable Impacts in Projects Funded under the European
Lifelong-Learning Programme (2007-2013)
Paul Talbot, M.Sc. B.A. (Hons)*
AB S T R A CT
The European Lifelong Learning Programme is amongst various policy instruments providing community
grants for projects related to the strategic policy goals of the European Union, but has been criticised for
the fact that project results are rarely mainstreamed. Not unique to European funding, many publically-
funded programmes fail to maintain long-term outputs. This study sets out to explore the factors facilitating
and inhibiting impact sustainability of European Lifelong Learning projects, looking specifically at project
sustainability from a systems perspective, using in-depth qualitative interviews with professionals currently
coordinating projects under the Lifelong Learning Programme to identify factors within the internal
organisational environment, its interaction with the external environment, and the influence of the
programmatic environment, and exposing the impact of systemic relationships between these parts.
Lacking the resources to conduct thorough iterative analysis and achieve theoretical saturation, this study
uses inductive methods to generate empirically-grounded hypotheses, drawing provisional conclusions and
implications concerning the policy and implementation of European Lifelong Learning projects, providing
a starting point for validation and verification through further qualitative and quantitative research whilst
hopefully serving to sensitise the policy and management research community, as well as project
coordinators, evaluators and funding-body officers to the importance of the topic.
*Paul Talbot is a project manager and consultant at die Berater® UnternehmehmensberatungsGmbH in
Vienna, Austria and is also the project officer of the European Prison Education Association (EPEA).
Talbot has coordinated, partnered and consulted on a range of projects in both the education and criminal
justice field, and is particularly interested in the use of European-funded projects as tools for helping
NGOs and civil-society organisations to develop capacity and work towards achieving their long-term
strategic objectives. This paper is an edited version of Talbot’s Master’s dissertation (M.Sc. Public Policy
and Management) at the University of London’s Centre for Financial and Management Studies.
Contact: p.r.talbot@live.com
1 IN T RO D U CT I O N
Lifelong learning is central to the strategic
framework for European cooperation in Education
and Training (ET2020),i
as well as to broader
international policy as a means of combating
exclusion and inequality, and promoting economic
sustainability,ii
The European Lifelong Learning
Programme (LLP)iii
is amongst various policy
instruments providing grants for “projects or
activities in relation to European Union Policies”.iv
There are few studies of the LLP, although an
Interim Programme Reportv
confirmed its
operational effectiveness and policy alignment,
but criticised project exploitation,vi
commenting
that results were rarely mainstreamed.vii
Not unique to European funding, many publically-
funded programmes fail to maintain long-term
outputs, with perhaps “as many as 40% of all new
programs [not being] sustained beyond the first
few years after termination of initial funding”.viii
Ingle (2005) argues that project effectiveness is
“measured in terms of benefits at the end of the
project funding cycle” and that projects
“frequently stop delivering the desired benefits as
soon as the money runs out”, due to “benchmarks
Copyright © Paul Talbot 2015. All Rights Reserved 2
[being] defined in terms of effectiveness,
neglecting institutional aspects concerning the
capacity to sustain the delivery of benefits after
donor funding ends”,ix
somewhat echoing Curtis’
(2004) criticism of centralised policy objectives
being turned into instruments for implementation
by other parties, subject to output measures with
“unintended consequence[s] of preventing better
or more appropriate standards or measures, or even
different products, from emerging in response to
learning experience during the intervention”.x
Project sustainability should be central to
evaluation,xi
yet Sequira and Moharara (2012)
argue that the qualitative nature institution-
building objectives are inherently difficult to
evaluate and neglected in favour of quantifiable
financial and economic indicators,xii
whilst
Savaya et al (2008) argue that “for sustainability to
become a routine component of program
evaluation, there is a need for greater clarity about
obstacles to sustainability and their causes”.xiii
Sustainability is amongst key appraisal criteria in
the LLP, however the realisation of sustainability
has not yet been critically investigated in this
policy context. Targeted research is critical for a
better understanding of project management
processes, and their relationship to policy
realisation. Understanding what influences project
sustainability will be useful for project applicants
developing proposals, and for European decision
makers in conducting project appraisals, and may
help in understanding how funding policies can be
optimised to increase the effectiveness of such
policy instruments as the LLP.
2 LI T E R AT U RE RE V I EW
2.1 DEFINITIONS AND DIMENSIONS OF
SUSTAINABILITY
Limited attention is given to impact sustainability
across the management disciplines.xiv
A project
may be sustainable “when it continues to deliver
benefits to the project beneficiaries and/or other
constituencies for an extended period after […]
financial assistance has been terminated”,xv
and
where “new ways of working and improved
outcomes become the norm.”xvi
It may include the
sustained use of facilities and infrastructure,
continuation of established services or of
approaches piloted through existing services,
utilisation by staff or beneficiaries of newly-
acquired skills, the adoption of ideas at a policy
level, the development and continuation of
partnerships and collaborations either for
continued service delivery or for the sake of
capacity building, sustained behavioural change,
or the integration of new information or
perspectives into the accepted body of knowledge
of a particular field.xvii
It is a dynamic process
within a local system in which interdependent
factors need to be balanced and is heavily
influenced by cultural, social and economic
factors.xviii
2.2 STATE OF THE LITERATURE
A limited pool of programmatic and research
literature reveals a range of relevant issues. An
extensive review by Savaya et al (2008) identifies
various factors based around (1) project design and
implementation, such as having clear
sustainability plans, securing financial resources
or developing financing plans, disseminating
evidence of success, adopting to changing
environments, having sufficiently-trained human
resources, and having ongoing evaluation; (2) the
organisational setting, such as organisational
stability and flexibility, having program
champions within the organisation, managerial
support and flexibility and integration into the
organisation; (3) the broader community, such as
having community support for the program,
political legitimation, and an appropriate socio-
economic context, and; (4) the funding body, such
as assuming responsibility and providing support
and allocating capacity-building resources to
recipient organisations.xix
The results of their own
study highlighted host-organisation leadership as a
critical factor, arguing “the heads of the host
organisations of [sustainable programs] fought
hard to keep the programs alive and understood
what was needed to do so” and that “the decisions
of the host organisation leadership […] ensured
sustenance of the surviving programs by giving
them high priority and integrating them in the
organisation’s structure”xx
but found that a number
of external factors such as competing programs,
power relations or government policiesxxi
could
threaten sustainability. Unfortunately this
literature is contextually removed from the current
Copyright © Paul Talbot 2015. All Rights Reserved 3
research problem. It is focused primarily on
evaluation practice, taking a static view of
program environments, is often disproportionately
based on projects in the health-care sector, and
fails to reflect on policy processes. However, a
broader search of the literature reveals a
relationship between project sustainability, change
management and organisational systems,
providing a new approach to classifying the
literature on sustainability outside of their original
thematic and programmatic context.
2.3 PROJECT SUSTAINABILITY AS A
SYSTEMS PROCESS
Emerging from the literature review were factors
that could be ‘categorised’ by their occurrence in
the operational environment; the internal
organisation, responses to the external operational
environment, impacts on the external operational
environment, or processes in the programmatic
environment, that is, policy and operations at the
level of the funding body.
Figure 1 – The literature categorised within an organisational system
Sustainability is subject to interdependent
components and some already see sustainability as
part of a local system.xxii
Organisations managing
projects, like all organisations, “exist in and relate
to environments that affect their operations”.xxiii
Sustainability is discussed at the level of
organisational change,xxiv
which may be variously
defined as transformations within an
organisation’s structure and operations, brought
about by environmental instability.xxv
This is
conceptually different from a project, which is
seen as “a temporary endeavour undertaken to
create a unique product, service or result”.xxvi
However, where Buchanen et al (2005) see
organisational change as something that either
persists or decays, as a result of various factors,
then strong correlations can be drawn with the
project concept. Both projects and organisational
change activities seek to produce outcomes that
disrupt existing systems, and require attention to
structural and systemic issues.xxvii
Pitagorsky
(2011) claims that “project managers, to be
effective, must be competent change managers”
but also sees change management as a project
activity.xxviii
This supports a systems view of project
sustainability, and whilst systems theory has
weaknesses, it serves here only as a lens through
which to categorise relevant literature. Savaya et
al (2008) argue that the factors they identified
“should be reflected in actions and processes
designed to enhance sustainability”;xxix
seeing
sustainability within a more ‘holistic’ theoretical
frameworkxxx
of systemic environmental
interactions to which these actions and processes
are exposedxxxi
provides a means for
distinguishing between sustainability factors,
forming a “context-free theoretical
framework”xxxii
through which to contextualise
Copyright © Paul Talbot 2015. All Rights Reserved 4
the factors on a more general level, avoiding
context-driven assumptions. Meta-
ethnographiesxxxiii
or meta-synthesesxxxiv
pull
corroborating concepts together whilst
highlighting those ‘refutational’ concepts,xxxv
producing interpretations that “will not be found
in any one research report but, rather, [which are]
derived from taking all of the reports in a sample
as a whole".xxxvi
Whilst there may be limited
scope for a truly systematic review due to the
ambiguous and multifaceted nature of
sustainability as a research concept,xxxvii
this
approach provides clarity to the researcher’s
thought processes, offering structure to the
interviews, helping to avoid imposing theoretical
ideas relating to the research question.
2.4 CATEGORISATION OF
SUSTAINABILITY LITERATURE WITHIN A
SYSTEMS MODEL
There is no scope within this paper to provide a
detailed qualitative description of the constructs
identified, however they are very briefly
presented as a means of supporting the
contextualisation of project sustainability within a
systems perspective.
2.4.1 INTERNAL ORGANISATIONAL
ENVIRONMENT
Project sustainability can be affected explicitly by
activities in the internal organisational
environment. Sustainability should be central to
project strategies, influencing all aspects of the
project cycle,xxxviii
and require experienced and
effective management teamsxxxix
and mechanisms
for effective planning and task allocation.xl
The
planning and pursuit of sustainability must be
supported by meaningful commitment, not only
forced by external actors,xli
and be subject to
monitoring, evaluation,xlii
and performance
measurementxliii
against a consistent set of
sustainability indicators across a range of project
system components.xliv
2.4.2 RESPONDING TO OPERATIONAL
ENVIRONMENTS
Project sustainability can be affected by the
activities conducted in response to external
stimuli. Projects need certain conditions in the
environment to sustain benefits,xlv
which need to
be thoroughly explored and understood.xlvi
Involving stakeholders and project participants at
every stage of the design processxlvii
helps identify
their various roles, interests and influencesxlviii
and may help ensure that project outputs are
sufficiently meeting stakeholder demands.xlix
Projects should seek to align with policy
directionsl
within a supportive and enabling
policy and regulative environmentli
with political
sustainability and government commitment,lii
securing policy support and legitimation.liii
Factors relating to the socio-economic context
such as the strength of civil society and
empowerment of women,liv
local market
conditionslv
and the natural environmentlvi
may
affect a project’s sustainability prospects.
2.4.3 INFLUENCING OPERATIONAL
ENVIRONMENTS
Project sustainability can be affected by the way
project actors seek to influence behaviour or
events in the external operational environment.
Projects may be designed to leave a legacy of
outputs with mainstreaming potentiallvii
requiring
minimal further financial investment.lviii
Ensuring sufficient resources to sustain project
outcomes are seen as critical in the sustainability
process.lix
Projects with limited access to funding
resources may face problems.lx
Some project
activities will be unfeasible when no further
funding is providedlxi
and so project implementers
need to create the ability to attract further funding
from external sources.lxii
Ownership of project
outcomes by the institutional environment, and
particularly from those expected to continue
project delivery is discussed in depth;lxiii
projects
should actively seek to find an ‘institutional
home’, and transfer results to the institutional
environment.lxiv
Projects need to build the
capacity of local institutions to be able to continue
delivering project outcomeslxv
ensuring
infrastructural capacity for service deliverylxvi
and
technical maintenance,lxvii
but also need to
encourage stakeholders to independently take
advantage of project outputs, which requires
capacity at the user or beneficiary level.lxviii
Although this aspect has received relatively little
attention in the literature, it may include resilience
to adapt to change,lxix
risk management capacitylxx
and the ability to preserve natural environmental
resources.lxxi
Stakeholders need to be made aware
of project activities and outcomes though
sufficient advertising of the project and its success
or impact,lxxii
especially where stakeholders
Copyright © Paul Talbot 2015. All Rights Reserved 5
critical to sustainability may be less involved in
implementation and have little exposure to the
project.lxxiii
2.4.4 PROGRAMMATIC ENVIRONMENT
Project sustainability can be affected by forces
acting within the programmatic environment,
which includes policy makers, programme
managers and project officers amongst others.
Support on a programmatic level can help to
achieve sustainability aims,lxxiv
through helping
develop sustainability plans, and develop staff
capacity.lxxv
Problems deriving from the design
and implementation of funding programmes will
impact sustainability potential. For example,
project funding periods may be too short to build
strong-enough relationships with necessary
partners,lxxvi
or long lead-times between project
application and implementation may mean initial
plans are no longer feasible due to situation
changes.lxxvii
Programmes emphasising
innovation and demanding ambitious, complex
outcomes, often on an unrealistic timescale may
inhibit sustainability.lxxviii
Sustainability forms a
more significant part of project evaluations where
it is reflected as a core value in the programme
and emphasised throughout the project delivery
cycle.lxxix
3 RE S E A R C H AP P R O A C H A N D TO O L S
3.1 RESEARCH QUESTIONS
This study explores factors influencing
sustainability in European LLP projects. The
Interim Evaluation of the LLP criticises project
exploitation and mainstreaming.lxxx
A review of
literature across diverse project fields revealed
factors influencing project sustainability within the
following settings:
(1) Internal Organisational Environment
(2) Responding to Operational Environments
(3) Influencing Operational Environments
(4) Programmatic Environment
These are not theoretical bases for sustainability in
the LLP, but rather ‘sensitising concepts’,lxxxi
‘constructs’lxxxii
or ‘salient factors’lxxxiii
providing
direction to data collection, serving as ‘points of
departure’ for exploring the perspectives of project
actors in this field,lxxxiv
and can be used to
formulate four initial questions, focused on the
LLP.
Q1. What are the internal organisational factors
which facilitate or inhibit project sustainability?
(Internal organisational environment)
As identified in the literature review,
organisational capacity and strategic orientation
impact the sustainability of project outcomes.
Furthermore, the ways in which people define and
measure project success may neglect sustainability
elements. Coded data emerging from the
interviews is analysed for its implications on the
relationship between project sustainability and the
organisational environment. An ‘organisational
factor’ refers to any aspect located within the
project organisation itself. A systems view of
organisations sees components such as
management, strategy, goals, structure, operations
and technology as well as organisational culture,
politics and leadership as forming the internal
environment.lxxxv
Q2. What are the processes by which individuals
and organisations respond to external
environments to promote project sustainability?
(Responding to operational environments)
The literature identifies a range of factors
associated with the way projects respond to
external environments, however given the context-
driven nature of existing work, there are few
generalizable explanations for these interactions.
This study explores how project organisations
interact with the external environment. Coded
data emerging from the interviews is analysed for
its implications on the relationship between project
sustainability and the processes involved in
responding to the external environment. Processes
may include activities responding to externally-
imposed conditions, including political, economic,
socio-cultural, technological, legal or ecological
conditions.lxxxvi
Copyright © Paul Talbot 2015. All Rights Reserved 6
Q3. What are the processes by which individuals
and organisations create change in external
environments to promote project sustainability?
(Influencing operational environments)
Long-term project sustainability requires changes
in the operating environment. Coded data
emerging from the interviews is analysed for its
implications on the relationship between project
sustainability and the processes involved in
influencing external environments, including
those in the political, economic, socio-cultural,
technological, legal or ecological
environmentlxxxvii
as mentioned above.
Q4. How does funding programme structure itself
facilitate or inhibit project sustainability?
(Programmatic environment)
Funding-programme structures appear to influence
pro-sustainability practices. This study looks at
how the funding programme itself affects the
sustainability potential of projects. Coded data
emerging from the interviews is analysed for its
implications on the relationship between project
sustainability and the programmatic environment.
Policy content, administrative processes,
contractual obligations, the flexibility and length
of projects and the perceived degree of support
from the funding body may be amongst the topics
discussed.
3.2 METHODOLOGY AND LIMITATIONS
Grounded theory - used to make and understand
observations through a developing conceptual
framework,lxxxviii
testing and refining theoretical
concepts that it generates, and building theories
which can later be tested against further datalxxxix
-
helps gain insights into previously neglected areas.
However resource restraints make theoretical
saturation difficult to achieve in this study.
Interviews are used to generate verbal data,
although as Halcomb and Davidson (2006) reason,
“the richness and depth of these data will vary
according to the specific research design and the
research question that the investigation is seeking
to address.”xc
To fully explore the values,
meanings and beliefs of project actors is
realistically outside the practical scope of the
study,xci
and although findings are grounded in the
available data, they only reflect one round of
analysis and cannot reach irrefutable theoretical
conclusions. They can, however, provide data on
a previously neglected topic, and establish a
starting point for more detailed research; rather
than producing full grounded theory, this study
produces empirically-grounded hypotheses for
further testing.
3.2.1 THE ROLE OF THEORY
The research falls outside a purely deductive-
inductive dichotomy.xcii
Existing theory, albeit
rooted largely in grey-literature, presents a
framework for pro-sustainability practices in
project management. This theory, however, is not
appropriate to the circumstances surrounding the
current research question; the study is not
concerned with general, proscriptive sustainability
practices, but rather in the organisational factors
that influence these practices in the specific
context of the European LLP. In their study on
entrepreneurialism, Ali and Birley (1999) had
enough ‘theory’ to formulate a deductive research
design, but this had not yet been applied to the
situation being researched, compelling them to
incorporate an inductive approach into a deductive
design.xciii
Their argument is that existing research
can be used to develop constructs – rather than
variables – helping to provide focus to the
research, whilst opening up the discussion to
different themes, not previously considered,
helping establish a context for the research,
determining questions to be asked and the assisting
in the tabulation of the interview data (Ali &
Birley, 1999). In this current study theoretical
constructs identify issues deemed relevant by
researchers and practitioners in a range of
contextualised project settings (Ali & Birley,
1999), whilst the research itself explores the
relationship between these constructs and the
facilitating or inhibiting factors within
participating organisations. Blumer (1954) coined
the term ‘sensitising concepts’ to describe the
initial ideas that “sensitize you to ask particular
kinds of questions about your topic”.xciv
Charmaz
(2006) uses these concepts, derived from
background assumptions and disciplinary
perspectives as “points of departure to form
interview questions, to look at data, to listen to
interviewees, and to think analytically about the
data”.xcv
Fernandes et al (2013) take a similar
perspective in identifying an ‘initial framework of
salient factors’ from the literature on improving
and embedding project management practices, and
then conducting interviews in order to identify new
Copyright © Paul Talbot 2015. All Rights Reserved 7
factors, confirm the factors identified and to
remove less important factors.xcvi
3.2.2 SAMPLING
Purposive- sequential- or theoretical sampling are
given as appropriate approaches to grounded
theoryxcvii
where the researcher “[carries] out
initial sampling, and from analysis of the results
[extends] the sample in ways guided by their
emerging theory”xcviii
or “examines particular
instances of the phenomenon of interest so that he
or she can define and elaborate on its various
manifestations”,xcix
however since this research is
concerned with European LLP project
implementation broadly, the full population of
European LLP project actors were of interest.
Using purposive sampling of all individuals who
coordinated LLP projects in 2012 and 2013, 285
potential interview participants were approached,
and offered entry into a draw to win a €50 book
voucher, and 21 individuals agreed to participate.
The sample of 3 higher-education managers, 6
project managers at private training providers, 1 at
a public training provider and 1 at an NGO, 7
university academics and 3 representatives of
independent research centres contained
individuals with a representative range of
characteristics, covering the public and private
sectors. This was the most financially- and time-
efficient method for sampling this population,c
although response bias may limit the
generalizability of the results, as considered in the
discussion of results.
3.2.3 CONDUCTING THE INTERVIEWS
Semi-structured interviews are regarded as being
“well-suited for the exploration of attitudes,
values, beliefs and motives” and can help explore
“the perceptions and opinions of respondents
regarding complex and sometimes sensitive issues
and enable probing for more information and
clarification of answers”.ci
They were conducted
through a mix of 6 individual and group
interviews, conducted by international
teleconference (5 interviews with 12 people) and
face-to-face meeting (1 interview with 2 people).
Attention was made to ensure that all questions
were as open as possible and to avoid leading
questions. An interview guide of open and non-
leading questions, initially piloted with colleagues,
and refined for use in the formal interviews, is used
to prompt respondents to discuss the processes
involved in project sustainability from various
different perspectives. The guide evolved over the
course of the interviews, helping focus the
questions and minimise non-relevant interview
data, and questioning followed Roulston’s (2010)
suggestions for delivering open questions and
responding to interviewees.cii
3.2.4 TRANSCRIPTION AND DATA ANALYSIS
Epistemological concerns dictate whether we
value transcripts “[corresponding to] the original
conversational event” or whether we prefer “re-
presentations and interpretations”.ciii
Interpretivist
researchers battle with distortions of meaning.civ
Transcriptions are “artificial constructions”,cv
emerging from interviewer perceptionscvi
and
decisions as to what information is required and
the value of its analytical contribution.cvii
The
current study involves cross-cultural and multi-
lingual groups of respondents. Welch and Piekkari
(2006) write that language is “critical to
understanding the qualitative research interview”
and that “language, symbols and discourses play [a
significant role] in shaping meaning and power in
interviews”.cviii
Tsang (1998) argues that
researchers should communicate in the
respondent’s language, enabling full expression,
good rapport and cultural understanding.cix
Resource constraints, however, require that
English be used as lingua franca in the interview
process. This presents a challenge to the
transcription of data punctuated with lexical and
grammatical inaccuracies, erroneous
pronunciation and imperfect discourse
management, but also to the analysis of data.cx
Responses may not reflect an authentic record, and
will lack “subtle nuances” that come with “being
confident enough to formulate extended and
individualised responses”.cxi
Furthermore, most
respondents were interviewed through low social-
presence telephonic media without the contextual
nuances afforded by non-verbal interaction,
further questions the value of the in verbatim
transcription, limiting the empirical potential of
the research. However, seeking to create
empirically-grounded hypotheses rather than
formal theory, this approach still enables the study
to highlight key factors, establishing the
groundwork for further investigation.
Copyright © Paul Talbot 2015. All Rights Reserved 8
4 IN T E RV I EW RE S U LT S
Eight in-depth interviews, with 20 respondentscxii
yielded a broad range of data. Initial and focused-
coding revealed that LLP project actors perceive a
number of factors influencing sustainability.
Factors are described here in relation to the
interview data, and discussed more analytically in
the next chapter.
4.1 THE INTERNAL ORGANISATIONAL
FACTORS
Internal organisational factors were identified at
the level of the partner organisation and of the
project consortium.
4.1.1 ORGANISATIONAL PARTICIPATION
AND OWNERSHIP
Discussing allocation of work resources,
respondents experienced projects being delivered
by semi-autonomous groups, distinct from
operational management.
“The idea, main concept and much of the
idealising comes from the staff involved”1
“Public relations, marketing and
dissemination, you need professionals for
that, but we are an organisation where they
leave it to the [scientists] and that's not
good, we should do that together”2
“The organisation’s bosses learnt about the
project issues from [external
stakeholders]”3
Some statements indicate that involving the
broader organisation in the project content may
promote sustainability.
“If your organisation doesn’t support the
project, it’s just your work, not the
1
I2R04 – Assistant University Professor
2
I6R16 – Lecturer at a University of Applied Science
3
I7R17 – Consultant and project manager in higher-
education sector
4
I7R17
5
I5R11 – Project manager at a public research centre
organisation's work [...] if the organisation
supports the project, […] it’s about
supporting the ideas, sharing the ideas of the
project”4
“A degree programme doesn’t only consist
of [the project actors], you are usually 15
people […] how do you reach these
people?” 5
“[Sustainability can be improved by]
working on the dissemination structure or
policy of a project at a general company
level […] and maybe share better the
resources you have to make it more
effective”6
Problems occur when projects become reliant on
individuals and their project know-how.
“Everybody brings different skills to a
project, so if somebody is missing, the skill
is missing”7
“[Those] involved in the management and
production of the results are there, that helps
a lot because you have the know-how in
place and you can use it later on”8
“You need one person at least for
sustainability who has the contacts, who
works with the material”9
Problems occur because projects finance project
teams. Expertise develops within isolated groups,
sometimes including external-contract staff,
preventing crossover into the organisation, as
nobody outside of the project team has ownership,
or stewardship, of project results.
6
I5R12 – Independent researcher and project coordinator in
the higher-education sector
7
I4R08 – Project manager at a state-funded adult education
provider
8
I2R03 – Managing director of a private consultancy
provider
9
I3R07 – Project manager at a private education, training,
and consultancy provider
Copyright © Paul Talbot 2015. All Rights Reserved 9
“It’s very difficult to get financing [once the
project] has ended, at a national or even a
local level, […] and time, more than
anything, needs to be paid somehow”10
“It’s difficult to implement [exploitation
after the project period] because we are
continuously involved in new projects”11
“If you don’t have a steward then it doesn’t
work […]We always have a mixed team of
internal and external people that we fund
with the project, and we can always make
sure that […] somebody from the internal
personnel becomes the steward”12
It follows that:
P1: Integrating project outputs into activities
outside of the project goals requires the
involvement of people in the day-to-day
operations of the organisation
4.1.2 GOALS AND MOTIVATIONS
Differential foci on acquiring and completing
projects, and pursuing long-term non-project goals
affect the planning, implementation and capacity
to exploit project results. Focus may lie on what
the project itself brings the organisation in money
or opportunities to complement present activities.
“My boss’s first question is ‘how much
money do we earn’“13
“You write a project based on your
competences as an institution, but from a
sustainability point of view, you have no idea
of the impact that you will have”14
Alternatively, projects may be tools to achieve
non-project objectives, such as contributing to
long-term research agendas, supporting the
10
I5R12
11
I6R15 – Project manager and consultant at an innovation
management firm
12
I8R20 – University Professor, Dean of Faculty
13
I4R08
14
I2R02 – Project manager in the higher-education sector
development of curricula, becoming competitive
on the real market, or creating resources to build
long-term capacity.
“We pursue projects with vested interests, to
improve the situation of our students and our
teachers”15
“Our intention [with our project] is to
become competitive on the real language
market”16
“There is a long-term strategy behind all
these projects, it’s not just […] to get some
extra money […] but it should be an integral
part of long-term strategy” 17
It follows that:
P2: Organisations have various motivations for
participating in projects, including project
acquisition and pursuit of non-project goals.
4.1.3 GENUINE AND ARTIFICIAL PLANNING
Project actors need to plan for sustainability in the
proposal, during and after the project, however
genuine planning and artificial planning are not
easily distinguishable. Genuine planning may
require constant long-term thinking, joining only
projects with sustainability potential, building
sustainability into product design, or including
sustainability amongst project monitoring criteria.
“If there is a way to add this dimension, then
it makes sense […] to think about, ok, what
is the goal of the project, and how can we
exploit that, or how can we use that
afterwards”18
“We’ve been looking at how we can start to
integrate concepts such as open access, free
15
I8R20
16
I4R10 – Project manager at a European NGO
17
I3R05 – University Professor at a University of Applied
Science
18
I8R19 – Managing director and proprietor of private
training, coaching and consultancy firm
Copyright © Paul Talbot 2015. All Rights Reserved 10
use, create commons, and things that enable
some of the results to go a little further”19
“Monitoring is good for sustainability by
helping develop milestones for what is
needed for sustainability and distributing
them throughout the project”20
However this may still reflect artificial planning.
There is no accountability for sustainability
planning yet the planning itself is compulsory.
Respondents regularly discussed projects using
expressions such as ‘have to’, ‘forced to do’ and
‘expectations’.
“Why do we do these things? First of all
because it’s compulsory to report it to
Brussels”21
“[the question is] how do I fill this in
according to the expectations […] and not
so much is the idea good or not”22
Project acquisition invites pragmatic proposal
writing, based on current possibilities, and subject
to time constraints, limiting the capacity to build
sustainability systematically into the project.
“It’s very important to think about
[sustainability] when writing projects.
Sometimes you just copy and paste
something that you’ve written on
sustainability in another project proposal”23
“We try to analyse the invitations we receive
and see if it fits with our core activities and
past experiences, and especially if you are
able to involve the future end users [...] the
problem is sometimes we don't receive
enough information to do that kind of
analysis”24
19
I2R04
20
I2R04
21
I1R01 – Project manager at a private education, training,
and consultancy provider
22
I2R04
23
I2R02
24
I6R15
“Not everyone understands the pace […]
and we try not to involve [stakeholders]
actually in the writing process, if we want to
have an application finished by the
deadline”25
Focusing on producing outputs, project staff may
view success in terms of the original goals being
reached within the budget, and all contractual
obligations being fulfilled, with little incentive for
sustainable approaches.
“It’s important to reach objectives in the
budget and not get any bad scores for the
evaluation”26
“Nobody says ‘ok we have to do a piloting,
so let’s see where we can [develop] a
sustainable network partner’, but you only
say, ‘I’ll call some people that I know and
hopefully someone will do this damned
piloting thing”27
However where projects contribute to
organisationally-mandated plans, planning may be
more long-term and utilise broader resources.
“The possibility of transposing [academic]
activities into European projects is
something we’ve been trying to do, so if we
propose a project, we usually work on it at
least a year or a year and a half before the
proposal”28
“We have the mandate to develop knowledge
with externals […] we have a longer cycle
than [the co-respondents], because we have
a research agenda that is 4-6 years long, and
[…] we use projects to help us further our
research agenda”29
“There are long-term strategic goals of the
university, and trying to create synergy with
25
I3R06 – Project manager at a private education, training,
and consultancy provider
26
I3R06
27
I3R07
28
I2R04
29
I3R05
Copyright © Paul Talbot 2015. All Rights Reserved 11
the topic you’re working on and the longer-
term goals of the university, you are
supported longer” 30
It follows that:
P3: projects are successful in terms of contractual
obligations even with artificial plans
And
P4: focusing on non-project goals encourages
genuine organisational planning, whereas
focusing purely on project goals permits
artificial, pragmatic planning
CONSORTIUM-LEVEL FACTORS
Commitment from- and ownership by partners is
important. Relations between partners are seen as
critical in impacting feelings of ownership.
“If you have problems with the consortium,
or partners are not interested in the project,
you will not reach sustainability”31
“The partnership wasn’t nice, there was
fighting from beginning to end, and I think
that has a huge impact on sustainability”32
“There can’t be much expected from the
sustainability of these results because people
are so unhappy with their ownership for
them […] We [try to] resolve conflict and
make sure they are actually working on
something they consider valuable for them
now, at the moment, but also for the
future”33
Sustainability should be taken seriously by
partners, through regular discussion and planning.
Official agreements and business plans can be
30
I3R05
31
I7R18 – Departmental Head in the higher-education sector
32
I3R05
33
I5R11
drawn-up, promoting enthusiasm for sustaining
results, and facilitating practical planning.
“you have to talk about it right from the start
with your partners and get back to the
question regularly, at each meeting we talk
about sustainability, what we can do, how
we can adapt our plans for the future”34
“part of the challenge was to develop a
business model, how to market it […]
pricing strategy and willingness to pay, and
find out, do a small market study”35
“As a [subject specialist] I had no idea
about business plans, it was a nightmare at
the beginning, but it helps, it helps having a
structure”36
Although consortium-level commitment is
important, organisational motivations are still
critical. One respondent saw personal idealism
and belief in the project as the source of their
commitment, yet as owner of their company they
were also a corporate decision-maker.
“I’m really convinced that we have done a
good job, it’s a good course and people love
it when then come to the course”
[…]
“I am willing to put in a lot of time and effort
to get these courses running […] as a self-
employed trainer and coach, I need to put
time aside for organising those courses, that
I’m not getting paid for” 37
It follows, therefore, that:
P5: Sustainability is promoted by consortium-
level ownership, but it is still insufficient without
organisational ownership.
34
I7R18
35
I8R20
36
I6R14 – Director and project manager of an independent
cultural research institute
37
I8R19
Copyright © Paul Talbot 2015. All Rights Reserved 12
This section shows that organisational factors
impacting sustainability include the organisational
motivation for participating, and the way this
affects planning and resource distribution, as well
as the commitment and ownership at a consortium
level.
4.2 RESPONDING TO THE OPERATIONAL
ENVIRONMENT
Factors here included responding to real needs
through sufficient analysis and understanding, and
involvement of externals in planning and
implementation, contributing to the creation of
valuable outputs that can be sufficiently and
sustainably utilised by target groups.
4.2.1 RESPONDING TO REAL NEEDS AND
CONTEXTS
Projects must analyse and address real – and
changing - needs and demands of target groups, the
market, and public policy. Producing good results
that cannot be used in practice discourages
sustainability.
“You have to be sure that your project
answers to a real need, a real demand”38
“The things your produce in the project,
maybe the world isn’t ready for them yet, or
maybe the world will never be ready”39
“You need to be aware of what already
exists on the market, because there are some
competitors of course that are offering
something that is similar”40
“Sometimes at the time of application, the
subject you are promoting is well
recognised, but sometimes there can be a
change in context, and then the project
becomes not so important for the
institution”41
38
I2R02
39
I3R05
40
I6R15
41
I7R18
42
I2R02
43
I2R05
They must also respond to the context and
behaviour of stakeholder groups, understanding
behaviour patterns and what to expect from
externals.
“Public institutions […] have an incredibly
complicated bureaucracy to deal with, so for
sure we have to take care and keep in mind
their structure”42
“We find that some organisations are not
able to deal with new learning styles [and
don’t] have the structures to lend
sustainability to what we have developed”43
“You need to know all that everything that
goes on in the organisation […] and
somehow to work together with them”44
Projects may benefit from responding to direct
input, involving externals in the development,
planning and implementation of the project.
“[potential end-users] actually contribute to
new ideas; […] raising project ideas”45
“In focus groups we design the products […]
we make them much more clear and concrete
with the help of the stakeholders”46
“Involve end-users from the very beginning,
and if possible in the partnership itself”47
“Early identification of who are the end-
users and stakeholders […] Try to involve
them in designing the application. It's very
important to understand the real needs and
real expectations”48
It follows that:
P6: Sustainability benefits from a detailed- and
comprehensive-as-possible understanding of real
44
I7R17
45
I1R01
46
I2R03
47
I2R02
48
I6R15
Copyright © Paul Talbot 2015. All Rights Reserved 13
needs and contexts in the external operating
environment
This section shows that sustainability is promoted
by ensuring alignment between project outputs and
real external needs by way of comprehensive
analysis and stakeholder input.
4.3 INFLUENCING THE OPERATIONAL
ENVIRONMENT
Factors here included the provision of quality and
accessible outputs that add value both to target
groups and delivering organisations, effective
communication and interaction with external
actors and the promotion of positive cognitive and
emotional responses in the external environment,
including encouraging external participation and
exploitation.
4.3.1 PRODUCING OUTPUTS
Taking into consideration P6
, the outward delivery
of outputs into the external environment which
meet given needs is essential, and increases
chances of real interest from target groups.
“We have now [after two years] people
asking ‘oh could you send us some material
because the topic is so interesting, and the
training fits very well’”49
“We found out that lots of practitioners like
it very much, also in different countries, and
they’ve been inviting our project members to
present and conduct workshops”50
However beyond responding to needs, products
should demonstrate sufficient market-quality.
“[sustainability doesn’t work] if it really
isn’t there, the product for the market, it
hasn’t reached that stage that the market
demands”51
“[one thing] for bad sustainability, is to
have a bad product”52
49
I3R07
50
I5R11
51
I3R06
52
I3R07
Furthermore, some respondents speculated on the
role that quality plays for organisations when
deciding to mainstream products.
“[there are organisations who think] if it
doesn’t look good, or it doesn’t meet the
standard, we will not promote it, because we
cannot associate our brand name with this
product”53
“I think [a partner who didn’t sustain the
outputs] thought that the quality which they
were contributing to was not high enough”
54
Where ‘market-ready’ means being having
sufficient quality and benefits to viably target
potential end-users on a competitive market, it
follows that:
P7: Delivering market-ready products increases
the potential for sustainability
Additionally, take-up of the product on the market
is enhanced by broadening potential target group
applicability, for example by increasing
accessibility or adaptability.
“If we ask our partners in Portugal to
translate these things into Portuguese […]
they are available to everyone in Portugal
[and] a side effect is that people in Brazil
can also use them”55
“Courses [are] usually designed to a special
target group, but can also be adopted to
other target groups very often”56
It follows that:
P8: Broadening target-group applicability
increases the potential for sustainability
53
I3R06
54
I4R09 – Senior lecturer at a University of Applied Science
55
I5R11
56
I8R19
Copyright © Paul Talbot 2015. All Rights Reserved 14
4.3.2 COMMUNICATION WITH AND
PARTICIPATION OF EXTERNALS
Project actors need to communicate information to
externals. Respondents placed emphasis on the
process of publicity.
“If you don’t make good publicity, so people
simply are not aware of it”57
“Establish contacts with as more as possible
stakeholders […] in order to reach really the
audience”58
“It’s important that the networks know
about the things they can use for example,
the products that we have done or the
services”59
Value is attached to methods which enable
interaction with, and feedback from potential end-
users, whether it is one-to-one interaction, group
interaction, or social-media interaction.
“Because we are a small settlement, it’s easy
to keep personal contacts”60
“Small focus groups […] seem to be very
interesting in receiving very direct opinions
about what is being developed”61
“[With Facebook] it’s easy for people to
respond, they are used to it, and if you can
sustain a reasonable response to social
media […] I think you can strengthen the
impact of your project for quite a while”62
This reiterates the value of external participation.
“I think that the basic concept […] co-
creating the actual project proposal, not just
sending out "do you want to participate in
this" with a complete proposal, but actually
57
I1R01
58
I4R10
59
I4R09
60
I1R01
61
I2R04
62
I2R03
developing the project proposal from the
beginning”63
“The direct involvement of stakeholders [...]
is very important, […] we always try to
involve as many as we can in [training]
activities”64
It follows that:
P9: Systematic and interactive communication of
project content and outputs promotes
sustainability
4.3.3 PROMPTING EXTERNAL RESPONSES
Independent action and response by externals
should be encouraged. Externals should feel
motivated to engage as a participant, feeling a
personal value or benefit in the project, although
this may require preparatory work.
“They have to see the project it really for
them, so they are not just used as an excuse
to have a project and to run something for
two years”65
“We have made [an external stakeholder]
care about the project, not just about the
product […] she also got interested on how
we are growing, and she got passionate
about it”66
“Sometimes […] people are not ready to
receive new approaches, if the context is not
prepared […] then you cannot really sustain
your excellent result”67
It follows, therefore, that:
63
I3R05
64
I6R14
65
I1R01
66
I3R06
67
I6R13 – Project manager at an independent educational
research institute
Copyright © Paul Talbot 2015. All Rights Reserved 15
P10: Encouraging individual connections to the
project and its outputs within external
environment promotes sustainability
Externals can exploit project goals in pursuit of
their own objectives, occurring when project goals
closely match those of external bodies.
“[The municipality] asks us to have some
presentations to channel it in their everyday
for the benefit of the local people”68
“Get a powerful institution who adopts your
product and you don’t need to worry about
sustainability”69
“We have […] an action, very much linked
to the goals of the project, to impact on the
schools, and educational centres of our
region, with the same goals of our project,
[…] it’s the way the make this live longer, at
least nationally”70
It follows that:
P11: Meeting external goals and objectives can
encourage external organisational ownership of
project results
This section shows the importance of the outputs
themselves, and of the systematic and effective
communication and interaction with external
actors. It also underscores the role of ownership
by external actors.
4.4 THE PROGRAMMATIC ENVIRONMENT
The operational activities of the funding body
affects the practicability of sustainability, whilst
an artificial market for grant funding and the
evolution of an abstract ‘project community’
affects how people perceive themselves as actors
68
I1R01
69
I2R03
70
I5R12
71
I2R03
within the programmatic environment, influencing
their approach to sustainability.
4.4.1 FUNDING BODY POLICY AND
DISCOURSE
Project actors are overwhelmed by bureaucratic
requirements and expectations of the funding body
at the proposal and close-down stages, leading
them to view sustainability as a bureaucratic
necessity, rather than a valuable opportunity.
“Coordinators are not encouraged to do
their best about exploitation, they are
encouraged to look at these deadly details of
their financial paper and bureaucratic
regulations and in the end they get
discouraged”71
“[During the] next 25% of our time, we will
still do some things that we think are good,
but we will spend the most time on satisfying
the reviewer”72
“Most organisations are seeing it as a must
and not as an opportunity, so maybe there
are ways to create it as a chance and to
promote it like this”73
Time at the end of the project is crucial to
sustainability work, yet resources are perceived to
be restrained by bureaucratic reporting demands.
“Sometimes the process of convincing the
end users of the benefits of the product or
results is too time consuming, and most of
the time you are just able to do that after the
project ends”74
“Exploitation often comes at the end once
you know what the real results are”75
Accepting P3
, that artificial planning facilitates
bureaucratically-successful projects, it follows
that:
72
I6R16
73
I4R08
74
I6R15
75
I6R16
Copyright © Paul Talbot 2015. All Rights Reserved 16
P12: The funding body is perceived as exercising
bureaucratic processes which inhibit genuine
planning by pressurising project actors towards
the realisation of contractual obligations
This premise notwithstanding, the emphasis on
sustainability within funding-body discourse is
perceived as increasing.
“The EC [is trying to] fund more and more
projects which they expect to have a serious
impact, and I think they’re putting more
attention on the issue”76
“A couple of years ago, if you were […]
using social media, you were thought to be a
genius. Now you really have to come up with
good ideas and sustainability has, I think,
because a major criteria”77
However there is a perceived lack of know-how
and limited support structures within the funding
body, and disaccord between this lack of support
and the expectations placed upon project actors.
“I always have the impression that this is
something on their list what they have to tell,
but they are happily surprised if somebody
has a kind of sustainability in their project
[…] they are like “can you tell us how to do
this sustainability, because we have no clue
about it”78
“It was extremely unfair to analyse projects
by this criteria, because the [funding body]
hasn’t set a standard […] and you don’t
have specialists capable of offering a help-
desk”79
“Lifelong learning is such a vast sector and
they cannot know everything. Maybe they
don't know what happens in [each specific
subject area] so sometimes the judgement
and the comments are quite superficial”80
76
I5R11
77
I4R09
78
I3R07
79
I3R06
Incompatibility within the funding policy, such as
innovation over sustainability or open-access over
marketization are perceived as confusing and
counter-productive:
“That project was not funded because it
wasn’t considered innovative [but] it’s not
about being innovative, it’s about being
sustainable”81
“On the one hand they said all products will
have reach a wide audience free of charge.
At the same time they said, to be sustained,
they allow you to make financeable courses
[…] but you never know what was in-
between the two”82
It follows therefore, that:
P13: funding body policies are perceived as
incompatible with sustainability
and
P14: the funding body is perceived as holding
unrealistic expectations of project actors
4.4.2 THE EUROPEAN PROJECT WORLD
The European LLP (possibly along with other
funding programmes) has established a closed-off
economic and professional domain with its own
identity distinct from both the private and public
sector. Respondents’ views suggested an artificial
market for project funding, where projects may
exist to support other projects, whilst some
produce material that is only, or mainly,
sustainable through grant funding.
“There is this handbook [about exploitation]
which is done in another European-funded
project”83
“You take the grant but then you work what
with what you have developed afterwards,
80
I6R14
81
I5R11
82
I1R01
83
I4R09
Copyright © Paul Talbot 2015. All Rights Reserved 17
like we have developed two in-service
courses [LLP funded courses] and we had
run the first one for about six years”84
Sustainability outside of this domain is valued
however EU funding may itself exist as a symptom
of market failure, necessarily financing public-
value goods.
“If the services or products are compatible
on the real market, not on the artificial
market of project ideas and proposals, it
means the project is extremely successful
and sustainable”85
“Either they are funded publically or you
are not going to be able to get funding for
them”86
This leads us to argue that:
P15: projects are funded through an artificial
grant-funding market not based on private
consumer preferences but on public-value policy
objectives
and
P16: structures are present whereby projects can
exist to serve the needs of others within an
artificial grant-funding market
Within this artificial market project actors may
identify themselves professionally in terms of
European projects, communities and networks
because they work primarily within this domain,
and may be part of various projects implemented
through different partnerships.
“We write projects, more than one or two, as
much as possible for each call for
proposal”87
84
I8R19
85
I4R10
86
I5R11
87
I3R06
88
I8R19
“If you work with organisations for a long
time, like you have done the 2nd or 3rd or
whatever project, now something special
develops between those partners”88
Events organised in the framework of the funding
programme tend to target project actors as a
distinct group, rather than as part of wider thematic
networks.
“The EU values [dissemination
conferences] as very important, so we have
to do these things, but I don't think they are
very efficient [...] we were thousands of
people talking for 10 minutes presenting our
projects, and I don't remember any of
them”89
“You have dissemination events for
teaching, for vocational education, for
research and so on [attended by] project
coordinators and project members, and
people from [the EU]. I think they could also
invite some [thematic] networks […] in my
subject there are lots of technological
platforms, but they are never invited to these
events”90
It may be that this strengthens the organisational
goal of project participation, rather than focusing
on complementing non-project goals (P2
),
somewhat supported in the idea that projects self-
perpetuate, and can result in the generation of new
projects.
“Strengthening of the networks created
during the project […] to further develop the
results obtained in the projects, or possibly
discuss new ideas and collaboration”91
“We have two projects where the
sustainability works really well. They are
both in [the same field] because we have
created not only one project but also a
follow-up project”92
89
I6R14
90
I7R18
91
I2R04
92
I3R07
Copyright © Paul Talbot 2015. All Rights Reserved 18
Accepting this, then:
P17: an artificial grant-funding market provides a
framework for organisations to establish
standard operational activities
However, respondents also identified intangible
capacity-building impacts from projects.
“Working on LLP projects and initiatives,
we increased our competence and our
awareness of sustaining the results of the
projects. The very real impact […] is to
make people working together from Europe.
Now if I make a phone call to Ireland or
Finland, there is someone that is on the line
[who knows me]”.93
“These people who are employed to carry
out the project together with us become
experts, and they can build up something for
the future […so] sustainability in terms of
personnel development […]it’s very
important to have a staff development
strategy”94
It follows, then, that:
P18: the LLP provided intangible long-term
impacts on project actors’ own staff and
organisational capacity
This section shows that funding body policies are
perceived as damaging to sustainability potential,
both in how they approach sustainability on a
practical level, and in how they allow the
perpetuation of non-sustainable projects through
an artificial grant-funding market, although the
LLP may serve as a general capacity building tool
for individual project actors and organisations.
5 DI S C U S S I O N
5.1 ANALYSIS OF FINDINGS
The data shows project actors’ perceptions of
factors influencing sustainability, whilst revealing
other systemic processes, demonstrating how
issues interlink, exposing deeper concerns.
P2
indicates different motivations for project
participation – acquisition of projects and pursuit
of non-project goals - P3
shows projects can be
contractually successful even when planning is
artificial, and P4
links these premises suggesting a
direct positive relationship between using projects
to achieve broader organisational goals and
genuine pro-sustainability planning. The
conclusion of this is that organisations should
manage projects with goals that complement their
organisational objectives. This is supported
throughout the data from various perspectives.
Firstly, sustainability is difficult when the broader
organisation is not involved, limiting the
integration of project outputs into other activities
93
I6R14
(P1
). Where focus is on project acquisition,
organisations may neglect to acquire a
sufficiently-detailed understanding of the external
environment (contrary to P6
), or to sufficiently
interact with external actors (contrary to P9
).
Secondly, organisational ownership appears to
precede individual ownership, suggesting that
even with commitment from individual project
actors – which is deemed important - the ability to
integrate outputs into the organisations operational
objectives is important (P5
) .
P11
provides that meeting objectives of external
organisations can also encourage their ownership
of results, suggesting there may be little difference
between encouraging external organisational
ownership and encouraging internal (partner)
organisational ownership. This highlights the
relevance of project isolation within the
organisation (P1
) but also raises the question of
whether we can extend to the internal organisation
those factors deemed important to the external
environment, namely the need for detailed
94
I8R20
Copyright © Paul Talbot 2015. All Rights Reserved 19
understanding (P6
) and systematic communication
(P9
) and encouraging individual connections to the
project and its outputs (P10
), ensuring that these
outputs meet professional standards and
expectations (P7
) and can be broadly applicable
(P8
).
This strengthens the conclusion that sustainability
requires understanding of the needs of those
required to facilitate sustainability, whether it be
the partner organisation or an external body.
Understanding these needs, and involving those
relevant parties (internal or external) outside of the
immediate project team during project
development may help to create projects with
sustainability potential by seeding ownership
outside of the project team.
In terms of the external programmatic
environment, there is a strong culture of blame-
shifting towards the funding body for
overwhelming bureaucracy (P12
) perceived policy-
incompatibilities (P13
), lack of support, and
unrealistic expectations (P14
). These concerns
may be valid, however they should not be so
crudely simplified.
On a practical level, bureaucratic demands may
distract project actors from sustainability work,
however this may reflect the problem of isolation
(P1
); if sustainability work could be shifted away
from project actors and towards operational staff
in the broader or external environment, then
project actors could focus on complicated
administrative processes, but whilst project actors
themselves retain stewardship of projects,
bureaucratic reporting will necessarily take
priority, if organisations wish to avoid negative
financial ramifications.
As such, the degree of bureaucracy experienced
may have effect of enabling and even encouraging
pragmatic, artificial planning through the
pervasive discourse of sustainability as necessity
rather than obligation, perhaps strengthened by
any the lack of support, incompatible policies and
unrealistic expectations identified by respondents.
The interview data suggests the existence of an
artificial market, limiting sustainability on the free
market (resulting fundamentally from market
failure) (P15
). Within this market organisations are
able to establish standard operational activities
(P17
), and may produce outputs to support the
needs of other projects within this market (P16
).
Taking into account P4
, that focusing purely on
project goals permits artificial, pragmatic planning
we may assume that organisations with this goal
are neglecting non-project objectives, however, if
the organisational objective is indeed to establish
standard operational activities within this market,
then the production of goods for this market
facilitates realignment between project
participation and non-project goals. Removal of
this artificial market would eliminate the
organisational goal. Whilst this supports the
argument that sustainable projects need to be
linked to long-term organisational goals, it may
also reflect policy failures on the part of the
funding body which essentially allow public funds
to be invested in the perpetuated acquisition of
more public funds with limited potential for the
mainstreaming of results outside of the funding
domain.
Whilst organisational goals remain propped up by
the funding body, there may be little incentive to
gear projects towards the pursuit of non-project-
based goals, limiting the extent to which
sustainability can be realistically built into, or
expected from, these projects.
However the revelation that the LLP may have
supported capacity-building in terms of European
exploitation and sustainability (P18
) presents the
view that it is not the outputs themselves that need
be the focus of sustainability, but rather the
intangible impacts on European cooperation and
coordination for those involved.
5.2 IMPLICATIONS OF THE FINDINGS
The findings have implications for the
management of LLP projects, the
operationalization of the funding programme, and
for best practice under the Erasmus+ programme.
They show that project sustainability requires
commitment and resources to implement results
after the funding. Aligning project goals with
partner- or external organisations’ own goals,
rather than seeing project goals as independent
objectives, is instrumental in securing these
resources, encouraging project actors to approach
planning and implementation with a view to
sustainability, seeing pro-sustainability practices
such as needs analysis and stakeholder
involvement as opportunities rather than
Copyright © Paul Talbot 2015. All Rights Reserved 20
bureaucratic hurdles. These efforts are essential
for the achievement both of project goals as
organisational goals and as public value goals, as
Bryson (2004) hypothesises “strategic
management processes that employ a reasonable
number of competently done stakeholder analyses
are more likely to be successful – that is, meet
mandates, fulfil missions and create public value –
than those that do not”.cxiii
Therefore project managers should align project
activities with long-term organisational goals,
focusing on building strong relationships with
those within their organisations or the external
environment whom may be able to integrate
results into their activities. Project teams should
avoid retaining complete ownership of projects,
but rather facilitating them within the organisation,
involving other on an operative basis, bridging
project and non-project activities, with a strong
view to providing strategic added-value.
The LLP appears to impose supposedly rational
policy processes on non-rational organisations.cxiv
Bureaucratic and conflicting demands encourage
pragmatic approaches to ensure contractual
fulfilment and financial remuneration,
exacerbating the principle-agent problem in which
“agents will seek ways to act in their own interests,
whatever contracts are written and whatever
monitoring is put in place”,cxv
creating hybrid
programmes attempting to satisfy all partiescxvi
,
and inhibiting pro-sustainability practices.
Although validity of this conclusion requires
verification, including from the perspectives of
those working in the funding body, factors such as
conflict between public-value policy demands and
expectations of mainstreaming along with the
evolution of a self-serving market for European
grant-funding appear to have further weakened
capacity and incentive to engage in genuine pro-
sustainability practices. Nonetheless, within this
artificial market, the evolution of a community of
‘Europeanised’ groups of sector professionals may
fulfil broader long-term policy objectives,
facilitating transnational cooperation and cohesion
and building capacity to work on a European level.
This highlights a need for the funding body to
provide better sustainability support for
organisations, with coherent, informed guidelines,
and finding ways to hold organisations
accountable for sustainable impacts. However
policy-makers should also reflect on the capacity-
building effects of projects and their contribution
to European strategy, building this into their
impact analyses. It is possible that problems faced
in sustainability are a symptom of Curtis’ (2004)
split between policy formation and
implementation; where projects are implemented
remote from the formulation of policy goals they
are intended to contribute to, the mental
frameworks underpinning the motivation and
rationale for acting are fundamentally different.cxvii
A critical analysis of policy-making processes at
the level of the funding-body would be necessary
to fully evaluate the material importance of
sustainability within decision-making frameworks
and in policy rhetoric, cxviii
and whether results are
fed back into the policy-making process.cxix
The findings have potential implications on how
the topic of project sustainability can be
approached. The data confirms many factors
raised by Savaya et al’s (2008) literature review,
however it departs from current discourse by
viewing project sustainability from a systems
perspective, highlighting the systemic interaction
between influencing factors rather than describing
them as independent processes.
These implications are based on empirically-
grounded hypotheses, not irrefutable theory.
Sustainability is complex and multifaceted.
Identified factors are perhaps neither sufficient nor
necessary conditions, but contribute to
sustainability in LLP projects. P5
indicates that
individual ownership may contribute to promoting
pro-sustainability practices, but without
organisational ownership, will be insufficient.
Individual ownership may influence
organisational ownership, although this would
depend on complex processes not addressed here.
Similarly, that bureaucratic demands or policy
incompatibilities present challenges for
sustainability does not mean that sustainability is
impossible, only that it is made more difficult -
there has been some degree of sustainability from
LLP-funded projectscxx
. This study does not test
each emerging factor for its respective impact, and
cannot determine the existence of sufficient or
necessary conditions.
The data reflects perceptions of a small number of
project actors (20), analysed through the
subjective interpretations of the researcher,
himself professionally active in the field of
enquiry, with his own biases and pre-conceived
Copyright © Paul Talbot 2015. All Rights Reserved 21
views. Interview participants represented a
response rate of 7%. That respondents chose to
participate may reflect a response-bias in that they
were interested in project sustainability; indeed
some have strongly-formulated views and
opinions. Responses ranged from descriptive
accounts of their organisational procedures and
what they perceive to impact sustainability, to
critical self-reflection, acknowledging
deficiencies in their own approaches, and blame
allocation, presenting passionate arguments about
what they perceive to be wrong. This may not
reflect respondents’ “true attitude or
sentiment”,cxxi
and we should “recognise that
informants can and do hold conflicting [and]
varying sentiments”.cxxii
Indeed the relevance of
this topic for those anxious of their projects failing
on sustainability criteria [i.e. “it was extremely
unfair to analyse projects by this criteria, because
the project officer or manager of the programme
hasn’t set a standard […] and for sure you don’t
have specialists that are capable of offering a
help-desk”] may have prompted more emotional,
defensive responses than if no emotional
connection to the topic was present. It is clear,
although patterns emerged from across the data,
that due to the inability to pursue iterative
theoretical saturation it might be difficult to
determine exactly how the data reflects the various
values, attitudes, opinions or emotional reactions
of respondents.
This study has revealed policy-relevant processes
in a formally neglected research field, but further
research is needed to verify and build on the
results. The processes identified are not
unimportant, and a greater understanding of these
processes may have significant value to
researchers and policy makers in all fields relating
to grant-funded projects and projects with public-
value policy objectives.
6 CO N C L U S I O N S A N D RE C O M ME N D AT I O N S
6.1 CONCLUSIONS
This study has identified sustainability as central
to the success of the LLP as a policy instrument.
In contrast to existing project-focused literature,
this study viewed projects within a systems
framework, identifying factors – often confirming
those identified within the literature – but
revealing potential interconnectedness of factors
within and between different operating
environments.
This study found that aligning project goals with
long-term organisational objectives is instrumental
in securing the capacity and resources to plan and
implement projects sustainably and continue
delivering impacts. Policy and processes at the
level of the funding body do little to encourage
pro-sustainability practices, and may have made it
easy for organisations to avoid systematic output
mainstreaming.
This study highlights clear implications of this for
both organisations and the funding body, namely
that the integration of projects into long-term goals
and broader organisational involvement (internal
or external) in project development and
implementation should be explicitly encouraged,
and that better support should be offered to project
actors, with coherent, informed guidelines and
greater accountability. However, whilst the LLP
fails to promote long-term impact of project
outputs, it promotes the Europeanisation of sector
professionals, promoting and facilitating
transnational cooperation and cohesion and
building capacity of members of the European
workforce to work on a European level, which
itself contributes to longer-term European political
strategies.
The findings are useful for project actors, agency
project officers and policy-makers as they reveal
systemic processes occurring in the
implementation of the LLP. Increasing
competition for grant-funding requires
organisations to demonstrate attention to
sustainability. This study identifies strategies to
make project outputs more valuable for
organisations and for European policy goals. At
the same time, increasing attention to project
sustainability within European policy discourse
demands practicable evaluation criteria by which
to make cost-effective policy and funding
decisions.
Copyright © Paul Talbot 2015. All Rights Reserved 22
Whilst these findings lack the concrete theoretical
foundations required to credibly influence
management and policy decisions, they provide a
starting point for verification through further
research and hopefully serve to sensitise the policy
and management research community to the
importance of the topic. Further research on the
policy-making process would also be helpful in
revealing why the LLP was selected as a means of
achieving policy, and the extent to which
sustainability played a role in this decision-making
process.
BI B L I O G RA P H Y
Ali, H., & Birley, S. (1999). Integrating deductive and inductive approaches in a study of new ventures and
customer perceived risk. Qualitative Market Research, 2(2), pp. 103-110.
Barriball, K. L., & White, A. (1994). Collecting data using a semi-structured interview: a discussion paper.
Journal of Advanced Nursing(19), pp. 328-335.
Bienzle, H., Hedman, E., Kirtley, R., Purokuro, V., Rus, C., Wiesinger, S., & Wilen, E. (2010). Survival
Kit. Managing Multilateral Projects in the Lifelong Learning Programme. Vienna: die Berater
Unternehmensberatungs.
Blumer, H. (1954). What is wrong with social theory. American Sociological Review, 19(13), pp. 3-10.
Bryson, J. (2004) What to do when stakeholders matter, in Public Management Review, 6, 1:26
Buchanan, D., Fitzgerald, L., & Ketley, D. (2005). No going back: a review of the literature on sustainining
organisational change. International Journal or Management Reviews, 7(3), pp. 189-205.
Centre for Financial and Management Studies, (2010). Unit 1: The Policy Analysis Model and Alternatives.
In C212: Public Policy and Strategy. SOAS, University of London
Charmaz, K. (2006). Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis.
London: Sage.
Cohen, M. D., March, J. G., and Olsen, J. P., (1972) A garbage can model or organisational choice,
Administrative Science Quarterly, 17,1. pp.1-25;
Council of the European Union, 2009. Council conclusions of 12 May 2009 on a strategic framework for
European cooperation in education and training (ET 2020). 28th May 2009 (2009/C 119/02)
Curtis, D. (2004). How we think they think: thought styles in the management of international aid. Public
Administration and Development, 24, pp. 415-423.
Dean, J. P., & Whyte, W. F. (1978). How do you know if the informant is telling the truth? In J. Bynner, &
K. Stribley, Social Research: Principles and procedures. London: Longman Group UK Ltd in
association with The Open Univeristy Press.
Deshpande, R. (1983). Paradigms lost: on theory and method in research in marketing. Journal of
Marketing, 47(Fall), pp. 101-110.
European Commission. Introduction to EU Funding. Retrieved from
http://ec.europa.eu/grants/introduction_en.htm. Last Accessed 27.8.14
European Communities. (2006). Sustainability of international cooperation projects in the field of higher
education and vocational training: Final Report. Luxembourg: Office for Official Publications of
the European Communities.
Fernandes, G., Ward, S., & Arajújo, M. (2013). Developing a Framework to Improve and Embed Project
Management Practices in Organisations”. Peocedia Technology(9), pp. 846-856.
Flynn, N. (2007) Public Sector Management, 5th Ed, London. Sage, p.128
Griffin, C. (2011). Policy and Lifelong Learning. In P. J. (ed), The Routledge International Handbook of
Lifelong Learning Oxon: Routledge. pp. 261-270
Halcomb, E. J., & Davidson, P. M. (2006). Is verbatim transcription of interview data always necessary?
Applied Nursing Research, 19, pp. 38-42.
Hogwood, B. W., and Gunn, L. A. (1984) Policy Analysis for the Real World, Oxford: OUP;
Ingle, M. D. (2005). Project Sustainability Manual: How to incorporate Sustainability into the Project Life
Cycle. Portland: Portland State University.
Copyright © Paul Talbot 2015. All Rights Reserved 23
Jarvis, P. (2011). The European Union and Lifelong Learning Policy. In P. Jarvis, The Routledge
International Handbook of Lifelong Learning Oxon: Routledge pp. 271-280
Kvale, S. (1996). InterViews: An introduction to qualitative research interviewing. Thousand Oaks: Sage
Publicaions Inc.
Lapadat, J. C. (2000). Problematizing transcription: purpose, paradign and quality. Social Research
Methodologiy, 3(3), pp. 203-219.
Lloyd, R., Baginsky, M., & Puchwein, I. (2006). Sustainability Research: The Big Lottery Fund. London:.
London: Big Lottery Fund.
Locke, K. (2001). Grounded Theory in Management Research. London: Sage.
McEllan, E., MacQueen, K. M., & Neidig, J. L. (2003). Beyond the Qualitative Interview: Data preparation
and Transcription. Field Methods(15), pp. 63-84.
Morgan, G, (1997) Images of Organisations. London. Sage.
Noblit, G. W., & Dwight Hare, R. (1988). Meta-Ethnography: Synthesizing Qualitative Studies. Qualitative
Research Methods Series 11. London: Sage Publications.
Nutt, P. C. (2002) Chapter 3: Decision.Making Processes Prone to Success and Failure. Why Decisions
Fail: avoiding the blunders and traps that lead to debacles. San Francisco. Berrett-Koehler for
discussions on the policy-making process
Ouane, A. (2011). UNESCO's drive for Lifelong Learning. In P. Jarvis, The Routlege International
Handbook of Lifelong Learning Oxon: Routledge. pp. 302-311
Pitagorsky, G. (2011, January 26). Project Managers are Change Managers. Retrieved from
http://www.projecttimes.com/george-pitagorsky/project-managers-are-change-managers.html.
Last accessed 27.7.14
Project Management Institute. (2004). A guide to the project management body of knowledge: PMBOK
guide (3. ed.). Pennsylvania: PMI Publications.
Public Policy and Management Institute. (2011). Interim Evaluation of the Lifelong Learning Programme
(2007-2013) Service Contract No. 2009-5173-PPMI. European Commission.
Punch, K. F. (1998). Introduction to Social Research Methods. London: Sage.
Quattrone, P., & Hopper, T. (2001). What does organisational change mean? Speculations on a taken for
granted category. Management Accounting Research, 12, pp. 403-435.
Robson, C. (2002). Sampling in Surveys - and Elsewhere. In C. Robson, Real World Research Oxford:
Blackwell. pp. 260-268
Roulston, K. (2010). Reflective Interviewing: a Gruide to Theory & Practice. London: Sage Publications.
Sarriot, E., Ricca, J., Yourkavitch, J., Ryan, L., & SHOUT. (2008). Taking the Long View: A Practical
Guide to Sustainability Planning and Measurement in Community-Oriented Health
Programming. Calverton, MD: Macro International Inc.
Savaya, R., Spiro, S., & Elran-Barak, R. (2008). Sustainability of Social Programs: A Comparative Case
Study Analysis. American Journal of Evaluation (29), pp. 478-493.
Senior, B., & Swales, S. (2010). Organisational Change (4 ed.). Harlow: Pearson Education.
Sequira, A. H., & Moharana, S. (2012). A Re-Look at Project Sustainability. Available at SSRN:
http://ssrn.com/abstract=2111031 or http://dx.doi.org/10.2139/ssrn.2111031.
Sheier, M. A. (2005). Is sustainability possible? American Journal of Evaluation, 26. 320-347
Shenhar, A. J., & Dvir, D. (2007). Reinventing Project Management: The Diamond Approach To
Successful Growth And Innovation. Massachusetts: Harvard Business Review Press.
TANGO. (2009). Sustainability of rural development projects: Best practices and lessons learned by IFAD
in Asia. IFAD.
Teddlie, C., & Yu, F. (2007). Mixed Method Sampling: A Typology with Examples. Journal of Mixed
Methods Research, 1, pp. 77-100.
Third International Roundtable on Managing for Development Results (2007) Planning and Budgeting:
Linking Policy, Planning and Budgeting – A background paper. Hanoi, Vietnam, 5-8 February
Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic
reviews. BMC Medical Research Methodology, 8(45).
Thorne, S., Jensen, L., Kearney, M. H., Noblit, G., & Sandelowski, M. (2004). Qualitative meta-synthesis:
reflections on methodological orientation and ideological agenda. Qualitative Health
Research(14), pp. 1342-1365.
Copyright © Paul Talbot 2015. All Rights Reserved 24
Turner, M., & Hulme, D. (1997). Organisational Environments: Comparisons, Contrasts and Significance.
In M. Turner, & D. Hulme, Governance, Administration and Development. Baisingstole:
Macmillan. pp. 22-61
Welch, C., & Piekkari, R. (2006). Crossing Language Boundaries: Qualitative Interviewing in International
Business. Management International Review, 46(4), pp. 417-437.
NOTES
i
Council of the European Union, 2009. Council conclusions of 12 May 2009 on a strategic framework for
European cooperation in education and training (ET 2020). 28th
May 2009 (2009/C 119/02); Jarvis, P.,
2011. The European Union and Lifelong Learning Policy. In P. Jarvis, The Routledge International
Handbook of Lifelong Learning Oxon: Routledge. pp. 271-280
ii
Griffin, C., 2011. Policy and Lifelong Learning. In: The Routledge International Handbook of Lifelong
Learning. Oxon: Routledge, pp. 261-270; Ouane, A., 2011. UNESCO's drive for Lifelong Learning. In:
The Routlege International Handbook of Lifelong Learning. Oxon: Routledge, pp. 302-311.
iii
The Lifelong Learning Programme ran from 2007-2013 and has now been succeeded by the programme
‘Erasmus+’, running from 2014-2020.
iv
http://ec.europa.eu/grants/introduction_en.htm (last accessed 27.8.14)
v
Public Policy and Management Institute, 2011
vi
Ibid:114
vii
Ibid:117
viii
Savaya, R., Spiro, S. & Elran-Barak, R., 2008. Sustainability of Social Programs: A Comparative Case
Study Analysis. American Journal of Evaluation, Issue 29, pp. 478
ix
Ingle, M. D., 2005. Project Sustainability Manual: How to incorporate Sustainability into the Project
Life Cycle.. Portland: Portland State University. §2-2
x
Ibid:421
xi
Sheier, M. A. (2005). Is sustainability possible? American Journal of Evaluation, 26, pp. 320-347
xii
Sequira, A. H. & Moharana, S., 2012. A Re-Look at Project Sustainability, Available at SSRN:
http://ssrn.com/abstract=2111031 or http://dx.doi.org/10.2139/ssrn.2111031
xiii
Savaya et al, 2008, pp. 478
xiv
Buchanan, D., Fitzgerald, L. & Ketley, D., 2005. No going back: a review of the literature on sustaining
organisational change. International Journal or Management Reviews, 7(3), pp. 189-205.
xv
European Communities, 2006. Sustainability of international cooperation projects in the field of higher
education and vocational training: Final Report., Luxembourg: Office for Official Publications of the
European Communities, p. edition notice
xvi
NHS Modernisation Agency, 2002 in Buchanan, Fitzgerald and Ketley, 2005, p. 190
xvii
Lloyd, R., Baginsky, M. & Puchwein, I., 2006. Sustainability Research: The Big Lottery Fund. London:,
London: Big Lottery Fund.
xviii
Sarriot et al (2008); Buchanan, Fitzgerald and Ketley (2005)
xix
Savaya et al, 2008, pp. 479-481
xx
Ibid:490
xxi
Ibid:491
xxii
Sarriot et al, 2008
xxiii
Turner, M. & Hulme, D., 1997. Organisational Environments: Comparisons, Contrasts and
Significance. In: Governance, Administration and Development. Baisingstole: Macmillan, p.22
xxiv
Buchanen et al, 2005
xxv
Senior, B. & Swales, S., 2010. Organisational Change. 4 ed. Harlow: Pearson Education; Quattrone, P.
& Hopper, T., 2001. What does organisational change mean? Speculations on a taken for granted category.
Management Accounting Research, Volume 12, pp. 03-435
xxvi
Project Management Institute, 2004. A guide to the Project Management Body of Knowledge. Third
edition. Pennsylvania: PMI Publications, p. 5
Copyright © Paul Talbot 2015. All Rights Reserved 25
xxvii
Sarriot et al, 2008, p.20
xxviii
Pitagorsky, 2011 ‘Project Managers are Change Managers’. http://www.projecttimes.com/george-
pitagorsky/project-managers-are-change-managers.html. Last accessed 27.8.14
xxix
Savaya et al, 2008, p.481
xxx
Ali, H., & Birley, S. (1999). Integrating deductive and inductive approaches in a study of new ventures
and customer perceived risk. Qualitative Market Research, 2(2), pp. 103-110.
xxxi
Morgan, G, 1997. Images of Organisations. London. Sage.
xxxii
Shenhar, A. J. & Dvir, D., 2007. Reinventing Project Management: The Diamond Approach To
Successful Growth And Innovation. Massachusetts: Harvard Business Review Press, p.222
xxxiii
Noblit, G. W. & Dwight Hare, R., 1988. Meta-Ethnography: Synthesizing Qualitative Studies.
Qualitative Research Methods Series 11. London: Sage Publications.
xxxiv
Thorne, S., Jensen, L., Kearney, M. H., Noblit, G., & Sandelowski, M, 2004. Qualitative meta-
synthesis: reflections on methodological orientation and ideological agenda. Qualitative Health Research
(14), pp. 1342-1365.
xxxv
Thomas, J. & Harden, A., 2008. Methods for the thematic synthesis of qualitative research in systematic
reviews. BMC Medical Research Methodology, 8(45).
xxxvi
Thorne et al, 2004, p. 1358
xxxvii
Buchanan, Fitzgerald, & Ketley, 2005, p.192
xxxviii
Ingle, 2005; Lloyd, Baginsky and Puchwein, 2006; TANGO, 2009. Sustainability of rural
development projects: Best practices and lessons learned by IFAD in Asia: IFAD; Sarriot et al, 2008;
Savaya et al, 2008; Bienzle, H. et al., 2010. Survival Kit. Managing Multilateral Projects in the Lifelong
Learning Programme. Vienna: die Berater Unternehmensberatungs
xxxix
European Communities, 2006; Lloyd, Baginsky and Puchwein, 2006
xl
Savaya, Spiro, & Elran-Barak, 2008; European Communities, 2006; Ingle, 2005
xli
Lloyd, Baginsky, and Puchwein, 2006; TANGO, 2009
xlii
Ingle, 2005; Lloyd, Baginsky, and Puchwein, 2006
xliii
Lloyd, Baginsky, and Puchwein, 2006
xliv
Sarriot et al, 2008
xlv
Lloyd, Baginsky, and Puchwein, 2006; Ingle, 2005; TANGO, 2009
xlvi
Ingle, 2005; TANGO, 2009; European Communities, 2006
xlvii
TANGO, 2009
xlviii
Ingle, 2005
xlix
Ingle, 2005; European Communities, 2006
l
Bienzle et al., 2010
li
Ingle, 2005; TANGO, 2009
lii
TANGO, 2009
liii
Savaya, Spiro, & Elran-Barak, 2008
liv
Sarriot et al, 2008
lv
TANGO, 2009
lvi
Sarriot et al 2008; Savaya, Spiro, & Elran-Barak, 2008
lvii
Lloyd, Baginsky and Puchwein, 2006
lviii
Ingle, 2005; Savaya, Spiro, & Elran-Barak, 2008; Lloyd, Baginsky and Puchwein, 2006
lix
Ingle, 2005
lx
Lloyd, Baginsky and Puchwein, 2006
lxi
European Communities, 2006
lxii
Lloyd, Baginsky, and Puchwein, 2006
lxiii
Ingle, 2005; Lloyd, Baginsky and Puchwein, 2006; Tango, 2009; Savaya, Spiro, & Elran-Barak, 2008;
Bienzle et al., 2010, Sarriot et al 2009; European Communities, 2006
lxiv
Bienzle et al, 2010
lxv
Ingle, 2005; Lloyd, Baginsky and Puchwein, 2006; Tango, 2009; Savaya, Spiro, & Elran-Barak, 2008;
Sarriot et al 2008
Copyright © Paul Talbot 2015. All Rights Reserved 26
lxvi
Lloyd, Baginsky and Puchwein, 2006
lxvii
TANGO, 2009
lxviii
TANGO, 2009; Lloyd, Baginsky, and Puchwein, 2006; Savaya, Spiro, & Elran-Barak, 2008
lxix
TANGO, 2009
lxx
Ibid
lxxi
Ibid
lxxii
Ingle, 2005; Lloyd, Baginsky, and Puchwein, 2006; Bienzle, et al., 2010; Sarriot et al 2008; European
Communities, 2006
lxxiii
Ingle, 2005
lxxiv
Lloyd, Baginsky, and Puchwein, 2006
lxxv
Lloyd, Baginsky, and Puchwein, 2006; European Communities, 2006
lxxvi
Lloyd, Baginsky, and Puchwein, 2006
lxxvii
Ibid
lxxviii
Ibid
lxxix
Ibid
lxxx
Public Policy and Management Institute, 2011
lxxxi
Blumer, H., 1954. What is wrong with social theory. American Sociological Review, 19(13), pp. 3-10.
lxxxii
Ali & Birley, 1999
lxxxiii
Fernandes, G., Ward, S. & Arajújo, M., 2013. Developing a Framework to Improve and Embed Project
Management Practices in Organisations”. Peocedia Technology, Issue 9, pp. 846-856
lxxxiv
Charmaz, K., 2006. Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis.
London: Sage
lxxxv
Senior & Swales, 2010, p. 5
lxxxvi
Senior & Swales, 2010, pp. 14-15.
lxxxvii
Senior & Swales, 2010, p. 14-15
lxxxviii
Locke, K., 2001. Grounded Theory in Management Research. London: Sage.
lxxxix
Punch, K. F., 1998. Introduction to Social Research Methods. London: Sage.
xc
Halcomb, E. J., & Davidson, P. M. (2006). Is verbatim transcription of interview data always necessary?
Applied Nursing Research, 19, p.39
xci
see, for example, the issue of meaning and linguistic diversity, §3.2.4
xcii
Deshpande, R., 1983. Paradigms lost: on theory and method in research in marketing. Journal of
Marketing, 47(Fall), pp. 101-110
xciii
Ali & Birley, 1999
xciv
Blumer, 1954 in Charmaz, 2006, p. 16
xcv
Charmaz, 2006, p. 17
xcvi
Fernandes, Ward, & Arajújo, 2013
xcvii
Teddie, C. & Yu, F., 2007. Mixed Method Sampling: A Typology with Examples. Journal of Mixed
Methods Research, Volume 1, pp. 77-100; Robson, C., 2002. Sampling in Surveys - and Elsewhere. In:
Real World Research. Oxford: Blackwell, pp. 260-268.
xcviii
Robson, 2002:265
xcix
Teddlie & Yu, 2007, p. 82
c
Teddlie & Yu, 2007
ci
Barriball, K. L. & White, A., 1994. Collecting data using a semi-structured interview: a discussion paper.
Journal of Advanced Nursing, Issue 19, pp. 329-330
cii
Roulston, K., 2010. Reflective Interviewing: a Gruide to Theory & Practice. London: Sage Publications.
ciii
Lapadat, J. C., 2000. Problematizing transcription: purpose, paradign and quality. Social Research
Methodologiy, 3(3), p. 210
civ
Lapadat, 2000
cv
Kvale, S. (1996). InterViews: An introduction to qualitative research interviewing. Thousand Oaks: Sage
Publications, Inc, p. 163
cvi
Halcomb & Davidson, 2006
Copyright © Paul Talbot 2015. All Rights Reserved 27
cvii
McEllan, E., MacQueen, K. M. & Neidig, J. L., 2003. Beyond the Qualitative Interview: Data
preparation and Transcription. Field Methods, Issue 15, p. 67
cviii
Welch, C. & Piekkari, R., 2006. Crossing Language Boundaries: Qualitative Interviewing in
International Business. Management International Review, 46(4), p. 418
cix
Tsang, 1998 in Welch & Piekkari, 2006, p. 422
cx
Welch and PIekkari, 2006
cxi
Ibid, p. 428
cxii
Interviews were conducted with 21 people. One respondent left the teleconference abruptly due to
technical difficulties having provided only superficial descriptive accounts of sustainability that could not
be probed further. These responses were not considered in the analysis, leaving a total of 20 respondents in
the final dataset.
cxiii
Bryson, J, 2004, What to do when stakeholders matter, in Public Management Review, 6, 1:26
cxiv
Centre for Financial and Management Studies, 2010. Unit 1: The Policy Analysis Model and
Alternatives. In C212: Public Policy and Strategy. SOAS, University of London
cxv
Flynn, N. 2007. Public Sector Management, 5th Ed, London. Sage:128
cxvi
Curtis, D. 2004. How we think they think: thought styles in the management of international aid. Public
Administration and Development, 24, pp. 415-423.
cxvii
Curtis, 2004
cxviii
See for example Hogwood, B. W., and L. A. Gunn, 1984. Policy Analysis for the Real World, Oxford:
OUP; Cohen, M. D., March, J. G., and Olsen, J. P., 1972. A garbage can model or organisational choice,
Administrative Science Quarterly, 17,1:1-25; or Nutt, P. C., 2002. Chapter 3: Decision.Making Processes
Prone to Success and Failure. Why Decisions Fail: avoiding the blunders and traps that lead to debacles.
San Francisco. Berrett-Koehler for discussions on the policy-making process
cxix
Third International Roundtable on Managing for Development Results, 2007. Planning and Budgeting:
Linking Policy, Planning and Budgeting – A background paper. Hanoi, Vietnam, 5-8 February.
cxx
For example the European Computer Driving License (EDCL), flagged by the funding agency as best
practice in sustainability
cxxi
Dean, J. P., & Whyte, W. F. (1978). How do you know if the informant is telling the truth? In J. Bynner,
& K. Stribley, Social Research: Principles and procedures. London: Longman Group UK Ltd in
association with The Open Univeristy Press, p. 188
cxxii
Ibid

More Related Content

What's hot

Data4Impact expert workshop. The Conceptual Framework of Societal Impact of R&I
Data4Impact expert workshop. The Conceptual Framework of Societal Impact of R&IData4Impact expert workshop. The Conceptual Framework of Societal Impact of R&I
Data4Impact expert workshop. The Conceptual Framework of Societal Impact of R&IData4Impact
 
The impact of Science Literacy delivery methods - what works?
The impact of Science Literacy delivery methods - what works?The impact of Science Literacy delivery methods - what works?
The impact of Science Literacy delivery methods - what works?NIDA-Net
 
WHAT CAN EXPERIENCE WITH CLUSTERS TEACH US ABOUT FOSTERING SMART SPECIALISATION?
WHAT CAN EXPERIENCE WITH CLUSTERS TEACH US ABOUT FOSTERING SMART SPECIALISATION?WHAT CAN EXPERIENCE WITH CLUSTERS TEACH US ABOUT FOSTERING SMART SPECIALISATION?
WHAT CAN EXPERIENCE WITH CLUSTERS TEACH US ABOUT FOSTERING SMART SPECIALISATION?Orkestra
 
Raffo genderhorizon2020
Raffo genderhorizon2020Raffo genderhorizon2020
Raffo genderhorizon2020Luigi Raffo
 
Towards Proactive and Flexible Agent-Based Generation of Policy Packages for ...
Towards Proactive and Flexible Agent-Based Generation of Policy Packages for ...Towards Proactive and Flexible Agent-Based Generation of Policy Packages for ...
Towards Proactive and Flexible Agent-Based Generation of Policy Packages for ...Araz Taeihagh
 
Embedding impact: effecting institutional change through the development and ...
Embedding impact: effecting institutional change through the development and ...Embedding impact: effecting institutional change through the development and ...
Embedding impact: effecting institutional change through the development and ...Julie Bayley
 
BAM IBNW ref impact workshop
BAM IBNW ref impact workshopBAM IBNW ref impact workshop
BAM IBNW ref impact workshopQi Cao
 
Ef mbu the_list_of_thesis_2012_2013(1)
Ef mbu the_list_of_thesis_2012_2013(1)Ef mbu the_list_of_thesis_2012_2013(1)
Ef mbu the_list_of_thesis_2012_2013(1)khanmuhamad
 
Understanding change through training for gender equality maram barqawi-full
Understanding change through training for gender equality maram barqawi-fullUnderstanding change through training for gender equality maram barqawi-full
Understanding change through training for gender equality maram barqawi-fullMaram Barqawi
 
ESRC Knowledge Exchange Opportunities
ESRC Knowledge Exchange OpportunitiesESRC Knowledge Exchange Opportunities
ESRC Knowledge Exchange OpportunitiesAberdeen CES
 
A virtual environment for formulation of policy packages
A virtual environment for formulation of policy packagesA virtual environment for formulation of policy packages
A virtual environment for formulation of policy packagesAraz Taeihagh
 
Where next for clusters & cluster policy
Where next for clusters & cluster policyWhere next for clusters & cluster policy
Where next for clusters & cluster policyJames Wilson
 
Karin Fröding NHPRC2013workshop app partnership for urban health promotion ju...
Karin Fröding NHPRC2013workshop app partnership for urban health promotion ju...Karin Fröding NHPRC2013workshop app partnership for urban health promotion ju...
Karin Fröding NHPRC2013workshop app partnership for urban health promotion ju...Høgskolen i Sørøst-Norge (HSN)
 

What's hot (20)

Data4Impact expert workshop. The Conceptual Framework of Societal Impact of R&I
Data4Impact expert workshop. The Conceptual Framework of Societal Impact of R&IData4Impact expert workshop. The Conceptual Framework of Societal Impact of R&I
Data4Impact expert workshop. The Conceptual Framework of Societal Impact of R&I
 
Evaluating transformative innovation policies. implications and challenges
Evaluating transformative innovation policies. implications and challengesEvaluating transformative innovation policies. implications and challenges
Evaluating transformative innovation policies. implications and challenges
 
The impact of Science Literacy delivery methods - what works?
The impact of Science Literacy delivery methods - what works?The impact of Science Literacy delivery methods - what works?
The impact of Science Literacy delivery methods - what works?
 
What can sustainability science offer for transformative policymaking
What can sustainability science offer for transformative policymakingWhat can sustainability science offer for transformative policymaking
What can sustainability science offer for transformative policymaking
 
Lessons from IFPRI Country Programs on Informing Policy Decisions and Strengt...
Lessons from IFPRI Country Programs on Informing Policy Decisions and Strengt...Lessons from IFPRI Country Programs on Informing Policy Decisions and Strengt...
Lessons from IFPRI Country Programs on Informing Policy Decisions and Strengt...
 
Logic Models
Logic ModelsLogic Models
Logic Models
 
WHAT CAN EXPERIENCE WITH CLUSTERS TEACH US ABOUT FOSTERING SMART SPECIALISATION?
WHAT CAN EXPERIENCE WITH CLUSTERS TEACH US ABOUT FOSTERING SMART SPECIALISATION?WHAT CAN EXPERIENCE WITH CLUSTERS TEACH US ABOUT FOSTERING SMART SPECIALISATION?
WHAT CAN EXPERIENCE WITH CLUSTERS TEACH US ABOUT FOSTERING SMART SPECIALISATION?
 
Raffo genderhorizon2020
Raffo genderhorizon2020Raffo genderhorizon2020
Raffo genderhorizon2020
 
Towards Proactive and Flexible Agent-Based Generation of Policy Packages for ...
Towards Proactive and Flexible Agent-Based Generation of Policy Packages for ...Towards Proactive and Flexible Agent-Based Generation of Policy Packages for ...
Towards Proactive and Flexible Agent-Based Generation of Policy Packages for ...
 
Embedding impact: effecting institutional change through the development and ...
Embedding impact: effecting institutional change through the development and ...Embedding impact: effecting institutional change through the development and ...
Embedding impact: effecting institutional change through the development and ...
 
Improving impact - do accountability mechanisms deliver results
Improving impact - do accountability mechanisms deliver resultsImproving impact - do accountability mechanisms deliver results
Improving impact - do accountability mechanisms deliver results
 
Bridging the gap between researchers and policy makers: GRIPP in Nigeria
Bridging the gap between researchers and policy makers: GRIPP in NigeriaBridging the gap between researchers and policy makers: GRIPP in Nigeria
Bridging the gap between researchers and policy makers: GRIPP in Nigeria
 
ICAR-IFPRI : Policy process and extension policy communications - Suresh Babu
ICAR-IFPRI : Policy process and extension policy communications - Suresh BabuICAR-IFPRI : Policy process and extension policy communications - Suresh Babu
ICAR-IFPRI : Policy process and extension policy communications - Suresh Babu
 
BAM IBNW ref impact workshop
BAM IBNW ref impact workshopBAM IBNW ref impact workshop
BAM IBNW ref impact workshop
 
Ef mbu the_list_of_thesis_2012_2013(1)
Ef mbu the_list_of_thesis_2012_2013(1)Ef mbu the_list_of_thesis_2012_2013(1)
Ef mbu the_list_of_thesis_2012_2013(1)
 
Understanding change through training for gender equality maram barqawi-full
Understanding change through training for gender equality maram barqawi-fullUnderstanding change through training for gender equality maram barqawi-full
Understanding change through training for gender equality maram barqawi-full
 
ESRC Knowledge Exchange Opportunities
ESRC Knowledge Exchange OpportunitiesESRC Knowledge Exchange Opportunities
ESRC Knowledge Exchange Opportunities
 
A virtual environment for formulation of policy packages
A virtual environment for formulation of policy packagesA virtual environment for formulation of policy packages
A virtual environment for formulation of policy packages
 
Where next for clusters & cluster policy
Where next for clusters & cluster policyWhere next for clusters & cluster policy
Where next for clusters & cluster policy
 
Karin Fröding NHPRC2013workshop app partnership for urban health promotion ju...
Karin Fröding NHPRC2013workshop app partnership for urban health promotion ju...Karin Fröding NHPRC2013workshop app partnership for urban health promotion ju...
Karin Fröding NHPRC2013workshop app partnership for urban health promotion ju...
 

Viewers also liked

Functional programming ruby mty
Functional programming   ruby mtyFunctional programming   ruby mty
Functional programming ruby mtyAdrianGzz2112
 
Tri-Star Steel Stock List Booklet
Tri-Star Steel Stock List BookletTri-Star Steel Stock List Booklet
Tri-Star Steel Stock List BookletMarisa Prevatt
 
Languages Behind Bars
Languages Behind BarsLanguages Behind Bars
Languages Behind BarsPaul Talbot
 
06UCL_NeedAssessment_SlumHomes
06UCL_NeedAssessment_SlumHomes06UCL_NeedAssessment_SlumHomes
06UCL_NeedAssessment_SlumHomesSobia Rafiq
 
Final Resume Nikita for neha review
Final Resume  Nikita for neha reviewFinal Resume  Nikita for neha review
Final Resume Nikita for neha reviewNikita Srivastava
 
Trade union civil rights (1865 – 1992)
Trade union civil rights (1865 – 1992)Trade union civil rights (1865 – 1992)
Trade union civil rights (1865 – 1992)SpiderSam2401
 
African American Civil Rights (1865 – 1992)
African American Civil Rights (1865 – 1992)African American Civil Rights (1865 – 1992)
African American Civil Rights (1865 – 1992)SpiderSam2401
 
Women civil rights (1865 – 1992)
Women civil rights (1865 – 1992)Women civil rights (1865 – 1992)
Women civil rights (1865 – 1992)SpiderSam2401
 
Cv paulina m_english2016
Cv paulina m_english2016Cv paulina m_english2016
Cv paulina m_english2016Paulina Morales
 

Viewers also liked (12)

Functional programming ruby mty
Functional programming   ruby mtyFunctional programming   ruby mty
Functional programming ruby mty
 
Resume 2015
Resume 2015Resume 2015
Resume 2015
 
Resume 1
Resume 1Resume 1
Resume 1
 
Tri-Star Steel Stock List Booklet
Tri-Star Steel Stock List BookletTri-Star Steel Stock List Booklet
Tri-Star Steel Stock List Booklet
 
Languages Behind Bars
Languages Behind BarsLanguages Behind Bars
Languages Behind Bars
 
DU_BrandCreative_8-1-12
DU_BrandCreative_8-1-12DU_BrandCreative_8-1-12
DU_BrandCreative_8-1-12
 
06UCL_NeedAssessment_SlumHomes
06UCL_NeedAssessment_SlumHomes06UCL_NeedAssessment_SlumHomes
06UCL_NeedAssessment_SlumHomes
 
Final Resume Nikita for neha review
Final Resume  Nikita for neha reviewFinal Resume  Nikita for neha review
Final Resume Nikita for neha review
 
Trade union civil rights (1865 – 1992)
Trade union civil rights (1865 – 1992)Trade union civil rights (1865 – 1992)
Trade union civil rights (1865 – 1992)
 
African American Civil Rights (1865 – 1992)
African American Civil Rights (1865 – 1992)African American Civil Rights (1865 – 1992)
African American Civil Rights (1865 – 1992)
 
Women civil rights (1865 – 1992)
Women civil rights (1865 – 1992)Women civil rights (1865 – 1992)
Women civil rights (1865 – 1992)
 
Cv paulina m_english2016
Cv paulina m_english2016Cv paulina m_english2016
Cv paulina m_english2016
 

Similar to Achieving Sustainable Impacts European Lifelong Learning Projects

EFFICIENCY PROBLEMS IN PUBLIC SECTOR PROJECTS PLANNING AND IMPLEMENTATION STAGE
EFFICIENCY PROBLEMS IN PUBLIC SECTOR PROJECTS PLANNING AND IMPLEMENTATION STAGEEFFICIENCY PROBLEMS IN PUBLIC SECTOR PROJECTS PLANNING AND IMPLEMENTATION STAGE
EFFICIENCY PROBLEMS IN PUBLIC SECTOR PROJECTS PLANNING AND IMPLEMENTATION STAGEEmils Pulmanis
 
The implementation of strategic
The implementation of strategicThe implementation of strategic
The implementation of strategicAbdul Qayyum
 
An Exploration of the Imperatives for Successful Strategy Execution in ODL In...
An Exploration of the Imperatives for Successful Strategy Execution in ODL In...An Exploration of the Imperatives for Successful Strategy Execution in ODL In...
An Exploration of the Imperatives for Successful Strategy Execution in ODL In...IOSRJBM
 
TED Talk – Rotomskiene – Role of Stakeholders
TED Talk – Rotomskiene – Role of StakeholdersTED Talk – Rotomskiene – Role of Stakeholders
TED Talk – Rotomskiene – Role of StakeholdersSociotechnical Roundtable
 
TheoryofChange2018_ENGLISH.PPT
TheoryofChange2018_ENGLISH.PPTTheoryofChange2018_ENGLISH.PPT
TheoryofChange2018_ENGLISH.PPTbasma750036
 
TheoryofChange2018_ENGLISH (1).PPT
TheoryofChange2018_ENGLISH (1).PPTTheoryofChange2018_ENGLISH (1).PPT
TheoryofChange2018_ENGLISH (1).PPTssuser867aeb1
 
A Review Of Program And Project Evaluation Models
A Review Of Program And Project Evaluation ModelsA Review Of Program And Project Evaluation Models
A Review Of Program And Project Evaluation ModelsNat Rice
 
Education policy aspects
Education policy aspects Education policy aspects
Education policy aspects Mafer Mena
 
Projects policy and digital literacy
Projects policy and digital literacyProjects policy and digital literacy
Projects policy and digital literacyJisc
 
Keynote 1. How can you tell if is not working? Evaluating the impact of educa...
Keynote 1. How can you tell if is not working? Evaluating the impact of educa...Keynote 1. How can you tell if is not working? Evaluating the impact of educa...
Keynote 1. How can you tell if is not working? Evaluating the impact of educa...CONUL Teaching & Learning
 
Monitoring evaluation
Monitoring evaluationMonitoring evaluation
Monitoring evaluationCarlo Magno
 
Scott2011 psgr literature_review
Scott2011 psgr literature_reviewScott2011 psgr literature_review
Scott2011 psgr literature_reviewDr Lendy Spires
 
06877 Topic Implicit Association TestNumber of Pages 1 (Doub.docx
06877 Topic Implicit Association TestNumber of Pages 1 (Doub.docx06877 Topic Implicit Association TestNumber of Pages 1 (Doub.docx
06877 Topic Implicit Association TestNumber of Pages 1 (Doub.docxsmithhedwards48727
 
A Framework for Assessing the Socio-Economic Impact of E-Gov.docx
A Framework for Assessing the Socio-Economic Impact of E-Gov.docxA Framework for Assessing the Socio-Economic Impact of E-Gov.docx
A Framework for Assessing the Socio-Economic Impact of E-Gov.docxsleeperharwell
 
1 Stakeholder Involvement In Evaluatio
1 Stakeholder Involvement In Evaluatio1 Stakeholder Involvement In Evaluatio
1 Stakeholder Involvement In EvaluatioTatianaMajor22
 
Open educational practice dimensions
Open educational practice dimensionsOpen educational practice dimensions
Open educational practice dimensionsgrainne
 

Similar to Achieving Sustainable Impacts European Lifelong Learning Projects (20)

EFFICIENCY PROBLEMS IN PUBLIC SECTOR PROJECTS PLANNING AND IMPLEMENTATION STAGE
EFFICIENCY PROBLEMS IN PUBLIC SECTOR PROJECTS PLANNING AND IMPLEMENTATION STAGEEFFICIENCY PROBLEMS IN PUBLIC SECTOR PROJECTS PLANNING AND IMPLEMENTATION STAGE
EFFICIENCY PROBLEMS IN PUBLIC SECTOR PROJECTS PLANNING AND IMPLEMENTATION STAGE
 
The implementation of strategic
The implementation of strategicThe implementation of strategic
The implementation of strategic
 
An Assessment of theEffectiveness of Monitoring and Evaluation Methods on the...
An Assessment of theEffectiveness of Monitoring and Evaluation Methods on the...An Assessment of theEffectiveness of Monitoring and Evaluation Methods on the...
An Assessment of theEffectiveness of Monitoring and Evaluation Methods on the...
 
Hefcereport gb
Hefcereport gbHefcereport gb
Hefcereport gb
 
An Exploration of the Imperatives for Successful Strategy Execution in ODL In...
An Exploration of the Imperatives for Successful Strategy Execution in ODL In...An Exploration of the Imperatives for Successful Strategy Execution in ODL In...
An Exploration of the Imperatives for Successful Strategy Execution in ODL In...
 
PEA.pdf
PEA.pdfPEA.pdf
PEA.pdf
 
TED Talk – Rotomskiene – Role of Stakeholders
TED Talk – Rotomskiene – Role of StakeholdersTED Talk – Rotomskiene – Role of Stakeholders
TED Talk – Rotomskiene – Role of Stakeholders
 
TheoryofChange2018_ENGLISH.PPT
TheoryofChange2018_ENGLISH.PPTTheoryofChange2018_ENGLISH.PPT
TheoryofChange2018_ENGLISH.PPT
 
TheoryofChange2018_ENGLISH (1).PPT
TheoryofChange2018_ENGLISH (1).PPTTheoryofChange2018_ENGLISH (1).PPT
TheoryofChange2018_ENGLISH (1).PPT
 
A Review Of Program And Project Evaluation Models
A Review Of Program And Project Evaluation ModelsA Review Of Program And Project Evaluation Models
A Review Of Program And Project Evaluation Models
 
Education policy aspects
Education policy aspects Education policy aspects
Education policy aspects
 
An Adaptive Learning Process for Developing and Applying Sustainability Indic...
An Adaptive Learning Process for Developing and Applying Sustainability Indic...An Adaptive Learning Process for Developing and Applying Sustainability Indic...
An Adaptive Learning Process for Developing and Applying Sustainability Indic...
 
Projects policy and digital literacy
Projects policy and digital literacyProjects policy and digital literacy
Projects policy and digital literacy
 
Keynote 1. How can you tell if is not working? Evaluating the impact of educa...
Keynote 1. How can you tell if is not working? Evaluating the impact of educa...Keynote 1. How can you tell if is not working? Evaluating the impact of educa...
Keynote 1. How can you tell if is not working? Evaluating the impact of educa...
 
Monitoring evaluation
Monitoring evaluationMonitoring evaluation
Monitoring evaluation
 
Scott2011 psgr literature_review
Scott2011 psgr literature_reviewScott2011 psgr literature_review
Scott2011 psgr literature_review
 
06877 Topic Implicit Association TestNumber of Pages 1 (Doub.docx
06877 Topic Implicit Association TestNumber of Pages 1 (Doub.docx06877 Topic Implicit Association TestNumber of Pages 1 (Doub.docx
06877 Topic Implicit Association TestNumber of Pages 1 (Doub.docx
 
A Framework for Assessing the Socio-Economic Impact of E-Gov.docx
A Framework for Assessing the Socio-Economic Impact of E-Gov.docxA Framework for Assessing the Socio-Economic Impact of E-Gov.docx
A Framework for Assessing the Socio-Economic Impact of E-Gov.docx
 
1 Stakeholder Involvement In Evaluatio
1 Stakeholder Involvement In Evaluatio1 Stakeholder Involvement In Evaluatio
1 Stakeholder Involvement In Evaluatio
 
Open educational practice dimensions
Open educational practice dimensionsOpen educational practice dimensions
Open educational practice dimensions
 

Achieving Sustainable Impacts European Lifelong Learning Projects

  • 1. Copyright © Paul Talbot 2015. All Rights Reserved 1 Achieving Sustainable Impacts in Projects Funded under the European Lifelong-Learning Programme (2007-2013) Paul Talbot, M.Sc. B.A. (Hons)* AB S T R A CT The European Lifelong Learning Programme is amongst various policy instruments providing community grants for projects related to the strategic policy goals of the European Union, but has been criticised for the fact that project results are rarely mainstreamed. Not unique to European funding, many publically- funded programmes fail to maintain long-term outputs. This study sets out to explore the factors facilitating and inhibiting impact sustainability of European Lifelong Learning projects, looking specifically at project sustainability from a systems perspective, using in-depth qualitative interviews with professionals currently coordinating projects under the Lifelong Learning Programme to identify factors within the internal organisational environment, its interaction with the external environment, and the influence of the programmatic environment, and exposing the impact of systemic relationships between these parts. Lacking the resources to conduct thorough iterative analysis and achieve theoretical saturation, this study uses inductive methods to generate empirically-grounded hypotheses, drawing provisional conclusions and implications concerning the policy and implementation of European Lifelong Learning projects, providing a starting point for validation and verification through further qualitative and quantitative research whilst hopefully serving to sensitise the policy and management research community, as well as project coordinators, evaluators and funding-body officers to the importance of the topic. *Paul Talbot is a project manager and consultant at die Berater® UnternehmehmensberatungsGmbH in Vienna, Austria and is also the project officer of the European Prison Education Association (EPEA). Talbot has coordinated, partnered and consulted on a range of projects in both the education and criminal justice field, and is particularly interested in the use of European-funded projects as tools for helping NGOs and civil-society organisations to develop capacity and work towards achieving their long-term strategic objectives. This paper is an edited version of Talbot’s Master’s dissertation (M.Sc. Public Policy and Management) at the University of London’s Centre for Financial and Management Studies. Contact: p.r.talbot@live.com 1 IN T RO D U CT I O N Lifelong learning is central to the strategic framework for European cooperation in Education and Training (ET2020),i as well as to broader international policy as a means of combating exclusion and inequality, and promoting economic sustainability,ii The European Lifelong Learning Programme (LLP)iii is amongst various policy instruments providing grants for “projects or activities in relation to European Union Policies”.iv There are few studies of the LLP, although an Interim Programme Reportv confirmed its operational effectiveness and policy alignment, but criticised project exploitation,vi commenting that results were rarely mainstreamed.vii Not unique to European funding, many publically- funded programmes fail to maintain long-term outputs, with perhaps “as many as 40% of all new programs [not being] sustained beyond the first few years after termination of initial funding”.viii Ingle (2005) argues that project effectiveness is “measured in terms of benefits at the end of the project funding cycle” and that projects “frequently stop delivering the desired benefits as soon as the money runs out”, due to “benchmarks
  • 2. Copyright © Paul Talbot 2015. All Rights Reserved 2 [being] defined in terms of effectiveness, neglecting institutional aspects concerning the capacity to sustain the delivery of benefits after donor funding ends”,ix somewhat echoing Curtis’ (2004) criticism of centralised policy objectives being turned into instruments for implementation by other parties, subject to output measures with “unintended consequence[s] of preventing better or more appropriate standards or measures, or even different products, from emerging in response to learning experience during the intervention”.x Project sustainability should be central to evaluation,xi yet Sequira and Moharara (2012) argue that the qualitative nature institution- building objectives are inherently difficult to evaluate and neglected in favour of quantifiable financial and economic indicators,xii whilst Savaya et al (2008) argue that “for sustainability to become a routine component of program evaluation, there is a need for greater clarity about obstacles to sustainability and their causes”.xiii Sustainability is amongst key appraisal criteria in the LLP, however the realisation of sustainability has not yet been critically investigated in this policy context. Targeted research is critical for a better understanding of project management processes, and their relationship to policy realisation. Understanding what influences project sustainability will be useful for project applicants developing proposals, and for European decision makers in conducting project appraisals, and may help in understanding how funding policies can be optimised to increase the effectiveness of such policy instruments as the LLP. 2 LI T E R AT U RE RE V I EW 2.1 DEFINITIONS AND DIMENSIONS OF SUSTAINABILITY Limited attention is given to impact sustainability across the management disciplines.xiv A project may be sustainable “when it continues to deliver benefits to the project beneficiaries and/or other constituencies for an extended period after […] financial assistance has been terminated”,xv and where “new ways of working and improved outcomes become the norm.”xvi It may include the sustained use of facilities and infrastructure, continuation of established services or of approaches piloted through existing services, utilisation by staff or beneficiaries of newly- acquired skills, the adoption of ideas at a policy level, the development and continuation of partnerships and collaborations either for continued service delivery or for the sake of capacity building, sustained behavioural change, or the integration of new information or perspectives into the accepted body of knowledge of a particular field.xvii It is a dynamic process within a local system in which interdependent factors need to be balanced and is heavily influenced by cultural, social and economic factors.xviii 2.2 STATE OF THE LITERATURE A limited pool of programmatic and research literature reveals a range of relevant issues. An extensive review by Savaya et al (2008) identifies various factors based around (1) project design and implementation, such as having clear sustainability plans, securing financial resources or developing financing plans, disseminating evidence of success, adopting to changing environments, having sufficiently-trained human resources, and having ongoing evaluation; (2) the organisational setting, such as organisational stability and flexibility, having program champions within the organisation, managerial support and flexibility and integration into the organisation; (3) the broader community, such as having community support for the program, political legitimation, and an appropriate socio- economic context, and; (4) the funding body, such as assuming responsibility and providing support and allocating capacity-building resources to recipient organisations.xix The results of their own study highlighted host-organisation leadership as a critical factor, arguing “the heads of the host organisations of [sustainable programs] fought hard to keep the programs alive and understood what was needed to do so” and that “the decisions of the host organisation leadership […] ensured sustenance of the surviving programs by giving them high priority and integrating them in the organisation’s structure”xx but found that a number of external factors such as competing programs, power relations or government policiesxxi could threaten sustainability. Unfortunately this literature is contextually removed from the current
  • 3. Copyright © Paul Talbot 2015. All Rights Reserved 3 research problem. It is focused primarily on evaluation practice, taking a static view of program environments, is often disproportionately based on projects in the health-care sector, and fails to reflect on policy processes. However, a broader search of the literature reveals a relationship between project sustainability, change management and organisational systems, providing a new approach to classifying the literature on sustainability outside of their original thematic and programmatic context. 2.3 PROJECT SUSTAINABILITY AS A SYSTEMS PROCESS Emerging from the literature review were factors that could be ‘categorised’ by their occurrence in the operational environment; the internal organisation, responses to the external operational environment, impacts on the external operational environment, or processes in the programmatic environment, that is, policy and operations at the level of the funding body. Figure 1 – The literature categorised within an organisational system Sustainability is subject to interdependent components and some already see sustainability as part of a local system.xxii Organisations managing projects, like all organisations, “exist in and relate to environments that affect their operations”.xxiii Sustainability is discussed at the level of organisational change,xxiv which may be variously defined as transformations within an organisation’s structure and operations, brought about by environmental instability.xxv This is conceptually different from a project, which is seen as “a temporary endeavour undertaken to create a unique product, service or result”.xxvi However, where Buchanen et al (2005) see organisational change as something that either persists or decays, as a result of various factors, then strong correlations can be drawn with the project concept. Both projects and organisational change activities seek to produce outcomes that disrupt existing systems, and require attention to structural and systemic issues.xxvii Pitagorsky (2011) claims that “project managers, to be effective, must be competent change managers” but also sees change management as a project activity.xxviii This supports a systems view of project sustainability, and whilst systems theory has weaknesses, it serves here only as a lens through which to categorise relevant literature. Savaya et al (2008) argue that the factors they identified “should be reflected in actions and processes designed to enhance sustainability”;xxix seeing sustainability within a more ‘holistic’ theoretical frameworkxxx of systemic environmental interactions to which these actions and processes are exposedxxxi provides a means for distinguishing between sustainability factors, forming a “context-free theoretical framework”xxxii through which to contextualise
  • 4. Copyright © Paul Talbot 2015. All Rights Reserved 4 the factors on a more general level, avoiding context-driven assumptions. Meta- ethnographiesxxxiii or meta-synthesesxxxiv pull corroborating concepts together whilst highlighting those ‘refutational’ concepts,xxxv producing interpretations that “will not be found in any one research report but, rather, [which are] derived from taking all of the reports in a sample as a whole".xxxvi Whilst there may be limited scope for a truly systematic review due to the ambiguous and multifaceted nature of sustainability as a research concept,xxxvii this approach provides clarity to the researcher’s thought processes, offering structure to the interviews, helping to avoid imposing theoretical ideas relating to the research question. 2.4 CATEGORISATION OF SUSTAINABILITY LITERATURE WITHIN A SYSTEMS MODEL There is no scope within this paper to provide a detailed qualitative description of the constructs identified, however they are very briefly presented as a means of supporting the contextualisation of project sustainability within a systems perspective. 2.4.1 INTERNAL ORGANISATIONAL ENVIRONMENT Project sustainability can be affected explicitly by activities in the internal organisational environment. Sustainability should be central to project strategies, influencing all aspects of the project cycle,xxxviii and require experienced and effective management teamsxxxix and mechanisms for effective planning and task allocation.xl The planning and pursuit of sustainability must be supported by meaningful commitment, not only forced by external actors,xli and be subject to monitoring, evaluation,xlii and performance measurementxliii against a consistent set of sustainability indicators across a range of project system components.xliv 2.4.2 RESPONDING TO OPERATIONAL ENVIRONMENTS Project sustainability can be affected by the activities conducted in response to external stimuli. Projects need certain conditions in the environment to sustain benefits,xlv which need to be thoroughly explored and understood.xlvi Involving stakeholders and project participants at every stage of the design processxlvii helps identify their various roles, interests and influencesxlviii and may help ensure that project outputs are sufficiently meeting stakeholder demands.xlix Projects should seek to align with policy directionsl within a supportive and enabling policy and regulative environmentli with political sustainability and government commitment,lii securing policy support and legitimation.liii Factors relating to the socio-economic context such as the strength of civil society and empowerment of women,liv local market conditionslv and the natural environmentlvi may affect a project’s sustainability prospects. 2.4.3 INFLUENCING OPERATIONAL ENVIRONMENTS Project sustainability can be affected by the way project actors seek to influence behaviour or events in the external operational environment. Projects may be designed to leave a legacy of outputs with mainstreaming potentiallvii requiring minimal further financial investment.lviii Ensuring sufficient resources to sustain project outcomes are seen as critical in the sustainability process.lix Projects with limited access to funding resources may face problems.lx Some project activities will be unfeasible when no further funding is providedlxi and so project implementers need to create the ability to attract further funding from external sources.lxii Ownership of project outcomes by the institutional environment, and particularly from those expected to continue project delivery is discussed in depth;lxiii projects should actively seek to find an ‘institutional home’, and transfer results to the institutional environment.lxiv Projects need to build the capacity of local institutions to be able to continue delivering project outcomeslxv ensuring infrastructural capacity for service deliverylxvi and technical maintenance,lxvii but also need to encourage stakeholders to independently take advantage of project outputs, which requires capacity at the user or beneficiary level.lxviii Although this aspect has received relatively little attention in the literature, it may include resilience to adapt to change,lxix risk management capacitylxx and the ability to preserve natural environmental resources.lxxi Stakeholders need to be made aware of project activities and outcomes though sufficient advertising of the project and its success or impact,lxxii especially where stakeholders
  • 5. Copyright © Paul Talbot 2015. All Rights Reserved 5 critical to sustainability may be less involved in implementation and have little exposure to the project.lxxiii 2.4.4 PROGRAMMATIC ENVIRONMENT Project sustainability can be affected by forces acting within the programmatic environment, which includes policy makers, programme managers and project officers amongst others. Support on a programmatic level can help to achieve sustainability aims,lxxiv through helping develop sustainability plans, and develop staff capacity.lxxv Problems deriving from the design and implementation of funding programmes will impact sustainability potential. For example, project funding periods may be too short to build strong-enough relationships with necessary partners,lxxvi or long lead-times between project application and implementation may mean initial plans are no longer feasible due to situation changes.lxxvii Programmes emphasising innovation and demanding ambitious, complex outcomes, often on an unrealistic timescale may inhibit sustainability.lxxviii Sustainability forms a more significant part of project evaluations where it is reflected as a core value in the programme and emphasised throughout the project delivery cycle.lxxix 3 RE S E A R C H AP P R O A C H A N D TO O L S 3.1 RESEARCH QUESTIONS This study explores factors influencing sustainability in European LLP projects. The Interim Evaluation of the LLP criticises project exploitation and mainstreaming.lxxx A review of literature across diverse project fields revealed factors influencing project sustainability within the following settings: (1) Internal Organisational Environment (2) Responding to Operational Environments (3) Influencing Operational Environments (4) Programmatic Environment These are not theoretical bases for sustainability in the LLP, but rather ‘sensitising concepts’,lxxxi ‘constructs’lxxxii or ‘salient factors’lxxxiii providing direction to data collection, serving as ‘points of departure’ for exploring the perspectives of project actors in this field,lxxxiv and can be used to formulate four initial questions, focused on the LLP. Q1. What are the internal organisational factors which facilitate or inhibit project sustainability? (Internal organisational environment) As identified in the literature review, organisational capacity and strategic orientation impact the sustainability of project outcomes. Furthermore, the ways in which people define and measure project success may neglect sustainability elements. Coded data emerging from the interviews is analysed for its implications on the relationship between project sustainability and the organisational environment. An ‘organisational factor’ refers to any aspect located within the project organisation itself. A systems view of organisations sees components such as management, strategy, goals, structure, operations and technology as well as organisational culture, politics and leadership as forming the internal environment.lxxxv Q2. What are the processes by which individuals and organisations respond to external environments to promote project sustainability? (Responding to operational environments) The literature identifies a range of factors associated with the way projects respond to external environments, however given the context- driven nature of existing work, there are few generalizable explanations for these interactions. This study explores how project organisations interact with the external environment. Coded data emerging from the interviews is analysed for its implications on the relationship between project sustainability and the processes involved in responding to the external environment. Processes may include activities responding to externally- imposed conditions, including political, economic, socio-cultural, technological, legal or ecological conditions.lxxxvi
  • 6. Copyright © Paul Talbot 2015. All Rights Reserved 6 Q3. What are the processes by which individuals and organisations create change in external environments to promote project sustainability? (Influencing operational environments) Long-term project sustainability requires changes in the operating environment. Coded data emerging from the interviews is analysed for its implications on the relationship between project sustainability and the processes involved in influencing external environments, including those in the political, economic, socio-cultural, technological, legal or ecological environmentlxxxvii as mentioned above. Q4. How does funding programme structure itself facilitate or inhibit project sustainability? (Programmatic environment) Funding-programme structures appear to influence pro-sustainability practices. This study looks at how the funding programme itself affects the sustainability potential of projects. Coded data emerging from the interviews is analysed for its implications on the relationship between project sustainability and the programmatic environment. Policy content, administrative processes, contractual obligations, the flexibility and length of projects and the perceived degree of support from the funding body may be amongst the topics discussed. 3.2 METHODOLOGY AND LIMITATIONS Grounded theory - used to make and understand observations through a developing conceptual framework,lxxxviii testing and refining theoretical concepts that it generates, and building theories which can later be tested against further datalxxxix - helps gain insights into previously neglected areas. However resource restraints make theoretical saturation difficult to achieve in this study. Interviews are used to generate verbal data, although as Halcomb and Davidson (2006) reason, “the richness and depth of these data will vary according to the specific research design and the research question that the investigation is seeking to address.”xc To fully explore the values, meanings and beliefs of project actors is realistically outside the practical scope of the study,xci and although findings are grounded in the available data, they only reflect one round of analysis and cannot reach irrefutable theoretical conclusions. They can, however, provide data on a previously neglected topic, and establish a starting point for more detailed research; rather than producing full grounded theory, this study produces empirically-grounded hypotheses for further testing. 3.2.1 THE ROLE OF THEORY The research falls outside a purely deductive- inductive dichotomy.xcii Existing theory, albeit rooted largely in grey-literature, presents a framework for pro-sustainability practices in project management. This theory, however, is not appropriate to the circumstances surrounding the current research question; the study is not concerned with general, proscriptive sustainability practices, but rather in the organisational factors that influence these practices in the specific context of the European LLP. In their study on entrepreneurialism, Ali and Birley (1999) had enough ‘theory’ to formulate a deductive research design, but this had not yet been applied to the situation being researched, compelling them to incorporate an inductive approach into a deductive design.xciii Their argument is that existing research can be used to develop constructs – rather than variables – helping to provide focus to the research, whilst opening up the discussion to different themes, not previously considered, helping establish a context for the research, determining questions to be asked and the assisting in the tabulation of the interview data (Ali & Birley, 1999). In this current study theoretical constructs identify issues deemed relevant by researchers and practitioners in a range of contextualised project settings (Ali & Birley, 1999), whilst the research itself explores the relationship between these constructs and the facilitating or inhibiting factors within participating organisations. Blumer (1954) coined the term ‘sensitising concepts’ to describe the initial ideas that “sensitize you to ask particular kinds of questions about your topic”.xciv Charmaz (2006) uses these concepts, derived from background assumptions and disciplinary perspectives as “points of departure to form interview questions, to look at data, to listen to interviewees, and to think analytically about the data”.xcv Fernandes et al (2013) take a similar perspective in identifying an ‘initial framework of salient factors’ from the literature on improving and embedding project management practices, and then conducting interviews in order to identify new
  • 7. Copyright © Paul Talbot 2015. All Rights Reserved 7 factors, confirm the factors identified and to remove less important factors.xcvi 3.2.2 SAMPLING Purposive- sequential- or theoretical sampling are given as appropriate approaches to grounded theoryxcvii where the researcher “[carries] out initial sampling, and from analysis of the results [extends] the sample in ways guided by their emerging theory”xcviii or “examines particular instances of the phenomenon of interest so that he or she can define and elaborate on its various manifestations”,xcix however since this research is concerned with European LLP project implementation broadly, the full population of European LLP project actors were of interest. Using purposive sampling of all individuals who coordinated LLP projects in 2012 and 2013, 285 potential interview participants were approached, and offered entry into a draw to win a €50 book voucher, and 21 individuals agreed to participate. The sample of 3 higher-education managers, 6 project managers at private training providers, 1 at a public training provider and 1 at an NGO, 7 university academics and 3 representatives of independent research centres contained individuals with a representative range of characteristics, covering the public and private sectors. This was the most financially- and time- efficient method for sampling this population,c although response bias may limit the generalizability of the results, as considered in the discussion of results. 3.2.3 CONDUCTING THE INTERVIEWS Semi-structured interviews are regarded as being “well-suited for the exploration of attitudes, values, beliefs and motives” and can help explore “the perceptions and opinions of respondents regarding complex and sometimes sensitive issues and enable probing for more information and clarification of answers”.ci They were conducted through a mix of 6 individual and group interviews, conducted by international teleconference (5 interviews with 12 people) and face-to-face meeting (1 interview with 2 people). Attention was made to ensure that all questions were as open as possible and to avoid leading questions. An interview guide of open and non- leading questions, initially piloted with colleagues, and refined for use in the formal interviews, is used to prompt respondents to discuss the processes involved in project sustainability from various different perspectives. The guide evolved over the course of the interviews, helping focus the questions and minimise non-relevant interview data, and questioning followed Roulston’s (2010) suggestions for delivering open questions and responding to interviewees.cii 3.2.4 TRANSCRIPTION AND DATA ANALYSIS Epistemological concerns dictate whether we value transcripts “[corresponding to] the original conversational event” or whether we prefer “re- presentations and interpretations”.ciii Interpretivist researchers battle with distortions of meaning.civ Transcriptions are “artificial constructions”,cv emerging from interviewer perceptionscvi and decisions as to what information is required and the value of its analytical contribution.cvii The current study involves cross-cultural and multi- lingual groups of respondents. Welch and Piekkari (2006) write that language is “critical to understanding the qualitative research interview” and that “language, symbols and discourses play [a significant role] in shaping meaning and power in interviews”.cviii Tsang (1998) argues that researchers should communicate in the respondent’s language, enabling full expression, good rapport and cultural understanding.cix Resource constraints, however, require that English be used as lingua franca in the interview process. This presents a challenge to the transcription of data punctuated with lexical and grammatical inaccuracies, erroneous pronunciation and imperfect discourse management, but also to the analysis of data.cx Responses may not reflect an authentic record, and will lack “subtle nuances” that come with “being confident enough to formulate extended and individualised responses”.cxi Furthermore, most respondents were interviewed through low social- presence telephonic media without the contextual nuances afforded by non-verbal interaction, further questions the value of the in verbatim transcription, limiting the empirical potential of the research. However, seeking to create empirically-grounded hypotheses rather than formal theory, this approach still enables the study to highlight key factors, establishing the groundwork for further investigation.
  • 8. Copyright © Paul Talbot 2015. All Rights Reserved 8 4 IN T E RV I EW RE S U LT S Eight in-depth interviews, with 20 respondentscxii yielded a broad range of data. Initial and focused- coding revealed that LLP project actors perceive a number of factors influencing sustainability. Factors are described here in relation to the interview data, and discussed more analytically in the next chapter. 4.1 THE INTERNAL ORGANISATIONAL FACTORS Internal organisational factors were identified at the level of the partner organisation and of the project consortium. 4.1.1 ORGANISATIONAL PARTICIPATION AND OWNERSHIP Discussing allocation of work resources, respondents experienced projects being delivered by semi-autonomous groups, distinct from operational management. “The idea, main concept and much of the idealising comes from the staff involved”1 “Public relations, marketing and dissemination, you need professionals for that, but we are an organisation where they leave it to the [scientists] and that's not good, we should do that together”2 “The organisation’s bosses learnt about the project issues from [external stakeholders]”3 Some statements indicate that involving the broader organisation in the project content may promote sustainability. “If your organisation doesn’t support the project, it’s just your work, not the 1 I2R04 – Assistant University Professor 2 I6R16 – Lecturer at a University of Applied Science 3 I7R17 – Consultant and project manager in higher- education sector 4 I7R17 5 I5R11 – Project manager at a public research centre organisation's work [...] if the organisation supports the project, […] it’s about supporting the ideas, sharing the ideas of the project”4 “A degree programme doesn’t only consist of [the project actors], you are usually 15 people […] how do you reach these people?” 5 “[Sustainability can be improved by] working on the dissemination structure or policy of a project at a general company level […] and maybe share better the resources you have to make it more effective”6 Problems occur when projects become reliant on individuals and their project know-how. “Everybody brings different skills to a project, so if somebody is missing, the skill is missing”7 “[Those] involved in the management and production of the results are there, that helps a lot because you have the know-how in place and you can use it later on”8 “You need one person at least for sustainability who has the contacts, who works with the material”9 Problems occur because projects finance project teams. Expertise develops within isolated groups, sometimes including external-contract staff, preventing crossover into the organisation, as nobody outside of the project team has ownership, or stewardship, of project results. 6 I5R12 – Independent researcher and project coordinator in the higher-education sector 7 I4R08 – Project manager at a state-funded adult education provider 8 I2R03 – Managing director of a private consultancy provider 9 I3R07 – Project manager at a private education, training, and consultancy provider
  • 9. Copyright © Paul Talbot 2015. All Rights Reserved 9 “It’s very difficult to get financing [once the project] has ended, at a national or even a local level, […] and time, more than anything, needs to be paid somehow”10 “It’s difficult to implement [exploitation after the project period] because we are continuously involved in new projects”11 “If you don’t have a steward then it doesn’t work […]We always have a mixed team of internal and external people that we fund with the project, and we can always make sure that […] somebody from the internal personnel becomes the steward”12 It follows that: P1: Integrating project outputs into activities outside of the project goals requires the involvement of people in the day-to-day operations of the organisation 4.1.2 GOALS AND MOTIVATIONS Differential foci on acquiring and completing projects, and pursuing long-term non-project goals affect the planning, implementation and capacity to exploit project results. Focus may lie on what the project itself brings the organisation in money or opportunities to complement present activities. “My boss’s first question is ‘how much money do we earn’“13 “You write a project based on your competences as an institution, but from a sustainability point of view, you have no idea of the impact that you will have”14 Alternatively, projects may be tools to achieve non-project objectives, such as contributing to long-term research agendas, supporting the 10 I5R12 11 I6R15 – Project manager and consultant at an innovation management firm 12 I8R20 – University Professor, Dean of Faculty 13 I4R08 14 I2R02 – Project manager in the higher-education sector development of curricula, becoming competitive on the real market, or creating resources to build long-term capacity. “We pursue projects with vested interests, to improve the situation of our students and our teachers”15 “Our intention [with our project] is to become competitive on the real language market”16 “There is a long-term strategy behind all these projects, it’s not just […] to get some extra money […] but it should be an integral part of long-term strategy” 17 It follows that: P2: Organisations have various motivations for participating in projects, including project acquisition and pursuit of non-project goals. 4.1.3 GENUINE AND ARTIFICIAL PLANNING Project actors need to plan for sustainability in the proposal, during and after the project, however genuine planning and artificial planning are not easily distinguishable. Genuine planning may require constant long-term thinking, joining only projects with sustainability potential, building sustainability into product design, or including sustainability amongst project monitoring criteria. “If there is a way to add this dimension, then it makes sense […] to think about, ok, what is the goal of the project, and how can we exploit that, or how can we use that afterwards”18 “We’ve been looking at how we can start to integrate concepts such as open access, free 15 I8R20 16 I4R10 – Project manager at a European NGO 17 I3R05 – University Professor at a University of Applied Science 18 I8R19 – Managing director and proprietor of private training, coaching and consultancy firm
  • 10. Copyright © Paul Talbot 2015. All Rights Reserved 10 use, create commons, and things that enable some of the results to go a little further”19 “Monitoring is good for sustainability by helping develop milestones for what is needed for sustainability and distributing them throughout the project”20 However this may still reflect artificial planning. There is no accountability for sustainability planning yet the planning itself is compulsory. Respondents regularly discussed projects using expressions such as ‘have to’, ‘forced to do’ and ‘expectations’. “Why do we do these things? First of all because it’s compulsory to report it to Brussels”21 “[the question is] how do I fill this in according to the expectations […] and not so much is the idea good or not”22 Project acquisition invites pragmatic proposal writing, based on current possibilities, and subject to time constraints, limiting the capacity to build sustainability systematically into the project. “It’s very important to think about [sustainability] when writing projects. Sometimes you just copy and paste something that you’ve written on sustainability in another project proposal”23 “We try to analyse the invitations we receive and see if it fits with our core activities and past experiences, and especially if you are able to involve the future end users [...] the problem is sometimes we don't receive enough information to do that kind of analysis”24 19 I2R04 20 I2R04 21 I1R01 – Project manager at a private education, training, and consultancy provider 22 I2R04 23 I2R02 24 I6R15 “Not everyone understands the pace […] and we try not to involve [stakeholders] actually in the writing process, if we want to have an application finished by the deadline”25 Focusing on producing outputs, project staff may view success in terms of the original goals being reached within the budget, and all contractual obligations being fulfilled, with little incentive for sustainable approaches. “It’s important to reach objectives in the budget and not get any bad scores for the evaluation”26 “Nobody says ‘ok we have to do a piloting, so let’s see where we can [develop] a sustainable network partner’, but you only say, ‘I’ll call some people that I know and hopefully someone will do this damned piloting thing”27 However where projects contribute to organisationally-mandated plans, planning may be more long-term and utilise broader resources. “The possibility of transposing [academic] activities into European projects is something we’ve been trying to do, so if we propose a project, we usually work on it at least a year or a year and a half before the proposal”28 “We have the mandate to develop knowledge with externals […] we have a longer cycle than [the co-respondents], because we have a research agenda that is 4-6 years long, and […] we use projects to help us further our research agenda”29 “There are long-term strategic goals of the university, and trying to create synergy with 25 I3R06 – Project manager at a private education, training, and consultancy provider 26 I3R06 27 I3R07 28 I2R04 29 I3R05
  • 11. Copyright © Paul Talbot 2015. All Rights Reserved 11 the topic you’re working on and the longer- term goals of the university, you are supported longer” 30 It follows that: P3: projects are successful in terms of contractual obligations even with artificial plans And P4: focusing on non-project goals encourages genuine organisational planning, whereas focusing purely on project goals permits artificial, pragmatic planning CONSORTIUM-LEVEL FACTORS Commitment from- and ownership by partners is important. Relations between partners are seen as critical in impacting feelings of ownership. “If you have problems with the consortium, or partners are not interested in the project, you will not reach sustainability”31 “The partnership wasn’t nice, there was fighting from beginning to end, and I think that has a huge impact on sustainability”32 “There can’t be much expected from the sustainability of these results because people are so unhappy with their ownership for them […] We [try to] resolve conflict and make sure they are actually working on something they consider valuable for them now, at the moment, but also for the future”33 Sustainability should be taken seriously by partners, through regular discussion and planning. Official agreements and business plans can be 30 I3R05 31 I7R18 – Departmental Head in the higher-education sector 32 I3R05 33 I5R11 drawn-up, promoting enthusiasm for sustaining results, and facilitating practical planning. “you have to talk about it right from the start with your partners and get back to the question regularly, at each meeting we talk about sustainability, what we can do, how we can adapt our plans for the future”34 “part of the challenge was to develop a business model, how to market it […] pricing strategy and willingness to pay, and find out, do a small market study”35 “As a [subject specialist] I had no idea about business plans, it was a nightmare at the beginning, but it helps, it helps having a structure”36 Although consortium-level commitment is important, organisational motivations are still critical. One respondent saw personal idealism and belief in the project as the source of their commitment, yet as owner of their company they were also a corporate decision-maker. “I’m really convinced that we have done a good job, it’s a good course and people love it when then come to the course” […] “I am willing to put in a lot of time and effort to get these courses running […] as a self- employed trainer and coach, I need to put time aside for organising those courses, that I’m not getting paid for” 37 It follows, therefore, that: P5: Sustainability is promoted by consortium- level ownership, but it is still insufficient without organisational ownership. 34 I7R18 35 I8R20 36 I6R14 – Director and project manager of an independent cultural research institute 37 I8R19
  • 12. Copyright © Paul Talbot 2015. All Rights Reserved 12 This section shows that organisational factors impacting sustainability include the organisational motivation for participating, and the way this affects planning and resource distribution, as well as the commitment and ownership at a consortium level. 4.2 RESPONDING TO THE OPERATIONAL ENVIRONMENT Factors here included responding to real needs through sufficient analysis and understanding, and involvement of externals in planning and implementation, contributing to the creation of valuable outputs that can be sufficiently and sustainably utilised by target groups. 4.2.1 RESPONDING TO REAL NEEDS AND CONTEXTS Projects must analyse and address real – and changing - needs and demands of target groups, the market, and public policy. Producing good results that cannot be used in practice discourages sustainability. “You have to be sure that your project answers to a real need, a real demand”38 “The things your produce in the project, maybe the world isn’t ready for them yet, or maybe the world will never be ready”39 “You need to be aware of what already exists on the market, because there are some competitors of course that are offering something that is similar”40 “Sometimes at the time of application, the subject you are promoting is well recognised, but sometimes there can be a change in context, and then the project becomes not so important for the institution”41 38 I2R02 39 I3R05 40 I6R15 41 I7R18 42 I2R02 43 I2R05 They must also respond to the context and behaviour of stakeholder groups, understanding behaviour patterns and what to expect from externals. “Public institutions […] have an incredibly complicated bureaucracy to deal with, so for sure we have to take care and keep in mind their structure”42 “We find that some organisations are not able to deal with new learning styles [and don’t] have the structures to lend sustainability to what we have developed”43 “You need to know all that everything that goes on in the organisation […] and somehow to work together with them”44 Projects may benefit from responding to direct input, involving externals in the development, planning and implementation of the project. “[potential end-users] actually contribute to new ideas; […] raising project ideas”45 “In focus groups we design the products […] we make them much more clear and concrete with the help of the stakeholders”46 “Involve end-users from the very beginning, and if possible in the partnership itself”47 “Early identification of who are the end- users and stakeholders […] Try to involve them in designing the application. It's very important to understand the real needs and real expectations”48 It follows that: P6: Sustainability benefits from a detailed- and comprehensive-as-possible understanding of real 44 I7R17 45 I1R01 46 I2R03 47 I2R02 48 I6R15
  • 13. Copyright © Paul Talbot 2015. All Rights Reserved 13 needs and contexts in the external operating environment This section shows that sustainability is promoted by ensuring alignment between project outputs and real external needs by way of comprehensive analysis and stakeholder input. 4.3 INFLUENCING THE OPERATIONAL ENVIRONMENT Factors here included the provision of quality and accessible outputs that add value both to target groups and delivering organisations, effective communication and interaction with external actors and the promotion of positive cognitive and emotional responses in the external environment, including encouraging external participation and exploitation. 4.3.1 PRODUCING OUTPUTS Taking into consideration P6 , the outward delivery of outputs into the external environment which meet given needs is essential, and increases chances of real interest from target groups. “We have now [after two years] people asking ‘oh could you send us some material because the topic is so interesting, and the training fits very well’”49 “We found out that lots of practitioners like it very much, also in different countries, and they’ve been inviting our project members to present and conduct workshops”50 However beyond responding to needs, products should demonstrate sufficient market-quality. “[sustainability doesn’t work] if it really isn’t there, the product for the market, it hasn’t reached that stage that the market demands”51 “[one thing] for bad sustainability, is to have a bad product”52 49 I3R07 50 I5R11 51 I3R06 52 I3R07 Furthermore, some respondents speculated on the role that quality plays for organisations when deciding to mainstream products. “[there are organisations who think] if it doesn’t look good, or it doesn’t meet the standard, we will not promote it, because we cannot associate our brand name with this product”53 “I think [a partner who didn’t sustain the outputs] thought that the quality which they were contributing to was not high enough” 54 Where ‘market-ready’ means being having sufficient quality and benefits to viably target potential end-users on a competitive market, it follows that: P7: Delivering market-ready products increases the potential for sustainability Additionally, take-up of the product on the market is enhanced by broadening potential target group applicability, for example by increasing accessibility or adaptability. “If we ask our partners in Portugal to translate these things into Portuguese […] they are available to everyone in Portugal [and] a side effect is that people in Brazil can also use them”55 “Courses [are] usually designed to a special target group, but can also be adopted to other target groups very often”56 It follows that: P8: Broadening target-group applicability increases the potential for sustainability 53 I3R06 54 I4R09 – Senior lecturer at a University of Applied Science 55 I5R11 56 I8R19
  • 14. Copyright © Paul Talbot 2015. All Rights Reserved 14 4.3.2 COMMUNICATION WITH AND PARTICIPATION OF EXTERNALS Project actors need to communicate information to externals. Respondents placed emphasis on the process of publicity. “If you don’t make good publicity, so people simply are not aware of it”57 “Establish contacts with as more as possible stakeholders […] in order to reach really the audience”58 “It’s important that the networks know about the things they can use for example, the products that we have done or the services”59 Value is attached to methods which enable interaction with, and feedback from potential end- users, whether it is one-to-one interaction, group interaction, or social-media interaction. “Because we are a small settlement, it’s easy to keep personal contacts”60 “Small focus groups […] seem to be very interesting in receiving very direct opinions about what is being developed”61 “[With Facebook] it’s easy for people to respond, they are used to it, and if you can sustain a reasonable response to social media […] I think you can strengthen the impact of your project for quite a while”62 This reiterates the value of external participation. “I think that the basic concept […] co- creating the actual project proposal, not just sending out "do you want to participate in this" with a complete proposal, but actually 57 I1R01 58 I4R10 59 I4R09 60 I1R01 61 I2R04 62 I2R03 developing the project proposal from the beginning”63 “The direct involvement of stakeholders [...] is very important, […] we always try to involve as many as we can in [training] activities”64 It follows that: P9: Systematic and interactive communication of project content and outputs promotes sustainability 4.3.3 PROMPTING EXTERNAL RESPONSES Independent action and response by externals should be encouraged. Externals should feel motivated to engage as a participant, feeling a personal value or benefit in the project, although this may require preparatory work. “They have to see the project it really for them, so they are not just used as an excuse to have a project and to run something for two years”65 “We have made [an external stakeholder] care about the project, not just about the product […] she also got interested on how we are growing, and she got passionate about it”66 “Sometimes […] people are not ready to receive new approaches, if the context is not prepared […] then you cannot really sustain your excellent result”67 It follows, therefore, that: 63 I3R05 64 I6R14 65 I1R01 66 I3R06 67 I6R13 – Project manager at an independent educational research institute
  • 15. Copyright © Paul Talbot 2015. All Rights Reserved 15 P10: Encouraging individual connections to the project and its outputs within external environment promotes sustainability Externals can exploit project goals in pursuit of their own objectives, occurring when project goals closely match those of external bodies. “[The municipality] asks us to have some presentations to channel it in their everyday for the benefit of the local people”68 “Get a powerful institution who adopts your product and you don’t need to worry about sustainability”69 “We have […] an action, very much linked to the goals of the project, to impact on the schools, and educational centres of our region, with the same goals of our project, […] it’s the way the make this live longer, at least nationally”70 It follows that: P11: Meeting external goals and objectives can encourage external organisational ownership of project results This section shows the importance of the outputs themselves, and of the systematic and effective communication and interaction with external actors. It also underscores the role of ownership by external actors. 4.4 THE PROGRAMMATIC ENVIRONMENT The operational activities of the funding body affects the practicability of sustainability, whilst an artificial market for grant funding and the evolution of an abstract ‘project community’ affects how people perceive themselves as actors 68 I1R01 69 I2R03 70 I5R12 71 I2R03 within the programmatic environment, influencing their approach to sustainability. 4.4.1 FUNDING BODY POLICY AND DISCOURSE Project actors are overwhelmed by bureaucratic requirements and expectations of the funding body at the proposal and close-down stages, leading them to view sustainability as a bureaucratic necessity, rather than a valuable opportunity. “Coordinators are not encouraged to do their best about exploitation, they are encouraged to look at these deadly details of their financial paper and bureaucratic regulations and in the end they get discouraged”71 “[During the] next 25% of our time, we will still do some things that we think are good, but we will spend the most time on satisfying the reviewer”72 “Most organisations are seeing it as a must and not as an opportunity, so maybe there are ways to create it as a chance and to promote it like this”73 Time at the end of the project is crucial to sustainability work, yet resources are perceived to be restrained by bureaucratic reporting demands. “Sometimes the process of convincing the end users of the benefits of the product or results is too time consuming, and most of the time you are just able to do that after the project ends”74 “Exploitation often comes at the end once you know what the real results are”75 Accepting P3 , that artificial planning facilitates bureaucratically-successful projects, it follows that: 72 I6R16 73 I4R08 74 I6R15 75 I6R16
  • 16. Copyright © Paul Talbot 2015. All Rights Reserved 16 P12: The funding body is perceived as exercising bureaucratic processes which inhibit genuine planning by pressurising project actors towards the realisation of contractual obligations This premise notwithstanding, the emphasis on sustainability within funding-body discourse is perceived as increasing. “The EC [is trying to] fund more and more projects which they expect to have a serious impact, and I think they’re putting more attention on the issue”76 “A couple of years ago, if you were […] using social media, you were thought to be a genius. Now you really have to come up with good ideas and sustainability has, I think, because a major criteria”77 However there is a perceived lack of know-how and limited support structures within the funding body, and disaccord between this lack of support and the expectations placed upon project actors. “I always have the impression that this is something on their list what they have to tell, but they are happily surprised if somebody has a kind of sustainability in their project […] they are like “can you tell us how to do this sustainability, because we have no clue about it”78 “It was extremely unfair to analyse projects by this criteria, because the [funding body] hasn’t set a standard […] and you don’t have specialists capable of offering a help- desk”79 “Lifelong learning is such a vast sector and they cannot know everything. Maybe they don't know what happens in [each specific subject area] so sometimes the judgement and the comments are quite superficial”80 76 I5R11 77 I4R09 78 I3R07 79 I3R06 Incompatibility within the funding policy, such as innovation over sustainability or open-access over marketization are perceived as confusing and counter-productive: “That project was not funded because it wasn’t considered innovative [but] it’s not about being innovative, it’s about being sustainable”81 “On the one hand they said all products will have reach a wide audience free of charge. At the same time they said, to be sustained, they allow you to make financeable courses […] but you never know what was in- between the two”82 It follows therefore, that: P13: funding body policies are perceived as incompatible with sustainability and P14: the funding body is perceived as holding unrealistic expectations of project actors 4.4.2 THE EUROPEAN PROJECT WORLD The European LLP (possibly along with other funding programmes) has established a closed-off economic and professional domain with its own identity distinct from both the private and public sector. Respondents’ views suggested an artificial market for project funding, where projects may exist to support other projects, whilst some produce material that is only, or mainly, sustainable through grant funding. “There is this handbook [about exploitation] which is done in another European-funded project”83 “You take the grant but then you work what with what you have developed afterwards, 80 I6R14 81 I5R11 82 I1R01 83 I4R09
  • 17. Copyright © Paul Talbot 2015. All Rights Reserved 17 like we have developed two in-service courses [LLP funded courses] and we had run the first one for about six years”84 Sustainability outside of this domain is valued however EU funding may itself exist as a symptom of market failure, necessarily financing public- value goods. “If the services or products are compatible on the real market, not on the artificial market of project ideas and proposals, it means the project is extremely successful and sustainable”85 “Either they are funded publically or you are not going to be able to get funding for them”86 This leads us to argue that: P15: projects are funded through an artificial grant-funding market not based on private consumer preferences but on public-value policy objectives and P16: structures are present whereby projects can exist to serve the needs of others within an artificial grant-funding market Within this artificial market project actors may identify themselves professionally in terms of European projects, communities and networks because they work primarily within this domain, and may be part of various projects implemented through different partnerships. “We write projects, more than one or two, as much as possible for each call for proposal”87 84 I8R19 85 I4R10 86 I5R11 87 I3R06 88 I8R19 “If you work with organisations for a long time, like you have done the 2nd or 3rd or whatever project, now something special develops between those partners”88 Events organised in the framework of the funding programme tend to target project actors as a distinct group, rather than as part of wider thematic networks. “The EU values [dissemination conferences] as very important, so we have to do these things, but I don't think they are very efficient [...] we were thousands of people talking for 10 minutes presenting our projects, and I don't remember any of them”89 “You have dissemination events for teaching, for vocational education, for research and so on [attended by] project coordinators and project members, and people from [the EU]. I think they could also invite some [thematic] networks […] in my subject there are lots of technological platforms, but they are never invited to these events”90 It may be that this strengthens the organisational goal of project participation, rather than focusing on complementing non-project goals (P2 ), somewhat supported in the idea that projects self- perpetuate, and can result in the generation of new projects. “Strengthening of the networks created during the project […] to further develop the results obtained in the projects, or possibly discuss new ideas and collaboration”91 “We have two projects where the sustainability works really well. They are both in [the same field] because we have created not only one project but also a follow-up project”92 89 I6R14 90 I7R18 91 I2R04 92 I3R07
  • 18. Copyright © Paul Talbot 2015. All Rights Reserved 18 Accepting this, then: P17: an artificial grant-funding market provides a framework for organisations to establish standard operational activities However, respondents also identified intangible capacity-building impacts from projects. “Working on LLP projects and initiatives, we increased our competence and our awareness of sustaining the results of the projects. The very real impact […] is to make people working together from Europe. Now if I make a phone call to Ireland or Finland, there is someone that is on the line [who knows me]”.93 “These people who are employed to carry out the project together with us become experts, and they can build up something for the future […so] sustainability in terms of personnel development […]it’s very important to have a staff development strategy”94 It follows, then, that: P18: the LLP provided intangible long-term impacts on project actors’ own staff and organisational capacity This section shows that funding body policies are perceived as damaging to sustainability potential, both in how they approach sustainability on a practical level, and in how they allow the perpetuation of non-sustainable projects through an artificial grant-funding market, although the LLP may serve as a general capacity building tool for individual project actors and organisations. 5 DI S C U S S I O N 5.1 ANALYSIS OF FINDINGS The data shows project actors’ perceptions of factors influencing sustainability, whilst revealing other systemic processes, demonstrating how issues interlink, exposing deeper concerns. P2 indicates different motivations for project participation – acquisition of projects and pursuit of non-project goals - P3 shows projects can be contractually successful even when planning is artificial, and P4 links these premises suggesting a direct positive relationship between using projects to achieve broader organisational goals and genuine pro-sustainability planning. The conclusion of this is that organisations should manage projects with goals that complement their organisational objectives. This is supported throughout the data from various perspectives. Firstly, sustainability is difficult when the broader organisation is not involved, limiting the integration of project outputs into other activities 93 I6R14 (P1 ). Where focus is on project acquisition, organisations may neglect to acquire a sufficiently-detailed understanding of the external environment (contrary to P6 ), or to sufficiently interact with external actors (contrary to P9 ). Secondly, organisational ownership appears to precede individual ownership, suggesting that even with commitment from individual project actors – which is deemed important - the ability to integrate outputs into the organisations operational objectives is important (P5 ) . P11 provides that meeting objectives of external organisations can also encourage their ownership of results, suggesting there may be little difference between encouraging external organisational ownership and encouraging internal (partner) organisational ownership. This highlights the relevance of project isolation within the organisation (P1 ) but also raises the question of whether we can extend to the internal organisation those factors deemed important to the external environment, namely the need for detailed 94 I8R20
  • 19. Copyright © Paul Talbot 2015. All Rights Reserved 19 understanding (P6 ) and systematic communication (P9 ) and encouraging individual connections to the project and its outputs (P10 ), ensuring that these outputs meet professional standards and expectations (P7 ) and can be broadly applicable (P8 ). This strengthens the conclusion that sustainability requires understanding of the needs of those required to facilitate sustainability, whether it be the partner organisation or an external body. Understanding these needs, and involving those relevant parties (internal or external) outside of the immediate project team during project development may help to create projects with sustainability potential by seeding ownership outside of the project team. In terms of the external programmatic environment, there is a strong culture of blame- shifting towards the funding body for overwhelming bureaucracy (P12 ) perceived policy- incompatibilities (P13 ), lack of support, and unrealistic expectations (P14 ). These concerns may be valid, however they should not be so crudely simplified. On a practical level, bureaucratic demands may distract project actors from sustainability work, however this may reflect the problem of isolation (P1 ); if sustainability work could be shifted away from project actors and towards operational staff in the broader or external environment, then project actors could focus on complicated administrative processes, but whilst project actors themselves retain stewardship of projects, bureaucratic reporting will necessarily take priority, if organisations wish to avoid negative financial ramifications. As such, the degree of bureaucracy experienced may have effect of enabling and even encouraging pragmatic, artificial planning through the pervasive discourse of sustainability as necessity rather than obligation, perhaps strengthened by any the lack of support, incompatible policies and unrealistic expectations identified by respondents. The interview data suggests the existence of an artificial market, limiting sustainability on the free market (resulting fundamentally from market failure) (P15 ). Within this market organisations are able to establish standard operational activities (P17 ), and may produce outputs to support the needs of other projects within this market (P16 ). Taking into account P4 , that focusing purely on project goals permits artificial, pragmatic planning we may assume that organisations with this goal are neglecting non-project objectives, however, if the organisational objective is indeed to establish standard operational activities within this market, then the production of goods for this market facilitates realignment between project participation and non-project goals. Removal of this artificial market would eliminate the organisational goal. Whilst this supports the argument that sustainable projects need to be linked to long-term organisational goals, it may also reflect policy failures on the part of the funding body which essentially allow public funds to be invested in the perpetuated acquisition of more public funds with limited potential for the mainstreaming of results outside of the funding domain. Whilst organisational goals remain propped up by the funding body, there may be little incentive to gear projects towards the pursuit of non-project- based goals, limiting the extent to which sustainability can be realistically built into, or expected from, these projects. However the revelation that the LLP may have supported capacity-building in terms of European exploitation and sustainability (P18 ) presents the view that it is not the outputs themselves that need be the focus of sustainability, but rather the intangible impacts on European cooperation and coordination for those involved. 5.2 IMPLICATIONS OF THE FINDINGS The findings have implications for the management of LLP projects, the operationalization of the funding programme, and for best practice under the Erasmus+ programme. They show that project sustainability requires commitment and resources to implement results after the funding. Aligning project goals with partner- or external organisations’ own goals, rather than seeing project goals as independent objectives, is instrumental in securing these resources, encouraging project actors to approach planning and implementation with a view to sustainability, seeing pro-sustainability practices such as needs analysis and stakeholder involvement as opportunities rather than
  • 20. Copyright © Paul Talbot 2015. All Rights Reserved 20 bureaucratic hurdles. These efforts are essential for the achievement both of project goals as organisational goals and as public value goals, as Bryson (2004) hypothesises “strategic management processes that employ a reasonable number of competently done stakeholder analyses are more likely to be successful – that is, meet mandates, fulfil missions and create public value – than those that do not”.cxiii Therefore project managers should align project activities with long-term organisational goals, focusing on building strong relationships with those within their organisations or the external environment whom may be able to integrate results into their activities. Project teams should avoid retaining complete ownership of projects, but rather facilitating them within the organisation, involving other on an operative basis, bridging project and non-project activities, with a strong view to providing strategic added-value. The LLP appears to impose supposedly rational policy processes on non-rational organisations.cxiv Bureaucratic and conflicting demands encourage pragmatic approaches to ensure contractual fulfilment and financial remuneration, exacerbating the principle-agent problem in which “agents will seek ways to act in their own interests, whatever contracts are written and whatever monitoring is put in place”,cxv creating hybrid programmes attempting to satisfy all partiescxvi , and inhibiting pro-sustainability practices. Although validity of this conclusion requires verification, including from the perspectives of those working in the funding body, factors such as conflict between public-value policy demands and expectations of mainstreaming along with the evolution of a self-serving market for European grant-funding appear to have further weakened capacity and incentive to engage in genuine pro- sustainability practices. Nonetheless, within this artificial market, the evolution of a community of ‘Europeanised’ groups of sector professionals may fulfil broader long-term policy objectives, facilitating transnational cooperation and cohesion and building capacity to work on a European level. This highlights a need for the funding body to provide better sustainability support for organisations, with coherent, informed guidelines, and finding ways to hold organisations accountable for sustainable impacts. However policy-makers should also reflect on the capacity- building effects of projects and their contribution to European strategy, building this into their impact analyses. It is possible that problems faced in sustainability are a symptom of Curtis’ (2004) split between policy formation and implementation; where projects are implemented remote from the formulation of policy goals they are intended to contribute to, the mental frameworks underpinning the motivation and rationale for acting are fundamentally different.cxvii A critical analysis of policy-making processes at the level of the funding-body would be necessary to fully evaluate the material importance of sustainability within decision-making frameworks and in policy rhetoric, cxviii and whether results are fed back into the policy-making process.cxix The findings have potential implications on how the topic of project sustainability can be approached. The data confirms many factors raised by Savaya et al’s (2008) literature review, however it departs from current discourse by viewing project sustainability from a systems perspective, highlighting the systemic interaction between influencing factors rather than describing them as independent processes. These implications are based on empirically- grounded hypotheses, not irrefutable theory. Sustainability is complex and multifaceted. Identified factors are perhaps neither sufficient nor necessary conditions, but contribute to sustainability in LLP projects. P5 indicates that individual ownership may contribute to promoting pro-sustainability practices, but without organisational ownership, will be insufficient. Individual ownership may influence organisational ownership, although this would depend on complex processes not addressed here. Similarly, that bureaucratic demands or policy incompatibilities present challenges for sustainability does not mean that sustainability is impossible, only that it is made more difficult - there has been some degree of sustainability from LLP-funded projectscxx . This study does not test each emerging factor for its respective impact, and cannot determine the existence of sufficient or necessary conditions. The data reflects perceptions of a small number of project actors (20), analysed through the subjective interpretations of the researcher, himself professionally active in the field of enquiry, with his own biases and pre-conceived
  • 21. Copyright © Paul Talbot 2015. All Rights Reserved 21 views. Interview participants represented a response rate of 7%. That respondents chose to participate may reflect a response-bias in that they were interested in project sustainability; indeed some have strongly-formulated views and opinions. Responses ranged from descriptive accounts of their organisational procedures and what they perceive to impact sustainability, to critical self-reflection, acknowledging deficiencies in their own approaches, and blame allocation, presenting passionate arguments about what they perceive to be wrong. This may not reflect respondents’ “true attitude or sentiment”,cxxi and we should “recognise that informants can and do hold conflicting [and] varying sentiments”.cxxii Indeed the relevance of this topic for those anxious of their projects failing on sustainability criteria [i.e. “it was extremely unfair to analyse projects by this criteria, because the project officer or manager of the programme hasn’t set a standard […] and for sure you don’t have specialists that are capable of offering a help-desk”] may have prompted more emotional, defensive responses than if no emotional connection to the topic was present. It is clear, although patterns emerged from across the data, that due to the inability to pursue iterative theoretical saturation it might be difficult to determine exactly how the data reflects the various values, attitudes, opinions or emotional reactions of respondents. This study has revealed policy-relevant processes in a formally neglected research field, but further research is needed to verify and build on the results. The processes identified are not unimportant, and a greater understanding of these processes may have significant value to researchers and policy makers in all fields relating to grant-funded projects and projects with public- value policy objectives. 6 CO N C L U S I O N S A N D RE C O M ME N D AT I O N S 6.1 CONCLUSIONS This study has identified sustainability as central to the success of the LLP as a policy instrument. In contrast to existing project-focused literature, this study viewed projects within a systems framework, identifying factors – often confirming those identified within the literature – but revealing potential interconnectedness of factors within and between different operating environments. This study found that aligning project goals with long-term organisational objectives is instrumental in securing the capacity and resources to plan and implement projects sustainably and continue delivering impacts. Policy and processes at the level of the funding body do little to encourage pro-sustainability practices, and may have made it easy for organisations to avoid systematic output mainstreaming. This study highlights clear implications of this for both organisations and the funding body, namely that the integration of projects into long-term goals and broader organisational involvement (internal or external) in project development and implementation should be explicitly encouraged, and that better support should be offered to project actors, with coherent, informed guidelines and greater accountability. However, whilst the LLP fails to promote long-term impact of project outputs, it promotes the Europeanisation of sector professionals, promoting and facilitating transnational cooperation and cohesion and building capacity of members of the European workforce to work on a European level, which itself contributes to longer-term European political strategies. The findings are useful for project actors, agency project officers and policy-makers as they reveal systemic processes occurring in the implementation of the LLP. Increasing competition for grant-funding requires organisations to demonstrate attention to sustainability. This study identifies strategies to make project outputs more valuable for organisations and for European policy goals. At the same time, increasing attention to project sustainability within European policy discourse demands practicable evaluation criteria by which to make cost-effective policy and funding decisions.
  • 22. Copyright © Paul Talbot 2015. All Rights Reserved 22 Whilst these findings lack the concrete theoretical foundations required to credibly influence management and policy decisions, they provide a starting point for verification through further research and hopefully serve to sensitise the policy and management research community to the importance of the topic. Further research on the policy-making process would also be helpful in revealing why the LLP was selected as a means of achieving policy, and the extent to which sustainability played a role in this decision-making process. BI B L I O G RA P H Y Ali, H., & Birley, S. (1999). Integrating deductive and inductive approaches in a study of new ventures and customer perceived risk. Qualitative Market Research, 2(2), pp. 103-110. Barriball, K. L., & White, A. (1994). Collecting data using a semi-structured interview: a discussion paper. Journal of Advanced Nursing(19), pp. 328-335. Bienzle, H., Hedman, E., Kirtley, R., Purokuro, V., Rus, C., Wiesinger, S., & Wilen, E. (2010). Survival Kit. Managing Multilateral Projects in the Lifelong Learning Programme. Vienna: die Berater Unternehmensberatungs. Blumer, H. (1954). What is wrong with social theory. American Sociological Review, 19(13), pp. 3-10. Bryson, J. (2004) What to do when stakeholders matter, in Public Management Review, 6, 1:26 Buchanan, D., Fitzgerald, L., & Ketley, D. (2005). No going back: a review of the literature on sustainining organisational change. International Journal or Management Reviews, 7(3), pp. 189-205. Centre for Financial and Management Studies, (2010). Unit 1: The Policy Analysis Model and Alternatives. In C212: Public Policy and Strategy. SOAS, University of London Charmaz, K. (2006). Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. London: Sage. Cohen, M. D., March, J. G., and Olsen, J. P., (1972) A garbage can model or organisational choice, Administrative Science Quarterly, 17,1. pp.1-25; Council of the European Union, 2009. Council conclusions of 12 May 2009 on a strategic framework for European cooperation in education and training (ET 2020). 28th May 2009 (2009/C 119/02) Curtis, D. (2004). How we think they think: thought styles in the management of international aid. Public Administration and Development, 24, pp. 415-423. Dean, J. P., & Whyte, W. F. (1978). How do you know if the informant is telling the truth? In J. Bynner, & K. Stribley, Social Research: Principles and procedures. London: Longman Group UK Ltd in association with The Open Univeristy Press. Deshpande, R. (1983). Paradigms lost: on theory and method in research in marketing. Journal of Marketing, 47(Fall), pp. 101-110. European Commission. Introduction to EU Funding. Retrieved from http://ec.europa.eu/grants/introduction_en.htm. Last Accessed 27.8.14 European Communities. (2006). Sustainability of international cooperation projects in the field of higher education and vocational training: Final Report. Luxembourg: Office for Official Publications of the European Communities. Fernandes, G., Ward, S., & Arajújo, M. (2013). Developing a Framework to Improve and Embed Project Management Practices in Organisations”. Peocedia Technology(9), pp. 846-856. Flynn, N. (2007) Public Sector Management, 5th Ed, London. Sage, p.128 Griffin, C. (2011). Policy and Lifelong Learning. In P. J. (ed), The Routledge International Handbook of Lifelong Learning Oxon: Routledge. pp. 261-270 Halcomb, E. J., & Davidson, P. M. (2006). Is verbatim transcription of interview data always necessary? Applied Nursing Research, 19, pp. 38-42. Hogwood, B. W., and Gunn, L. A. (1984) Policy Analysis for the Real World, Oxford: OUP; Ingle, M. D. (2005). Project Sustainability Manual: How to incorporate Sustainability into the Project Life Cycle. Portland: Portland State University.
  • 23. Copyright © Paul Talbot 2015. All Rights Reserved 23 Jarvis, P. (2011). The European Union and Lifelong Learning Policy. In P. Jarvis, The Routledge International Handbook of Lifelong Learning Oxon: Routledge pp. 271-280 Kvale, S. (1996). InterViews: An introduction to qualitative research interviewing. Thousand Oaks: Sage Publicaions Inc. Lapadat, J. C. (2000). Problematizing transcription: purpose, paradign and quality. Social Research Methodologiy, 3(3), pp. 203-219. Lloyd, R., Baginsky, M., & Puchwein, I. (2006). Sustainability Research: The Big Lottery Fund. London:. London: Big Lottery Fund. Locke, K. (2001). Grounded Theory in Management Research. London: Sage. McEllan, E., MacQueen, K. M., & Neidig, J. L. (2003). Beyond the Qualitative Interview: Data preparation and Transcription. Field Methods(15), pp. 63-84. Morgan, G, (1997) Images of Organisations. London. Sage. Noblit, G. W., & Dwight Hare, R. (1988). Meta-Ethnography: Synthesizing Qualitative Studies. Qualitative Research Methods Series 11. London: Sage Publications. Nutt, P. C. (2002) Chapter 3: Decision.Making Processes Prone to Success and Failure. Why Decisions Fail: avoiding the blunders and traps that lead to debacles. San Francisco. Berrett-Koehler for discussions on the policy-making process Ouane, A. (2011). UNESCO's drive for Lifelong Learning. In P. Jarvis, The Routlege International Handbook of Lifelong Learning Oxon: Routledge. pp. 302-311 Pitagorsky, G. (2011, January 26). Project Managers are Change Managers. Retrieved from http://www.projecttimes.com/george-pitagorsky/project-managers-are-change-managers.html. Last accessed 27.7.14 Project Management Institute. (2004). A guide to the project management body of knowledge: PMBOK guide (3. ed.). Pennsylvania: PMI Publications. Public Policy and Management Institute. (2011). Interim Evaluation of the Lifelong Learning Programme (2007-2013) Service Contract No. 2009-5173-PPMI. European Commission. Punch, K. F. (1998). Introduction to Social Research Methods. London: Sage. Quattrone, P., & Hopper, T. (2001). What does organisational change mean? Speculations on a taken for granted category. Management Accounting Research, 12, pp. 403-435. Robson, C. (2002). Sampling in Surveys - and Elsewhere. In C. Robson, Real World Research Oxford: Blackwell. pp. 260-268 Roulston, K. (2010). Reflective Interviewing: a Gruide to Theory & Practice. London: Sage Publications. Sarriot, E., Ricca, J., Yourkavitch, J., Ryan, L., & SHOUT. (2008). Taking the Long View: A Practical Guide to Sustainability Planning and Measurement in Community-Oriented Health Programming. Calverton, MD: Macro International Inc. Savaya, R., Spiro, S., & Elran-Barak, R. (2008). Sustainability of Social Programs: A Comparative Case Study Analysis. American Journal of Evaluation (29), pp. 478-493. Senior, B., & Swales, S. (2010). Organisational Change (4 ed.). Harlow: Pearson Education. Sequira, A. H., & Moharana, S. (2012). A Re-Look at Project Sustainability. Available at SSRN: http://ssrn.com/abstract=2111031 or http://dx.doi.org/10.2139/ssrn.2111031. Sheier, M. A. (2005). Is sustainability possible? American Journal of Evaluation, 26. 320-347 Shenhar, A. J., & Dvir, D. (2007). Reinventing Project Management: The Diamond Approach To Successful Growth And Innovation. Massachusetts: Harvard Business Review Press. TANGO. (2009). Sustainability of rural development projects: Best practices and lessons learned by IFAD in Asia. IFAD. Teddlie, C., & Yu, F. (2007). Mixed Method Sampling: A Typology with Examples. Journal of Mixed Methods Research, 1, pp. 77-100. Third International Roundtable on Managing for Development Results (2007) Planning and Budgeting: Linking Policy, Planning and Budgeting – A background paper. Hanoi, Vietnam, 5-8 February Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8(45). Thorne, S., Jensen, L., Kearney, M. H., Noblit, G., & Sandelowski, M. (2004). Qualitative meta-synthesis: reflections on methodological orientation and ideological agenda. Qualitative Health Research(14), pp. 1342-1365.
  • 24. Copyright © Paul Talbot 2015. All Rights Reserved 24 Turner, M., & Hulme, D. (1997). Organisational Environments: Comparisons, Contrasts and Significance. In M. Turner, & D. Hulme, Governance, Administration and Development. Baisingstole: Macmillan. pp. 22-61 Welch, C., & Piekkari, R. (2006). Crossing Language Boundaries: Qualitative Interviewing in International Business. Management International Review, 46(4), pp. 417-437. NOTES i Council of the European Union, 2009. Council conclusions of 12 May 2009 on a strategic framework for European cooperation in education and training (ET 2020). 28th May 2009 (2009/C 119/02); Jarvis, P., 2011. The European Union and Lifelong Learning Policy. In P. Jarvis, The Routledge International Handbook of Lifelong Learning Oxon: Routledge. pp. 271-280 ii Griffin, C., 2011. Policy and Lifelong Learning. In: The Routledge International Handbook of Lifelong Learning. Oxon: Routledge, pp. 261-270; Ouane, A., 2011. UNESCO's drive for Lifelong Learning. In: The Routlege International Handbook of Lifelong Learning. Oxon: Routledge, pp. 302-311. iii The Lifelong Learning Programme ran from 2007-2013 and has now been succeeded by the programme ‘Erasmus+’, running from 2014-2020. iv http://ec.europa.eu/grants/introduction_en.htm (last accessed 27.8.14) v Public Policy and Management Institute, 2011 vi Ibid:114 vii Ibid:117 viii Savaya, R., Spiro, S. & Elran-Barak, R., 2008. Sustainability of Social Programs: A Comparative Case Study Analysis. American Journal of Evaluation, Issue 29, pp. 478 ix Ingle, M. D., 2005. Project Sustainability Manual: How to incorporate Sustainability into the Project Life Cycle.. Portland: Portland State University. §2-2 x Ibid:421 xi Sheier, M. A. (2005). Is sustainability possible? American Journal of Evaluation, 26, pp. 320-347 xii Sequira, A. H. & Moharana, S., 2012. A Re-Look at Project Sustainability, Available at SSRN: http://ssrn.com/abstract=2111031 or http://dx.doi.org/10.2139/ssrn.2111031 xiii Savaya et al, 2008, pp. 478 xiv Buchanan, D., Fitzgerald, L. & Ketley, D., 2005. No going back: a review of the literature on sustaining organisational change. International Journal or Management Reviews, 7(3), pp. 189-205. xv European Communities, 2006. Sustainability of international cooperation projects in the field of higher education and vocational training: Final Report., Luxembourg: Office for Official Publications of the European Communities, p. edition notice xvi NHS Modernisation Agency, 2002 in Buchanan, Fitzgerald and Ketley, 2005, p. 190 xvii Lloyd, R., Baginsky, M. & Puchwein, I., 2006. Sustainability Research: The Big Lottery Fund. London:, London: Big Lottery Fund. xviii Sarriot et al (2008); Buchanan, Fitzgerald and Ketley (2005) xix Savaya et al, 2008, pp. 479-481 xx Ibid:490 xxi Ibid:491 xxii Sarriot et al, 2008 xxiii Turner, M. & Hulme, D., 1997. Organisational Environments: Comparisons, Contrasts and Significance. In: Governance, Administration and Development. Baisingstole: Macmillan, p.22 xxiv Buchanen et al, 2005 xxv Senior, B. & Swales, S., 2010. Organisational Change. 4 ed. Harlow: Pearson Education; Quattrone, P. & Hopper, T., 2001. What does organisational change mean? Speculations on a taken for granted category. Management Accounting Research, Volume 12, pp. 03-435 xxvi Project Management Institute, 2004. A guide to the Project Management Body of Knowledge. Third edition. Pennsylvania: PMI Publications, p. 5
  • 25. Copyright © Paul Talbot 2015. All Rights Reserved 25 xxvii Sarriot et al, 2008, p.20 xxviii Pitagorsky, 2011 ‘Project Managers are Change Managers’. http://www.projecttimes.com/george- pitagorsky/project-managers-are-change-managers.html. Last accessed 27.8.14 xxix Savaya et al, 2008, p.481 xxx Ali, H., & Birley, S. (1999). Integrating deductive and inductive approaches in a study of new ventures and customer perceived risk. Qualitative Market Research, 2(2), pp. 103-110. xxxi Morgan, G, 1997. Images of Organisations. London. Sage. xxxii Shenhar, A. J. & Dvir, D., 2007. Reinventing Project Management: The Diamond Approach To Successful Growth And Innovation. Massachusetts: Harvard Business Review Press, p.222 xxxiii Noblit, G. W. & Dwight Hare, R., 1988. Meta-Ethnography: Synthesizing Qualitative Studies. Qualitative Research Methods Series 11. London: Sage Publications. xxxiv Thorne, S., Jensen, L., Kearney, M. H., Noblit, G., & Sandelowski, M, 2004. Qualitative meta- synthesis: reflections on methodological orientation and ideological agenda. Qualitative Health Research (14), pp. 1342-1365. xxxv Thomas, J. & Harden, A., 2008. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8(45). xxxvi Thorne et al, 2004, p. 1358 xxxvii Buchanan, Fitzgerald, & Ketley, 2005, p.192 xxxviii Ingle, 2005; Lloyd, Baginsky and Puchwein, 2006; TANGO, 2009. Sustainability of rural development projects: Best practices and lessons learned by IFAD in Asia: IFAD; Sarriot et al, 2008; Savaya et al, 2008; Bienzle, H. et al., 2010. Survival Kit. Managing Multilateral Projects in the Lifelong Learning Programme. Vienna: die Berater Unternehmensberatungs xxxix European Communities, 2006; Lloyd, Baginsky and Puchwein, 2006 xl Savaya, Spiro, & Elran-Barak, 2008; European Communities, 2006; Ingle, 2005 xli Lloyd, Baginsky, and Puchwein, 2006; TANGO, 2009 xlii Ingle, 2005; Lloyd, Baginsky, and Puchwein, 2006 xliii Lloyd, Baginsky, and Puchwein, 2006 xliv Sarriot et al, 2008 xlv Lloyd, Baginsky, and Puchwein, 2006; Ingle, 2005; TANGO, 2009 xlvi Ingle, 2005; TANGO, 2009; European Communities, 2006 xlvii TANGO, 2009 xlviii Ingle, 2005 xlix Ingle, 2005; European Communities, 2006 l Bienzle et al., 2010 li Ingle, 2005; TANGO, 2009 lii TANGO, 2009 liii Savaya, Spiro, & Elran-Barak, 2008 liv Sarriot et al, 2008 lv TANGO, 2009 lvi Sarriot et al 2008; Savaya, Spiro, & Elran-Barak, 2008 lvii Lloyd, Baginsky and Puchwein, 2006 lviii Ingle, 2005; Savaya, Spiro, & Elran-Barak, 2008; Lloyd, Baginsky and Puchwein, 2006 lix Ingle, 2005 lx Lloyd, Baginsky and Puchwein, 2006 lxi European Communities, 2006 lxii Lloyd, Baginsky, and Puchwein, 2006 lxiii Ingle, 2005; Lloyd, Baginsky and Puchwein, 2006; Tango, 2009; Savaya, Spiro, & Elran-Barak, 2008; Bienzle et al., 2010, Sarriot et al 2009; European Communities, 2006 lxiv Bienzle et al, 2010 lxv Ingle, 2005; Lloyd, Baginsky and Puchwein, 2006; Tango, 2009; Savaya, Spiro, & Elran-Barak, 2008; Sarriot et al 2008
  • 26. Copyright © Paul Talbot 2015. All Rights Reserved 26 lxvi Lloyd, Baginsky and Puchwein, 2006 lxvii TANGO, 2009 lxviii TANGO, 2009; Lloyd, Baginsky, and Puchwein, 2006; Savaya, Spiro, & Elran-Barak, 2008 lxix TANGO, 2009 lxx Ibid lxxi Ibid lxxii Ingle, 2005; Lloyd, Baginsky, and Puchwein, 2006; Bienzle, et al., 2010; Sarriot et al 2008; European Communities, 2006 lxxiii Ingle, 2005 lxxiv Lloyd, Baginsky, and Puchwein, 2006 lxxv Lloyd, Baginsky, and Puchwein, 2006; European Communities, 2006 lxxvi Lloyd, Baginsky, and Puchwein, 2006 lxxvii Ibid lxxviii Ibid lxxix Ibid lxxx Public Policy and Management Institute, 2011 lxxxi Blumer, H., 1954. What is wrong with social theory. American Sociological Review, 19(13), pp. 3-10. lxxxii Ali & Birley, 1999 lxxxiii Fernandes, G., Ward, S. & Arajújo, M., 2013. Developing a Framework to Improve and Embed Project Management Practices in Organisations”. Peocedia Technology, Issue 9, pp. 846-856 lxxxiv Charmaz, K., 2006. Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. London: Sage lxxxv Senior & Swales, 2010, p. 5 lxxxvi Senior & Swales, 2010, pp. 14-15. lxxxvii Senior & Swales, 2010, p. 14-15 lxxxviii Locke, K., 2001. Grounded Theory in Management Research. London: Sage. lxxxix Punch, K. F., 1998. Introduction to Social Research Methods. London: Sage. xc Halcomb, E. J., & Davidson, P. M. (2006). Is verbatim transcription of interview data always necessary? Applied Nursing Research, 19, p.39 xci see, for example, the issue of meaning and linguistic diversity, §3.2.4 xcii Deshpande, R., 1983. Paradigms lost: on theory and method in research in marketing. Journal of Marketing, 47(Fall), pp. 101-110 xciii Ali & Birley, 1999 xciv Blumer, 1954 in Charmaz, 2006, p. 16 xcv Charmaz, 2006, p. 17 xcvi Fernandes, Ward, & Arajújo, 2013 xcvii Teddie, C. & Yu, F., 2007. Mixed Method Sampling: A Typology with Examples. Journal of Mixed Methods Research, Volume 1, pp. 77-100; Robson, C., 2002. Sampling in Surveys - and Elsewhere. In: Real World Research. Oxford: Blackwell, pp. 260-268. xcviii Robson, 2002:265 xcix Teddlie & Yu, 2007, p. 82 c Teddlie & Yu, 2007 ci Barriball, K. L. & White, A., 1994. Collecting data using a semi-structured interview: a discussion paper. Journal of Advanced Nursing, Issue 19, pp. 329-330 cii Roulston, K., 2010. Reflective Interviewing: a Gruide to Theory & Practice. London: Sage Publications. ciii Lapadat, J. C., 2000. Problematizing transcription: purpose, paradign and quality. Social Research Methodologiy, 3(3), p. 210 civ Lapadat, 2000 cv Kvale, S. (1996). InterViews: An introduction to qualitative research interviewing. Thousand Oaks: Sage Publications, Inc, p. 163 cvi Halcomb & Davidson, 2006
  • 27. Copyright © Paul Talbot 2015. All Rights Reserved 27 cvii McEllan, E., MacQueen, K. M. & Neidig, J. L., 2003. Beyond the Qualitative Interview: Data preparation and Transcription. Field Methods, Issue 15, p. 67 cviii Welch, C. & Piekkari, R., 2006. Crossing Language Boundaries: Qualitative Interviewing in International Business. Management International Review, 46(4), p. 418 cix Tsang, 1998 in Welch & Piekkari, 2006, p. 422 cx Welch and PIekkari, 2006 cxi Ibid, p. 428 cxii Interviews were conducted with 21 people. One respondent left the teleconference abruptly due to technical difficulties having provided only superficial descriptive accounts of sustainability that could not be probed further. These responses were not considered in the analysis, leaving a total of 20 respondents in the final dataset. cxiii Bryson, J, 2004, What to do when stakeholders matter, in Public Management Review, 6, 1:26 cxiv Centre for Financial and Management Studies, 2010. Unit 1: The Policy Analysis Model and Alternatives. In C212: Public Policy and Strategy. SOAS, University of London cxv Flynn, N. 2007. Public Sector Management, 5th Ed, London. Sage:128 cxvi Curtis, D. 2004. How we think they think: thought styles in the management of international aid. Public Administration and Development, 24, pp. 415-423. cxvii Curtis, 2004 cxviii See for example Hogwood, B. W., and L. A. Gunn, 1984. Policy Analysis for the Real World, Oxford: OUP; Cohen, M. D., March, J. G., and Olsen, J. P., 1972. A garbage can model or organisational choice, Administrative Science Quarterly, 17,1:1-25; or Nutt, P. C., 2002. Chapter 3: Decision.Making Processes Prone to Success and Failure. Why Decisions Fail: avoiding the blunders and traps that lead to debacles. San Francisco. Berrett-Koehler for discussions on the policy-making process cxix Third International Roundtable on Managing for Development Results, 2007. Planning and Budgeting: Linking Policy, Planning and Budgeting – A background paper. Hanoi, Vietnam, 5-8 February. cxx For example the European Computer Driving License (EDCL), flagged by the funding agency as best practice in sustainability cxxi Dean, J. P., & Whyte, W. F. (1978). How do you know if the informant is telling the truth? In J. Bynner, & K. Stribley, Social Research: Principles and procedures. London: Longman Group UK Ltd in association with The Open Univeristy Press, p. 188 cxxii Ibid