This document is a dissertation submitted by a student for their MSc in Development Management in 2016. The dissertation assesses the utility of beneficiary feedback systems and user-generated content for participatory monitoring of development programmes. The introduction provides background on the failure of top-down approaches to achieving the Millennium Development Goals and the need for more participatory processes in development under the new Sustainable Development Goals framework. The research aims to evaluate feedback systems and how they can empower beneficiaries to influence programme implementation by addressing local priorities. The dissertation will review literature on participatory monitoring and social accountability tools. It will also analyze case studies of feedback systems including Integrity Action's DevelopmentCheck platform and its "fix rate

1. DV410
Page
1
of
40
41743
Candidate
Number:
41743
MSc
in
Development
Management
2016
Dissertation
submitted
in
partial
fulfilment
of
the
requirements
of
the
degree
Power
to
the
People:
Assessing
the
Utility
of
Beneficiary
Feedback
Systems
and
User-­‐Generated
Content
for
the
Participatory
Monitoring
of
Development
Programmes
Word
Count:
10070

2. DV410
Page
2
of
40
41743
Abstract
Feedback
systems
are
emerging
as
an
innovative
approach
to
monitoring
programme
effectiveness
from
the
perspective
of
beneficiaries.
A
main
finding
of
this
paper
is
that
for
these
systems
to
be
effective,
there
must
be
a
significant
revaluation
of
the
incentives
that
motivate
development
organisations
to
make
“beneficiary
oriented”
management
decisions.
Additionally,
though
feedback
systems
provide
beneficiaries
with
the
power
to
steer
programme
implementation
so
that
its
outputs
and
outcomes
more
closely
address
local
priorities,
the
tools
are
still
very
much
reliant
on
sustained
engagement
of
human
intermediaries.
The
study
further
recommends
multi-­‐channel
approaches
to
programme
monitoring,
as
feedback
systems
should
be
seen
as
a
complementary
tool
rather
than
a
replacement
for
more
conventional
approaches.

3. DV410
Page
3
of
40
41743
Contents
Abstract
........................................................................................................................................
2
List
of
Acronyms
...........................................................................................................................
4
I. Introduction
..............................................................................................................................
5
I.1
Background
..........................................................................................................................
5
I.2
Justification
for
the
Research
..............................................................................................
7
I.3
Research
Questions
.............................................................................................................
7
I.4
Structure
..............................................................................................................................
8
II.
Methodology
............................................................................................................................
9
II.1
Methodology
and
Research
Design
....................................................................................
9
II.2
Limitations
........................................................................................................................
10
III.
Literature
Review
..................................................................................................................
11
III.1
The
Millennium
Development
Goals
...............................................................................
11
III.1.1
Criticisms
of
Approaches
Used
to
the
Monitor
the
MDGs
.......................................
12
III.2
The
Sustainable
Development
Goals
–
Leaving
No
One
Behind
.....................................
13
III.3
Participation
and
Social
Accountability
...........................................................................
13
III.4
Participatory
Monitoring
.................................................................................................
15
III.5
Beneficiary
Feedback
......................................................................................................
16
III.5.1
Obstacles
to
Collecting
Beneficiary
Feedback
..........................................................
17
III.6
Information
and
Communication
Technologies
and
the
Promise
of
User-­‐Generated
Content
...................................................................................................................................
19
IV.
Case
Analysis
.........................................................................................................................
24
IV.1
Check
My
School
.............................................................................................................
24
IV.
2
Integrity
Action
and
DevelopmentCheck
.......................................................................
26
IV.2.1
The
Fix-­‐Rate
..............................................................................................................
28
IV.2.2
Obstacles
to
Mainstreaming
the
Fix-­‐Rate
as
a
Key
Performance
Indicator
for
International
Development
................................................................................................
29
V.
Discussion
...............................................................................................................................
30
VI.
Conclusion
.............................................................................................................................
34
VI.1
Further
Research
.............................................................................................................
35
Bibliography
...............................................................................................................................
36
Appendix
....................................................................................................................................
40
Appendix
A.
List
of
Interviewees
............................................................................................
40

4. DV410
Page
4
of
40
41743
List
of
Acronyms
ANSA-­‐EAP
Affiliated
Network
for
Social
Accountability
in
East
Asia
and
the
Pacific
BFS
Beneficiary
Feedback
Systems
CIB
Community
Integrity
Building
CMS
Check
My
School
CPI
Corruption
Perceptions
Index
CSO
Civil
Society
Organisation
DepEd
Department
of
Education
ICT
Information
Communication
Technology
LDC
Least
Developed
Country
KPI
Key
Performance
Indicator
MDGs
Millennium
Development
Goals
MIS
Management
Information
System
NGO
Non-­‐Governmental
Organisation
SDGs
Sustainable
Development
Goals
SMS
Short
Service
Messaging
TPM
Third
Party
Monitoring
UN
United
Nations
UGC
User
Generated
Content

5. DV410
Page
5
of
40
41743
I. Introduction
I.1
Background
‘Even
in
extreme
poverty,
a
person
has
ideas.
If
these
ideas
aren’t
recognized,
people
fall
even
deeper
into
poverty’
(Coyne
et
al.,
2014:
19).
These
are
the
words
of
an
Ouagadougou
resident
speaking
at
a
regional
seminar
organised
to
gather
input
from
local
communities
about
their
attitudes
towards
the
Millennium
Development
Goals
(MDGs).
Specifically,
the
seminar
sought
to
take
a
collaborative
approach
in
designing
a
framework
for
the
post-­‐2015
development
agenda,
Agenda
2030
(Coyne
et
al.,
2014:
19).
The
international
community’s
failure
to
achieve
the
eight
MDGs
exposed
the
pitfalls
of
a
top-­‐down
development
scheme
that
inadvertently
excluded
those
living
in
poverty
from
meaningfully
engaging
in
the
development
process
(Callendar,
2014:
2).
Moving
forward
with
the
global
development
agenda
towards
achieving
the
Sustainable
Development
Goals
(SDGs),
participatory
processes
are
now
being
encouraged
to
‘be
built
into
every
single
stage
of
the
development
process:
from
inception
to
design;
from
implementation
to
learning,
monitoring
and
evaluation’
(Leavy
&
Howard,
2013:
37).
By
treating
the
beneficiaries
of
programmes
as
active
participants
in
the
development
process,
participatory
approaches
to
development
are
widely
thought
to
contribute
to
greater
project
effectiveness
and
efficiency
through
the
establishment
of
‘strong
and
timely
feedback
loops’
(Whittle,
2013:
16).
From
the
‘wider
use
of
participatory
methods
in
development
practice’
(Jacobs
et
al.,
2010:
39),
participatory
monitoring
has
emerged
as
a
tool
that
aligns
‘monitoring
processes
with
the
core
development
principle
of
supporting
local
analysis
and
action’
and
‘places
marginalised
groups
at
the
centre
of
systems,
focusing
on
their
views’
(Jacobs
et
al.,
2010:
40).
Participatory
monitoring
is
defined
here
as
a
strategy
that
enables
project
beneficiaries
to
‘take
the
lead
in
tracking
and
analysing
progress
towards
jointly
agreed
results
and
deciding
on
corrective
action’
(Hilhorst
&
Guijt,
2006:
1).
The
effectiveness
of
participatory
monitoring
is
contingent
upon
the
level
of
commitment

6. DV410
Page
6
of
40
41743
that
all
stakeholders
of
a
project
are
willing
to
engage
in
the
processes
of
joint
learning
and
collaborative
efforts
to
take
corrective
action
against
any
problems
that
may
arise
during
the
implementation
of
a
programme
(Hilhorst
&
Guijt,
2006:
4-­‐5).
Participatory
monitoring
requires
a
substantial
investment
in
both
money
and
time.
As
it
is
often
rare
to
see
this
concept
being
brought
into
practice,
many
implementing
organisations
and
donors
still
hold
the
view
that
the
costs
outweigh
the
benefits
(Jacobs,
2010:
62).
It
is
therefore
common
for
participatory
monitoring
to
be
employed
only
as
far
as
a
“box-­‐
ticking
exercise”
in
a
manner
that
merely
fulfils
the
minimum
standards
set
forth
by
the
project’s
managing
stakeholders
(Jacobs,
2010:
62).
It
is
apparent
that
to
improve
the
effectiveness
of
participatory
monitoring
practices
within
the
development
sector,
the
organisational
and
external
incentives
which
inform
management
decisions
must
be
strengthened
(Jacobs,
2010:
57).
Some
practitioners
are
beginning
to
suggest
that
principles
from
the
commercial
sector,
such
as
open
consumer
feedback
systems
and
user-­‐generated
content
(UGC),
can
possibly
be
translated
into
development
in
order
to
restructure
incentive
systems
to
become
more
“customer-­‐
oriented”,
or
in
the
context
of
the
development
sector,
“beneficiary-­‐oriented”
(Jacobs,
2010:
57;
Jacobs
et
al.,
2010:
43).
Similar
to
how
consumer
feedback
published
on
stand-­‐
alone
review
sites
such
as
Trip
Advisor
are
used
to
assess
quality
of
consumer
goods
and
services,
beneficiary
feedback
systems
(BFS)
in
development
could
be
used
to
‘assess
the
quality
and
impact
of
work
undertaken
by
a
development
agency’
(Jacob,
2010:
57).
Sourcing
feedback
and
information
from
the
beneficiaries
themselves
would
make
measurements
of
the
performance
of
organisations
and
the
outcomes
of
projects
contingent
on
the
local
people’s
opinions
and
satisfaction
(Jacobs
et
al.,
2010:
41).
If
used
effectively,
feedback
systems
could
be
used
to
generate
UGC
that
could
inform
management
decisions
to
work
more
closely
to
meeting
local
priorities,
thereby
improving
programme
outcomes
and
the
likelihood
of
achieving
development
goals
such
as
the
SDGs
(Jacobs
et
al.,
2010:
41).
It
should
be
noted
that
the
most
effective
feedback
systems
in
operation
within
the
commercial
sector
are
user-­‐initiated
and
owned,
meaning
that
they
function
external
to
the
operations
of
the
companies
or
service
providers
receiving
consumer
reviews.

7. DV410
Page
7
of
40
41743
I.2
Justification
for
the
Research
Participatory
monitoring
and
beneficiary
feedback
systems
have
shown
to
be
capable
of
‘promoting
strong
and
timely
feedback
loops’,
while
adding
dimensions
of
accountability
and
transparency
to
development
programme
management
(Whittle,
2013:
16).
However,
the
initial
research
has
shown
that
‘very
few
development
organizations
apply
these
methods
to
monitor
their
fieldwork’
(Jacobs
et
al.,
2010:
4).
When
these
approaches
do
end
up
being
deployed
in
the
field,
they
are
‘often
implemented
as
one-­‐off
pilots
than
widely
applied’
(Jacobs,
2010:
58).
Few
attempts
have
been
made
to
establish
stand-­‐alone
user-­‐initiated
and
owned
feedback
systems
that
could
be
used
to
assess
the
performance
of
development
organisations
in
relation
to
sector-­‐wide
benchmarks.
In
turn,
much
less
has
been
done
to
shed
light
on
the
successes
or
the
challenges
faced
by
those
implementing
such
systems.
The
purpose
of
this
dissertation
is
then
to
shed
light
on
the
experiences
of
the
few
organisations
who
have
initiated
pilots
of
user-­‐initiated
and
owned
feedback
systems,
and
draw
inferences
about
their
functionality
within
the
development
sector
and
the
effect
that
it
may
have
on
large-­‐scale
development
initiatives
such
as
the
SDGs.
What
distinguishes
this
dissertation
from
previous
work
is
the
focus
placed
on
strategies
through
which
principles
and
concepts
from
the
commercial
sector
can
translated
into
development
to
improve
monitoring
practices.
I.3
Research
Questions
The
primary
aim
of
this
research
is
to
assess
the
utility
of
BFS
when
employed
to
manage
and
monitor
development
projects
as
a
stand-­‐alone
tool,
which
this
paper
will
later
refer
as
user-­‐owned
and
initiated
feedback
systems.
This
dissertation
is
guided
by
the
following
research
questions:
1. What
are
the
common
barriers
that
limit
the
effectiveness
of
participatory
monitoring
and
BFS
within
the
development
sector?

8. DV410
Page
8
of
40
41743
2. What
are
the
added-­‐benefits
of
introducing
user-­‐owned
and
initiated
feedback
systems
–
typically
used
in
the
commercial
sector
–into
development
for
the
purpose
of
modifying
or
complementing
existing
participatory
monitoring
strategies?
3. Based
on
the
review
of
the
literature
and
case
evidence,
do
existing
applications
used
in
international
development
resemble
stand-­‐alone
user-­‐initiated
review
systems?
4. What
implications
might
user-­‐owned
and
initiated
feedback
systems
have
on
the
achievement
of
the
SDGs?
I.4
Structure
The
remainder
of
the
paper
is
divided
as
follows:
Section
II
gives
an
overview
of
the
empirical
methodology
and
limitations
of
the
research.
Section
III
provides
a
review
of
the
literature
and
the
theoretical
background
which
the
analysis
of
this
paper
is
grounded
-­‐-­‐
participatory
monitoring
and
BFS
framed
within
the
larger
of
theoretical
framework
of
participatory
approaches
to
development,
social
accountability,
and
result-­‐based
programme
management.
Section
IV
provides
a
case
analysis
of
organisations
attempting
to
pilot
stand-­‐alone
user-­‐initiated
review
systems
in
the
development
sector.
Section
V
draws
inferences
based
on
the
analysis
of
the
cases
within
the
broader
theory
presented
in
literature
review.
Section
VI
concludes
the
paper
with
the
key
points
from
the
researcher’s
findings
with
the
addition
of
suggestions
for
further
research.

9. DV410
Page
9
of
40
41743
II.
Methodology
II.1
Methodology
and
Research
Design
Formulation
of
the
dissertation’s
research
aims
and
questions
was
shaped
by
a
thorough
review
of
relevant
literature.
Sources
included
development
policy
briefings,
unpublished
grey
literature,
organisational
manuals
and
guides,
and
peer-­‐reviewed
academic
articles.
To
establish
a
general
understanding
of
the
key
concepts
and
theories
of
the
research
questions,
the
researcher
conducted
a
series
of
14
semi-­‐structured
interviews
with
development
consultants
and
practitioners,
experts
in
citizen
engagement
and
participatory
monitoring,
as
well
as
heads
and
CEOs
of
development
organisations
(Appendix
A).
These
steps
allowed
the
researcher
to
shed
light
on
gaps
in
the
literature
and,
in
doing
so,
helped
focus
the
research
questions
to
address
some
of
the
most
current
topics
of
discussion
surrounding
participatory
monitoring
and
BFS.
In
conducting
a
comprehensive
review
of
published
organization
publications,
the
researcher
gained
a
key
understanding
of
the
linkages
between
theories
of
participation
in
development
and
the
innovative
approaches,
tools,
and
mechanisms
for
participatory
monitoring
of
projects
currently
being
deployed
in
the
field.
Personal
communication
with
development
practitioners
was
also
used
to
fill
in
any
knowledge
gaps
that
existed
due
to
a
lack
of
published
organisation
documents.
To
understand
existing
theories
and
contribute
to
the
on-­‐going
discussions
surrounding
participatory
approaches
to
monitoring
in
development,
the
researcher
executed
a
case
analysis.
The
choice
and
construction
of
the
cases
analysed
were
guided
by
the
writings
of
Hancké
(2009)
and
de
Vaus
(2001).
The
cases
chosen
are
classified
as
‘theory
testing’
cases
as
the
selection
process
was
meant
to
find
an
organisation
that
has
successfully
introduced
type
C
feedback
systems
to
development
project
monitoring
(de
Vaus,
2001:
222).
The
point
of
the
case
analysis
was
then
to
assess
whether
the
theoretical
underpinnings
-­‐-­‐
suggesting
that
a
stand-­‐alone
BFS
could
function
as
a
practical
monitoring
tool
in
development
-­‐-­‐
actually
played
out
in
the
real-­‐world
(de
Vaus,
2001:
222).
In
regards
to
the
selection
of
cases,
choice
was
limited
as
few
actors
are
currently

10. DV410
Page
10
of
40
41743
attempting
to
pilot
such
systems
in
development.
Cases
were
chosen
on
the
basis
of
how
close
the
BFS
is
to
resembling
one
that
would
be
found
in
the
commercial
sector.
The
case
analysis
was
used
to
draw
lessons
from
organisations’
experience
of
initiating
stand-­‐alone
BFS
for
the
monitoring
of
development
projects,
including
their
overall
utility.
An
analysis
of
the
cases
allowed
the
researcher
to
infer
the
current
potential
of
commercial
sector
mechanisms
such
as
consumer
feedback
systems,
to
be
translated
into
the
development
sector.
The
analysis
of
the
literature
and
the
cases
helped
the
researcher
also
draw
inferences
about
what
strategies
might
be
taken
to
improve
the
functionality
of
BFS
systems
and
UGC
in
the
context
of
development.
II.2
Limitations
Though
the
concept
of
using
beneficiary
feedback
for
development
programme
monitoring
has
gained
a
significant
amount
of
attention
in
recent
years,
‘very
few
organisations
apply
these
methods
to
monitor
their
fieldwork’
(Jacobs
et
al.,
2010:
41).
Organisations
such
as
the
World
Bank
and
World
Vision,
who
have
made
corporate
commitments
to
ensure
that
BFS
is
incorporated
into
their
programme
implementation
practices,
have
only
recently
began
to
assess
the
added
long-­‐term
benefits
its
use
in
practice.
This
significantly
constrained
the
ability
of
the
researcher
to
gain
an
in-­‐depth
insight
of
how
well
and
to
what
extent
BFS
are
being
utilised
by
implementing
organisations.
The
researcher
encountered
a
lack
of
organisation
publications
and
real-­‐
world
reporting
of
these
systems
as
a
limitation
to
the
research.
As
a
qualitative
approach
was
used
in
the
case
analysis,
researcher
bias
emerges
a
key
limitation
of
this
study.
The
subjectivity
of
researcher
in
relation
to
their
selection
of
the
cases
reviewed
means
that
one
cannot
objectively
verify
or
replicate
the
findings.
Researcher
bias
ultimately
means
that
it
is
difficult
to
generalise
the
inferences
drawn
from
the
researcher’s
analysis
of
the
cases
and
literature.

11. DV410
Page
11
of
40
41743
III.
Literature
Review
Top-­‐down
planning
continues
to
be
rampant
in
development
despite
it
being
proven
that
‘participation
of
the...poor
in
their
own
development
has
been
measured
as
a
key
success
factor
in
the
successes
of
projects’
(Burkey,
1993:
56).
Since
the
conclusion
of
the
MDGs,
there
has
been
an
increased
call
for
development
actors
to
work
towards
ensuring
that
their
programmes
have
‘some
participatory
or
consultative
element
in
their
programmes’
(Jacobs
et
al.,
2010:
39).
Demands
for
greater
accountability
and
for
organisations
to
implement
projects
that
exhibit
positive
outcomes
and
impacts
have
increased
increased
calls
for
more
holistic,
results-­‐based
monitoring
systems
with
support
effective
coordination
of
projects
(Bernardi
and
De
Chiara,
2011:
24).
Seen
as
an
answer
to
these
calls,
participatory
monitoring
and
BFS
are
being
promoted
as
key
to
the
success
of
the
SDGS.
III.1
The
Millennium
Development
Goals
The
MDGs,
a
set
of
universal
development
targets,
were
ushered
into
the
mainstream
international
development
agenda
following
the
signing
of
the
Millennium
Declaration
in
September
2000
(Masset
&
White,
2004:
279;
United
Nations,
2003:
4).
They
represent
an
ambitious
international
effort
to
adopt
a
results-­‐based
approach
to
the
amelioration
of
chronic
poverty
(Masset
&
White,
2004:
279;
United
Nations,
2003:
4).
However,
they
also
demonstrate
the
pitfalls
of
a
development
agenda
that
fails
to
place
local
participation
as
a
priority,
and
restrains
the
ability
of
beneficiaries
to
articulate
their
own
needs
and
priorities
(Kusek
&
Rist,
2004:
3).
Despite
the
initial
global
fervour
surrounding
the
MDGs,
the
agenda
has
drawn
strong
criticism
for
its
design,
implementation,
and
approaches
to
monitoring
-­‐-­‐
all
of
which
were
exclusive
of
the
involvement
of
the
poor
themselves
(Callendar,
2014:
2).
The
MDG’s
targets
simply
lack
richness
and
depth,
and
are
not
‘grounded
in
the
realities
of
people’s
needs’
(Manning
et
al.,
2013:
5).
Manning
et
al.
(2013:
4)
argue
that
the
sets
of
targets
were
‘broken
down
into
silos’
in
a
way
that
detracted
from
the
‘holistic
nature
of
the
challenges
that
poor
people
face.’
Analysing
the
operations
of
the
MDGs,
it
is
clear
that
the
approaches
to

12. DV410
Page
12
of
40
41743
monitoring
the
progress
of
the
MDGs
had
too
much
of
a
focus
on
measuring
success
at
the
national
level,
and
failed
to
capture
the
granular
information
found
at
the
local
level,
proven
to
be
necessary
for
a
development
programme’s
or
initiative’s
success.
III.1.1
Criticisms
of
Approaches
Used
to
the
Monitor
the
MDGs
The
MDGs
were
centrally
monitored,
as
data
collection
and
reporting
was
conducted
at
the
‘national,
regional,
thematic,
and
global
levels’
and
‘prepared
by
national
governments,
regional
commissions,
multi-­‐stakeholder
coalitions’,
all
linked
to
the
UN
system
(Suter
&
Fishman,
2015:
1).
An
emphasis
on
national
averages
led
to
failures
in
recognising
changes
and
impacts
occurring
at
sub-­‐national
levels
and
within
specific
income
and
social
groups
(Callendar,
2014:
2).
By
failing
to
disaggregate
measures
downward,
the
measurements
of
the
MDGs
were
stripped
of
the
richness
and
context-­‐
relevance
that
is
contained
in
data
sourced
from
local-­‐level
stakeholders
(Callendar,
2014:
80).
The
detachment
from
the
realities
of
those
living
in
poverty
had
massive
implications
on
the
effectiveness
of
the
MDGs
(Bernardi
&
De
Chiara,
2011:
36).
By
failing
to
disaggregate
data
down
to
societal
sub-­‐categories
such
as
‘gender,
race,
or
other
vulnerable
groupings’,
inequalities
within
communities
were
rarely
recognised
and
subsequently
not
addressed
with
appropriate
solutions
(Suter
&
Fishman,
2015:
2).
This
of
course
led
to
the
lack
of
a
“trickling-­‐down”
of
macro-­‐level
improvements
across
all
socioeconomic
groups
within
a
country
(United
Nations
Development
Programme,
2013:
5).
These
failures
were
often
the
result
of
the
national
statistics
systems
of
developing
countries
and
front-­‐line
workers
lacking
the
capacity
to
“reach”
the
beneficiaries
as
well
as
filter
through
and
aggregate
feedback
in
order
to
make
it
“actionable”,
in
regards
to
using
the
data
to
inform
management
decisions
of
implementing
agents
(Sanga,
2011:
113;
Suter
&
Fishman,
2015:
2).
The
disparities
between
what
aid
providers
and
beneficiaries
believed
to
be
issues
in
need
of
corrective
action,
resulted
in
weak
targeting
of
programmes,
and
subsequently
low
instances
of
positive
programme
impacts
and
outcomes.

13. DV410
Page
13
of
40
41743
III.2
The
Sustainable
Development
Goals
–
Leaving
No
One
Behind
The
MDG
experience
exemplifies
the
prevailing
disconnect
between
development
providers
and
beneficiaries
that
has
led
to
widespread
gaps
in
‘transparency,
accountability,
and
effectiveness
of
development
assistance’
(Gigler
et
al.,
2014:
212).
It
demonstrates
the
need
for
beneficiaries
to
be
given
more
jurisdiction
over
decisions
related
to
the
setting
of
development
targets
and
solutions
to
problems
(Manning
et
al.,
2013:
5).
It
is
for
this
reason
that
“Leaving
No
One
Behind”
resonates
as
the
theme
for
Agenda
2030.
Success
of
the
SDGs
requires
a
more
radical
shift
towards
a
results-­‐based
approach
to
programme
management
and
monitoring.
This
approach
prioritises
true
and
effective
participation
and
social
accountability
mechanisms,
as
development
actors
are
more
effectively
bound
to
their
commitments
and
accountable
to
the
project
beneficiaries
(Manning
et
al.,
2013:
7).
By
prioritising
engagement
with
beneficiaries
and
ensuring
that
their
input
is
used
as
actionable
indicators
which
guide
the
targeting
of
programmes
and
policies,
the
SDGs
have
a
better
opportunity
to
contribute
to
the
success
and
impacts
of
development
projects
(CARE
International,
2016:
16-­‐17).
This
paradigmatic
shift
in
the
field
of
international
development
towards
collaborative
approaches
has
placed
participatory
monitoring
as
a
key
mechanism
capable
of
providing
the
SDGs
with
‘local
relevance
and
ownership’
and
closed
feedback
loop
mechanisms
–
both
characteristically
missing
from
the
MDGs
(CARE
International,
2016:
15).
Given
the
much
larger
breadth
of
targets
of
the
SDGs,
many
encourage
that
multi-­‐
channel
and
multi-­‐stakeholder
approaches
be
adapted
into
the
framework’s
processes
to
ensure
that
local
dimensions
of
poverty
are
integrated
into
all
of
the
agenda’s
objectives
and
indicators
to
improve
its
effectiveness
(Leadership
Council
of
the
Sustainable
Development
Solutions
Network,
2015:
3-­‐7).
III.3
Participation
and
Social
Accountability
The
prioritisation
of
participation
in
the
planning
and
implementation
of
development
projects
is
not
an
entirely
contemporary
aspiration,
as
the
beginning
of
the
sector’s
deployment
of
bottom-­‐up
procedures
coincided
with
the
decline
of
top-­‐down
and

14. DV410
Page
14
of
40
41743
centralised
approaches
back
in
the
1980s
(Gigler
et
al.,
2014:
211;
Elliot,
2013:
31).
Encouraged
by
the
findings
that
a
high
degree
of
local-­‐relevance
is
invariably
linked
to
improved
project
outcomes,
development
organisations
are
increasingly
striving
to
ensure
that
all
aspects
of
a
programme’s
design
are
representative
of
‘the
specific
local,
historical,
socio-­‐cultural,
and
institutional
conditions’
of
the
context
in
which
it
is
being
implemented
(Elliot,
2013:
31).
The
shortcomings
of
the
MDGs
to
improve
global
poverty
have
put
increasing
pressure
on
organisations
to
adopt
a
results-­‐based
approach
to
programme
management.
This
is
‘a
strategy
by
which
all
actors,
contributing
directly
or
indirectly
to
achieving
a
set
of
results,
ensure
that
their
processes,
products
and
services
contribute
to
the
achievement
of
desired
results
(outputs,
outcomes
and
higher
level
goals
or
impact)’
(United
Nations
Development
Group,
2011:
2).
At
the
core
of
the
approach
is
the
concept
of
social
accountability,
which
is
best
defined
as
a
system
of
accountability
that
can
be
exercised
on
demand
and
amplify
the
voices
of
beneficiaries
through
direct
interaction
and
participation
throughout
all
stages
of
a
development
project’s
implementation
(United
Nations
Development
Program,
2013:
3).
In
a
practical
sense,
social
accountability
is
composed
of
two
key
elements:
a
sustained
dialogue
between
the
administrators
and
recipients
of
development
assistance,
and
a
platform
that
allows
for
collaborative
problem
solving
(Shkabatur,
2014:
158).
These
key
elements
are
best
maintained
when
implementing
agents
assume
a
participatory
approach
to
programme
management.
Participatory
approaches
to
development
management
may
be
distinguished
from
one
another
by
the
degree
to
which
beneficiaries
are
engaged
in
informing
and
facilitating
a
project’s
design
or
policy’s
formulation
(Van
Wicklin
III
&
Gurkan,
2013:
8).
Engagement
with
beneficiaries
should
be
initiated
with
the
intention
of
meeting
their
‘expectations
of
change’
and
use
their
input
to
facilitate
improved
development
outcomes
(World
Bank,
2013:
8).
Ultimately,
programmes
should
contain
well-­‐functioning
feedback
loops,
which,
according
to
Whittle
(2013:
1),
are
‘mechanisms
that
allow
regular
citizens
to
influence
both
the
selection
and
quality
of
development
initiatives
implemented
by
aid
providers
and
government
agencies.’
Participatory
monitoring
is
one
such
mechanism
that
allows
beneficiaries
to
continuously
provide
input
on
their
own
satisfaction
or

15. DV410
Page
15
of
40
41743
perceptions
of
a
development
programme,
giving
organisations
the
ability
to
utilise
this
input
to
guide
programme
design
and
implementation
(CARE
International,
2016:
3).
III.4
Participatory
Monitoring
Conventional
monitoring
strategies
provide
‘information
on
where
a
policy,
program,
or
project
is
at
any
given
time
(and
over
time)
relative
to
respective
targets
and
outcomes’
(Kusek
&
Rist,
2004:
13).
Given
the
rise
in
popularity
of
bottom-­‐up
approaches
to
development
management,
conventional
monitoring
practices
have
come
under
scrutiny,
as
it
is
no
longer
feasible
or
sufficient
to
apply
standardised
indicators
to
the
monitoring
of
locally
implemented
projects
as
they
tend
to
ignore
the
‘diverse
needs
and
priorities
of
local
communities’
(CARE
International,
2016:
5).
Moving
away
from
these
tenets
of
traditional
approaches,
participatory
monitoring
makes
use
of
an
‘active
participation
of
project
beneficiaries,
project-­‐affected
people,
communities
and
other
primary
stakeholders’
(Van
Wicklin
III
&
Gurkan,
2013:
2).
The
general
paradigmatic
shift
to
collaborative
approaches
in
development
has
led
to
a
radical
alteration
of
‘who
initiates
and
undertakes
the
process,
and
who
learns
and
benefits
from
the
findings’
(Guijt
and
Gaventa,
1998,
cited
in
CARE
International,
2016:
3).
Increasingly,
project
managers
are
encouraged
to
do
more
than
simply
listen
to
the
voices
of
beneficiaries,
and
make
use
of
a
system
that
encourages
all
programme
stakeholders
to
actively
collaborate
with
one
another
to
develop
a
consensus
around
what
the
expected
outcomes
of
a
project
should
be
and
how
achievement
or
the
progress
towards
these
outcomes
will
be
measured.
Participatory
monitoring
enables
beneficiaries
to
integrate
themselves
in
the
processes
of
‘collecting
required
data,
undertaking
analysis,
and
developing
practical
action
plans
to
resolve
identified
problems’
(CARE
International,
2016:
3,
Van
Wicklin
III
&
Gurkan,
2013:
2).
Through
participatory
monitoring,
gaps
in
contextual
knowledge
are
filled,
equipping
programme
managers
with
improved
capabilities
to
holistically
assess
the
quality
of
projects
through
the
eyes
of
beneficiaries.
By
assessing
the
“real
experiences”
associated
with
a
project

16. DV410
Page
16
of
40
41743
from
the
perspective
of
its
beneficiaries,
assessments
of
the
quality
of
its
outcomes
and
impacts
become
more
comprehensive
(CARE
International,
2016:
3-­‐23).
Participatory
monitoring
helps
create
an
‘indigenous
accountability
mechanism’,
which
enables
average
citizens
to
engage
in
“shadow
reporting”
and
‘oversight
of
the
data
passing
through
the
official
systems’
of
development
organisations
(CARE
International,
2016:
2).
Development
programmes
benefit
tremendously
from
participatory
monitoring
as
elements
of
‘accountability,
responsiveness,
and
effectiveness’
become
inbuilt
to
their
design,
all
of
which
are
capable
of
fostering
feelings
of
local
ownership
within
recipient
communities,
as
well
as
mutual
understanding
between
the
beneficiaries
and
programme
implementers
(CARE
International,
2016:
12-­‐13).
Local
ownership
improves
the
working
relationship
between
stakeholders,
and
provides
enhanced
incentives
for
beneficiaries
to
engage
in
the
programme
at
every
stage
in
order
to
ensure
results
are
achieved
and
long-­‐lasting
(CARE
International,
2016:
2-­‐14).
III.5
Beneficiary
Feedback
Beneficiary
feedback,
a
subset
of
participatory
monitoring,
is
closely
associated
with
the
concept
of
social
accountability.
Beneficiary
feedback
broadly
refers
to
‘comments,
suggestions,
statements
of
appreciation
and
criticisms
expressed
by
the
recipients
of
aid
projects’
(Jump,
2013:
10).
Analogous
with
beneficiary
feedback,
beneficiary
feedback
systems
(BFS)
are
the
tools
that
monitors
use
to
take
a
‘systematic
approach
to
collecting
the
views
of…key
stakeholders
about
the
quality
and
impact
of
work
undertaken
by
a
development
agency’
(Jacobs,
2010:
40).
Recognising
that
‘development
results
are
subjective
and
vary
between
interest
groups’,
BFS
contributes
to
‘learning
at
the
local
level
by
measuring
‘different
dimensions
of
satisfaction’
based
on
the
experiences
of
beneficiaries
located
across
all
sites
of
a
project
(Jacobs,
2010:
39-­‐40).
Through
this,
‘data
can
capture
the
perspectives
of
those
who
are
often
most
marginalised’
(Jacobs,
2010:
40),
adding
much
needed
robustness
to
“generic
indicators”
most
often
used
in
practice
(CARE
International,
2016:
14).
“Granular
information”
can
provide
much
needed
clarity
to
perplexing
‘inconsistencies
between
complementary
indicators’,
such

17. DV410
Page
17
of
40
41743
as
the
case
of
an
education
project
where
monitoring
reports
have
indicated
that
school
enrolment
rates
have
risen,
but
literacy
levels
of
the
students
either
remain
stagnant
or
are
falling
(CARE
International,
2016:
14).
Contextual
knowledge
gained
through
BFS
can
provide
project
managers
a
better
understanding
of
the
linkages
between
the
inputs
and
outputs
of
programmes,
and
therefore
are
much
more
capable
of
addressing
the
discrepancy
of
the
results
from
the
bottom-­‐up
(CARE
International,
2016:
2-­‐14).
It
is
important
to
note
that
such
discrepancies
were
often
flagged
in
MDG
monitoring
reports;
therefore,
it
is
crucial
that
BFS
and
participatory
monitoring
are
used
more
throughout
the
SDG
campaign
to
avoid
circumstances
such
as
these.
These
systems
can
assist
in
providing
more
holistic
and
optimal
reports
from
project
monitoring.
III.5.1
Obstacles
to
Collecting
Beneficiary
Feedback
Despite
the
emerging
wave
of
interest
in
participatory
approaches
to
development
management,
actually
translating
them
from
an
‘ideal
into
reality
has
proven
to
be
elusive’
(Gigler
et
al.,
2014:
211).
International
development
agencies,
governments
and
non-­‐governmental
organisations
(NGOs)
are
constantly
being
faced
with
time,
cost,
and
distance
limitations
when
attempting
to
collect
beneficiary
feedback.
Internal
organisational
constraints
ultimately
impede
upon
bridging
of
the
gap
between
listening
and
responding
to
beneficiary
feedback.
This
section
will
provide
an
overview
of
some
of
the
key
challenges
faced
by
organisations
attempting
to
integrate
more
participatory
aspects
into
their
monitoring
practices
and
collect
beneficiary
feedback.
The
subsequent
sections
will
then
outline
some
of
the
strategies
and
tools
being
adopted
to
overcome
these
obstacles,
and
the
implications
that
their
introduction
can
provide
to
future
monitoring
of
development
programmes.
“The
most
significant
barrier
to
implementing
feedback
systems
appears
to
be
the
incentives
that
shape
management
and
organisational
behaviour”
(Jacobs,
2010:
62).
Installing
results-­‐based
monitoring
systems
into
the
design
of
a
project
or
within
an
organisational
framework
is
not
an
easy
task
as
it
‘requires
continuous
commitment,
time,
effort,
and
resources’
(Kusek
&
Rist,
2004:
2).
With
these
challenges,
it
is
difficult

18. DV410
Page
18
of
40
41743
for
organisations
to
see
how
the
benefits
of
BFS
can
ever
outweigh
the
costs
(Jacobs,
2010:
62).
Without
the
complete
buy-­‐in
from
and
willingness
within
organisations
to
‘review
their
procedures
and
attitudes,
and
change
them
where
necessary’,
BFS
cannot
be
truly
operationalised
(Guijt
&
Gaventa,
1998:
5).
According
to
the
CDA,
most
recipient
feedback
‘does
not
make
its
way
up
the
organizational
chain
to
effectively
influence
the
overall
strategies
and
policies’
(2011:
13).
Without
strict
quality
standards
and
indicators
to
ensure
that
the
feedback
systems
of
development
organisations
‘are
based
on
mechanisms
that
do
offer
users
significant
levels
of
control’,
BFS
are
often
deployed
in
a
manner
that
only
meets
what
minimum
standards
may
be
in
place
(Brett,
2003:
6).
In
turn,
programme
managers
have
become
complacent
in
conducting
consultations
and
engaging
with
beneficiaries
to
the
highest
of
standards,
often
failing
to
gain
the
optimal
amount
of
feedback
necessary
to
offer
any
added-­‐benefits
to
whatever
conventional
monitoring
strategies
may
be
in
use
(Brett,
2003:
6).
By
performing
what
appears
to
be
simply
a
box-­‐ticking
exercise,
organisations
are
able
to
designate
labels
such
as
participatory,
accountable,
and
transparent
onto
their
practices
and
“brand-­‐image”.
“The
ways
that
aid
providers
evaluate
their
staff,
partners,
programs,
and
overall
impacts
need
to
include
assessments
of
how
effectively
they
engage
with
a
broad
range
of
people
–
and
to
what
end”
(Anderson
et
al.,
2012:
133).
From
the
perspective
of
project
implementers,
engaging
with
beneficiaries
and
collecting
feedback
is
sometimes
seen
as
an
insurmountable
task.
As
the
pressures
mount
for
development
organisations
to
deliver
‘greater
development
effectiveness’
and
‘tangible
results’
(Kusek
&
Rist,
2004:
xi),
field
staff
are
overburdened
with
the
responsibility
to
produce
extensive
reports
for
their
superiors
and
donors.
It
is
worth
mentioning
that
the
reports
produced
by
field
staff
are
rarely
ever
used
for
immediate
and
on-­‐the-­‐ground
purposes,
as
the
information
is
generally
first
‘systematized,
reported
to
headquarters,
or
collated
and
analysed
at
an
agency-­‐wide
level’
(CDA,
2011:
2).
Gaining
an
understanding
of
the
local
priorities
and
existing
capacities
of
a
recipient
community
entails
a
significant
amount
of
time
be
spent
engaging
with
beneficiaries,
time
which
implementers
admit
they
do
not
have
(Anderson
et
al.,
2012:
126).
On
another
note,
access
to
beneficiaries
is
often
impeded
for
a
numbers
of
reasons,

19. DV410
Page
19
of
40
41743
including
but
not
limited
to
security
threats
posing
a
risk
to
field
staff,
as
well
as
geographic
isolation
making
it
physically
difficult
or
expensive
to
gain
access
to
target
groups
(Anderson
et
al.,
2012:
128).
From
the
perspective
of
beneficiaries,
it
is
common
for
them
to
lack
an
understanding
of
the
purpose
of
feedback
systems,
and
this
leads
to
many
being
fearful
of
the
retribution
that
would
precede
the
disclosure
of
personal
information
or
critical
assessment
of
a
project,
particularly
those
that
are
government
sponsored
or
initiated.
For
many
living
in
poverty,
there
can
high
perceived
costs
of
engagement.
Often
times,
beneficiaries
lack
the
time
to
properly
utilise
an
established
feedback
system,
as
they
are
often
obliged
to
work
long
hours
throughout
the
day
or
they
do
not
have
the
resources
such
as
vehicles
to
travel
to
wherever
feedback
collection
may
be
taking
place
(Leavy
&
Howard,
2013,
10).
Tools
for
engagement
can
also
be
inaccessible
to
marginalised
groups,
such
as
women
and
girls,
if
societal
norms
bar
them
from
expressing
themselves
or
leaving
the
confines
of
their
household.
III.6
Information
and
Communication
Technologies
and
the
Promise
of
User-­‐Generated
Content
The
diffusion
of
simple
technologies
such
as
mobile
phones
across
the
developing
world
has
increased
the
potential
of
information
and
communication
technologies
(ICTs)
to
assist
in
scaling
up
existing
participatory
monitoring
methods
and
overcoming
the
aforementioned
challenges
(Callendar,
2014:
4).
Through
technology-­‐enhanced
tools,
programme
managers
are
better
able
to
engage
with
last-­‐mile
users
of
programmes
–
the
most
geographically-­‐isolated
or
socially-­‐marginalised
–
which
consequently
improves
the
utility
and
value
of
monitoring
data
as
they
become
more
demonstrative
of
problems
that
monitors
would
have
otherwise
been
unable
to
assess
or
overlooked.
Through
this
technology,
a
project’s
targeting
is
sharpened,
and
therefore
enhance
its
outcomes
and
impacts.
With
added
pressure
to
provide
holistic
evidence
of
the
outcomes
and
impacts
of
projects,
development
practitioners
are
calling
for
innovative
feedback
systems
designed
in
formats
that
lend
themselves
to
‘easy
reporting
and
use

20. DV410
Page
20
of
40
41743
by
managers
and
other
decision-­‐makers
at
various
levels
in
organizations’
(CDA,
2011:
13).
Currently,
feedback
systems
fall
into
one
of
the
following
three
types:
1. Type
A:
Principal-­‐initiated
and
managed
feedback
systems
are
utilised
most
often
by
the
directors
of
government
or
development
agencies.
They
enable
principals
to
gain
‘real-­‐time
feedback
on
problem
hot-­‐spots,
the
effectiveness
of
their
departments
and
to
initiate
appropriate
remedial
actions’
(Galtung,
2014).
2. Type
B:
Manager-­‐initiated
and
controlled
feedback
systems
work
best
for
government
administrators
trying
to
assess
the
performance
of
a
‘specific
health
service
or
school
district’,
or
programme
managers
monitoring
the
implementation
of
infrastructure
development
project
(Galtung,
2014).
3. Type
C:
User-­‐initiated
and
owned
feedback
systems
are
used
the
least
within
development
due
to
a
general
lack
of
cited
evidence
of
its
utility
for
programme
monitoring.
Currently,
Type
C
systems
are
more
prominent
within
the
commercial
sector
as
crowdsourcing
review
sites
such
as
Yelp
and
Trip
Advisor
have
revolutionised
how
average
citizens
are
able
to
express
their
perceptions
and
ratings
of
a
product
or
service
to
other
consumers.
In
the
development
context,
Type
C
systems
engage
with
citizens
to
allow
them
to
take
matters
into
their
own
hands
to
report
and
resolve
the
issues
that
affect
them
directly
(Galtung,
2014).
It
should
be
noted
that
these
classifications
are
to
act
as
key
points
of
reference
for
the
remainder
of
the
thesis.
According
to
Galtung
(2014),
a
‘robust
and
resilient’
feedback
system
is
one
that
contains
a
combination
of
all
three
types
so
that
the
shortcomings
of
each
are
offset
by
the
others.
However,
this
ideal
is
most
often
unfulfilled
in
practice,
as
the
feedback
systems
used
by
large
development
organisations
often
only
resemble
types
A
or
B.
With
no
complementary
systems
present,
type
A
and
B
systems
do
not
function
optimally,
as

21. DV410
Page
21
of
40
41743
their
effectiveness
is
then
ultimately
contingent
upon
the
organisational
politics,
management
structures,
bureaucracy,
and
budget
of
managing
stakeholders.
The
decision
to
use
beneficiary
feedback
as
“actionable”
indicators
to
assess
a
programme
is
left
ultimately
to
whether
or
not
managing
stakeholders
perceive
that
the
benefits
of
the
use
of
such
systems
significantly
outweigh
its
costs
(Jacobs,
2010:
62).
The
lack
of
sufficient
sanctions
that
development
actors
face
when
making
use
of
type
A
and
B
systems,
compounds
the
problems
of
field
staff
feeling
already
overburdened
with
tasks
and
the
lack
of
resources
to
be
able
to
employ
an
effective
BFS
(Brett,
2003:
18).
Often,
the
only
force
that
encourages
the
use
of
these
systems
are
the
demands
made
from
managing
stakeholders
and
donors
of
projects.
For
these
reasons,
despite
having
the
intentions
of
using
BFS
to
better
engage
with
beneficiaries,
the
lack
of
incentives
inbuilt
to
type
A
and
B
systems
makes
it
so
that
the
feedback
loop
between
agencies
and
users
remains
un-­‐closed
(Brett,
2003:
19).
Type
A
and
B
systems
are
also
both
prone
to
be
driven
by
a
project
cycle,
meaning
that
they
can
“turned
on
and
off”
by
principals
or
managers
at
any
given
moment
(Galtung,
2014).
In
these
circumstances,
there
is
no
way
for
beneficiaries
to
ensure
the
sustainability
of
the
feedback
system
and
collaborative
engagement
with
programme
implementers
after
the
project
has
been
completed.
For
these
reasons,
many
practitioners
are
recommending
third-­‐party
monitoring
(TPM)
systems
as
a
solution.
TPM
systems
can
be
differentiated
from
standard
systems
in
the
way
that
its
procedures
rely
upon
the
efforts
of
‘parties
that
are
external
to
the
project
or
the
program’s
direct
beneficiary
chain
or
management
structure
to
assess
whether
intended
outputs,
outcomes,
and
impacts
have
been
achieved
by
the
project’
(Van
Wicklin
III
&
Gurkan,
2013:
2).
Existing
TPM
systems
within
the
development
sector
are
quite
flawed
in
regards
to
how
much
they
empower
beneficiaries
to
influence
the
management
decisions
in
regards
to
a
project’s
implementation.
Therefore,
it
has
been
suggested
that
models
should
be
sought
from
exogenous
sources,
potentially
the
private
sector,
a
point
which
this
paper
will
touch
further
upon.
Closest
to
what
an
ideal
TPM
system
would
resemble
are
Type
C
systems.
Type
C
systems
are
comparable
to
what
are
known
in
the
commercial
sector
as
open
consumer

22. DV410
Page
22
of
40
41743
feedback
systems.
In
well-­‐functioning
private
markets,
firms
are
driven
to
provide
consumers
with
a
‘constantly
improving
stream
of
high-­‐quality
products
and
services’
as
they
must
continually
modify
their
goods
in
accordance
with
consumers’
variable
wants
and
needs
to
increase
their
revenues
and
profits
(Whittle,
2013).
In
this
way,
companies
are
held
accountable
to
consumers
through
their
purchase
decisions
which
are
made
on
the
basis
of
perceived
expectations
of
the
standard
of
goods
and
services
that
should
be
offered
to
them.
Technological
advancements
and
the
growth
of
the
World
Wide
Web
since
the
mid-­‐1990s
have
given
rise
to
the
popularity
of
social
content
sites,
which
allow
consumers
to
submit
unsolicited
product
reviews
(O’Connor,
2010:
754).
This
type
of
peer-­‐to-­‐peer
sharing
of
information
is
known
in
the
literature
as
user-­‐generated
content
(UGC),
which
‘refers
to
media
content
created
or
produced
by
the
general
public
rather
than
by
paid
professionals
and
primarily
distributed
on
the
Internet’
(Daugherty
et
al.,
2008:
16).
‘User-­‐generated
content
is
rapidly
gaining
traction
as
an
input
into
the
consumer
purchase
decision
making
process’
(O’Connor,
2010:
754-­‐755).
“Collaboration
between
individuals
has
come
to
the
force
in
a
manner
unimaginable
in
the
past,
making
it
more
difficult
for
marketers
to
craft
sales
messages
and
position
them
in
front
of
the
consumer”
(O’Connor,
2010:
754-­‐755).
In
a
study
conducted
by
Wasserman,
one
in
three
internet
users
claimed
that
their
purchase
decisions
are
influenced
by
the
content
submitted
by
other
consumers
onto
review
sites
(2007:
cited
in
Williams
et
al.,
2010:
118).
A
recent
study
shows
that
people
prefer
the
reviews
of
other
consumers
over
the
opinions
of
experts,
with
peer
reviews
being
preferred
by
a
margin
of
6
to
1
(Creamer,
2007
cited
in
Williams
et
al.,
2010:
118).
Particularly
within
the
hospitality
industry,
review
sites
such
as
Yelp
or
Trip
Advisor,
have
demonstrated
how
online
between-­‐customer
communication,
or
UGC,
has
massive
implications
on
consumer
decision-­‐making
(Chevalier
&
Mayzlin,
2006:
2;
Niinen
et
al.,
2007
cited
in
Barreda
&
Bilgihan,
2013:
263).
Instead
of
establishing
a
website
that
simply
provides
users
with
a
consolidated
list
of
professional
reviews
of
destinations
and
hotels,
Trip
Advisor
has
turned
the
travel
and
tourism
industry
on
its
head.
Today,
Trip
Advisor
is
the
‘largest
online
network
of
travel
consumers’
(O’Connor,
2010:
754).
With
the
rise
in
legitimacy
of
UGC,
hotel
chains
and
managers
have
become
increasingly

23. DV410
Page
23
of
40
41743
focused
on
meeting
the
expectations
of
guests,
but
also
in
engaging
with
them
at
a
level
which
closed
feedback
loop
are
created,
as
management
decisions
are
taken
to
increase
the
overall
quality
of
their
services
(Berreda
&
Bilgihan,
2013:
266;
O’Connor,
2008:
57).
According
to
Berreda
and
Bilgihan
(2013:
274),
‘the
real
brand
is
whatever
people
say
it
is’.
Measuring
customer
satisfaction
is
proving
to
be
the
dominant
approach
to
assessing
and
analysing
performance
in
the
commercial
sector,
which
has
led
companies
to
shift
their
operations
to
be
more
customer-­‐oriented
(Jacobs
et
al.,
2010:
41).
Recognising
the
benefits
that
user-­‐owned
feedback
systems
have
conferred
within
the
commercial
sector,
variations
of
feedback
systems
analogous
to
those
utilised
in
the
commercial
sector
have
begun
to
be
piloted
in
an
effort
to
create
positive
feedback
loops
missing
from
the
development
sector.
By
definition,
type
C
systems
do
not
rely
on
the
buy-­‐in
or
permission
of
either
principals
or
managers,
therefore
in
theory
they
should
be
able
to
be
initiated
at
any
given
moment
by
users
(i.e.
beneficiaries)
(Galtung,
2014).
According
to
Galtung
(2014),
a
greater
sense
of
ownership
over
the
tool
leads
to
citizens
becoming
more
likely
to
‘contribute
their
time
and
efforts
to
resolving
problems’,
which
then
leads
to
a
shift
in
the
onus
of
responsibility
to
solve
problems
to
a
‘greater
sense
of
shared
responsibility
and
an
orientation
towards
finding
lasting
solutions’
(Galtung,
2014).
Unfortunately,
few
organisations
have
succeeded
in
building
a
feedback
system
that
resembles
or
is
capable
of
generating
the
impact
that
a
stand-­‐
alone
user-­‐owned
feedback
system
such
as
Trip
Advisor,
has
had
on
the
commercial
sector.
Many
of
those
who
have
attempted
to
do
so,
face
challenges
in
terms
of
securing
the
commitment
and
buy-­‐in
of
service
providers,
government
agencies,
or
development
organisations.
The
system
requires
this
legitimacy
if
it
is
to
be
effective
in
providing
beneficiaries
with
a
platform
through
which
contributions
of
their
feedback
contribute
to
the
enhancement
in
accountability,
transparency
and
effectiveness
of
development
programmes.
The
wider
perceptions
of
BFS
ultimately
determine
the
level
at
which
an
organisation
feels
obliged
to
engage
with
beneficiaries
in
a
collaborative
manner
to
acknowledge
the
concerns
that
they
have
reported
and
follow-­‐up
with
improvements
to
the
project
or
policy.

24. DV410
Page
24
of
40
41743
IV.
Case
Analysis
IV.1
Check
My
School
Developed
by
the
Affiliated
Network
for
Social
Accountability
in
East
Asia
and
the
Pacific
(ANSA-­‐EAP),
Check
My
School
(CMS)
is
a
participatory
monitoring
initiative
‘that
aims
to
promote
transparency
and
social
accountability
in
the
Philippine
education
sector’
(Shkabatur,
2014b:
4).
CMS
demonstrates
an
innovative
approach
to
performance-­‐
based
tracking
—
‘tracking
the
provision
of
services
in
schools’
in
a
sustained
and
long-­‐
term
manner
(World
Bank,
2013)
that
combines
‘on-­‐the-­‐ground
community
monitoring
with
the
use
of
[ICTs]’
(Shkabatur,
2014a:
150).
The
model
operates
under
the
assumption
that
‘community-­‐driven
validation
and
easy
access
to
data
via
the
Internet
will
enable
government
officials
and
citizens
to
highlight
issues
of
concern
and
identify
potential
solutions’
(Shkabatur,
2014a:
151).
It
involves
the
use
of
three
key
levers
responsible
for
the
initiative’s
success:
data
validation,
community
engagement,
and
issue
resolution
(Check
My
School,
n.d.;
Shkabatur,
2014a:
151).
Infomediaries
are
recruited
from
local
communities,
and
are
trained
to
visit
public
schools
and
compare
data
provided
to
ANSA-­‐EAP
by
the
Department
of
Education
(DepEd)
with
data
collected
during
the
surveillance
of
the
actual
conditions
in
the
schools
(Shkabatur,
2014a:
150-­‐151).
With
internet
penetration
levels
in
the
Philippines
around
approximately
30%,
and
ICT
literacy
still
quite
low,
the
infomediaries
act
as
an
important
bridge
between
the
technological
application
and
a
large
swath
of
the
population
(Shkabatur,
2014a:
161).
Monitoring
data
is
consolidated
and
uploaded
onto
the
CMS
online
platform
which,
in
order
to
remain
transparent,
is
made
accessible
to
the
general
public
to
allow
them
to
comment
on
the
published
UGC
(Shkabatur,
2014b:
4).
With
the
official
Department
of
Education
data
covering
more
than
44,000
public
schools
online,
CMS
encourages
parents,
students,
school
administrators,
and
CSOs
to
conduct
their
own
monitoring
of
schools
and
upload
information
onto
the
platform
via

25. DV410
Page
25
of
40
41743
SMS
messaging,
email
and
social
media
platforms
such
as
Facebook
and
Twitter
(Whittle,
2013:
159).
In
recent
years,
the
‘text
hotline’
has
become
one
of
the
most
widely
used
tools
due
to
the
simplicity
of
the
process
–
citizens
upload
their
feedback
to
the
CMS
platform
via
SMS
in
an
easy-­‐to-­‐follow
format.
Similar
to
Trip
Advisor,
which
generates
profiles
complete
with
traveler
reviews,
comments,
and
raw
photos,
CMS
is
in
the
process
of
creating
profiles
for
each
school
that
is
registered
for
the
service.
Each
of
the
profiles
display
‘key
indicators
and
measures
of
performance’
and
‘present
official
DepEd
information
alongside
the
data
validated
by
CMS
in
an
easily
accessible
and
user-­‐
friendly
way’
(World
Bank,
2015).
By
establishing
a
quality
standard
through
the
use
of
the
key
indicators,
DepEd
and
school
administrators
are
able
to
track
the
improvements
or
changes
in
performance
of
each
school,
and
assess
how
their
performance
compares
to
those
of
other
schools.
By
setting
benchmark
indicators
and
standards
of
performance,
school
superintendents
become
more
motivated
to
not
only
sign
onto
the
CMS
service
but
make
the
necessary
changes
to
improve
their
school’s
overall
performance.
Key
to
the
application’s
success
are
the
strong
linkages
between
CSOs
and
state
agencies,
and
a
strong
endorsement
from
DepEd.
CMS
emerged
at
a
time
when
the
federal
government
was
calling
for
the
development
of
third-­‐party
monitoring
systems
to
coincide
with
decentralization
reforms
(World
Bank,
2015).
DepEd
had
even
developed
its
own
manager-­‐owned
online
feedback
system.
However,
it
never
achieved
as
much
public
buy-­‐in
as
CMS
because
of
its
poor
functionality
(World
Bank,
2015).
Furthermore,
data
was
not
presented
in
a
clear
and
accessible
manner
for
the
public
(World
Bank,
2015).
The
government’s
commitment
to
responding
immediately
to
legitimate
complaints
uploaded
onto
the
CMS
platform
motivates
average
citizens
to
voice
their
grievances
(Shkabatur,
2014a:
173;
Whittle,
2013:
12).
The
relationship
between
citizens
and
DepEd
forged
through
the
CMS
platform
assures
users
that
their
assessments
and
collaborative
problem-­‐solving
efforts
are
initiatives
worth
taking,
as
they
have
been
proven
to
be
sufficient
mechanisms
for
achieving
true
and
long-­‐lasting
reforms
in
the
Philippine
education
system.
Though
only
introduced
in
2011,
it
is
remarkable
how
large
the
user-­‐base
of
CMS
has
grown
within
such
a
short
timeframe,

26. DV410
Page
26
of
40
41743
and
the
reforms
that
have
been
made
in
the
public
school
systems.
Through
these
impactful
reforms
shaped
by
the
feedback
of
ordinary
citizens,
the
application
has
proven
to
the
public
that
they
are
all
capable
of
becoming
agents
of
change
within
their
local
communities.
As
the
willingness
and
the
buy-­‐in
of
the
public
to
the
use
the
application
is
high,
which
therefore
ensures
the
sustainability
of
the
programme.
Though
the
CMS
platform
was
launched
independently
by
ANSA-­‐EAP,
an
organisation
external
to
DepEd,
securing
the
commitment
from
government
agencies
was
a
necessary
pre-­‐condition
for
initiating
the
CMS
system
(Shkabatur,
2014b:
12).
Monitoring
of
schools
can
also
only
occur
with
permission
given
by
both
‘local
education
education
officials
(division
supervisors
and
superintendents)
and
the
administrators
of
the
selected
schools’
(Shkabatur,
2014b:
26).
Gaining
this
permission
is
a
key
task
of
the
infomediaries,
who
often
present
the
officials
with
a
DepEd
letter
of
endorsement
for
CMS,
which
assists
in
reassuring
them
of
CMS’s
legitimacy
(Shkabatur,
2014b:
26).
Without
DepEd’s
support,
CMS
would
otherwise
become
inoperable.
Though
it
shares
some
commonalities
with
a
stand-­‐alone
feedback
system
such
as
TripAdvisor,
DepEd
has
now
become
a
key
stakeholder
of
CMS
and
has
in
a
sense
appropriated
itself
as
the
application’s
manager
(Shkabatur,
2014b:
12).
For
this
reason,
CMS
can
only
be
classified
as
a
type
B
feedback
system.
Based
on
an
assessment
of
current
prototypes
of
user-­‐
owned
and
initiated
feedback
systems,
no
system
can
be
found
that
presents
itself
as
a
stand-­‐alone
system
capable
of
monitoring
across
all
sectors
of
international
development
or
an
agenda
that
tackles
a
diverse
range
of
dimensions
of
poverty
such
as
the
SDGs.
However,
one
particular
organisation,
Integrity
Action,
is
currently
demonstrating
a
substantial
amount
of
potential.
IV.
2
Integrity
Action
and
DevelopmentCheck
Referred
to
as
the
“Trip
Advisor
for
International
Development”,
Integrity
Action
is
an
anti-­‐corruption
organisation
based
in
London
that
is
making
headways
within
the
development
sector.
Integrity
Action
has
developed
a
set
of
tools
and
mechanisms
that
enable
local
communities
to
monitor
projects
and
public
service
delivery
initiatives.

27. DV410
Page
27
of
40
41743
Influenced
by
private
sector
models
of
closed
consumer
feedback
loops,
Integrity
Action
has
managed
adapt
conventional
open
consumer
feedback
systems
to
assist
in
the
monitoring
of
international
development
projects
and
public
service
delivery.
UGC
is
chief
among
the
private
sector
devices
adapted
into
Integrity
Action’s
monitoring
strategies,
as
it
is
a
powerful
mechanism
that
ensures
that
accountability
and
transparency
remain
core
principles
throughout
a
programme’s
implementation
(Galtung,
2014).
In
briefs
to
leading
development
organisations
such
as
the
World
bank,
Fredrik
Galtung
of
Integrity
Action,
encourages
everyone
to
realise
that
an
effective
agency-­‐wide
feedback
system
is
one
that
makes
use
of
a
combination
of
all
three
aforementioned
feedback
systems
(i.e.
types
A,
B,
and
C)
(Galtung,
2014).
Integrity
Action
is
striving
to
fill
the
gap
in
market
for
a
stand-­‐alone
user-­‐generated
review
site
that
compliments
existing
type
A
and
B
systems
of
development
institutions.
Integrity
Action
was
founded
on
three
principles
–
autonomy,
transparency,
and
achieving
individualised
and
positive
feedback
loops
–
which
Galtung
(2014)
cites
as
characteristics
that
must
be
a
part
of
any
open
feedback
system.
The
principles
have
provided
Integrity
Action
with
the
foundation
to
be
able
to
carry
out
more
constructive
identification
of
local
issues,
collaborative
formulation
of
solutions,
and
assurance
that
the
agreed-­‐upon
solutions
materialise
into
concrete
actions
(Integrity
Action,
n.d.).
The
strategy
taken
by
Integrity
Action
to
introduce
closed
feedback
loops
to
development
project
implementation
and
service
delivery
initiatives
is
one
that
the
organisation
has
developed
independently,
known
as
the
Community
Integrity
Building
(CIB)
approach
(Integrity
Action,
n.d.).
This
approach
aims
to
re-­‐establish
the
working
relations
between
all
stakeholders
involved
in
a
project’s
implementation.
CIB
consists
of
five
key
phases:
context
sensitivity,
joint
learning,
building
an
evidence
base
through
data
collection,
constructive
engagement
and
collaborative
problem
solving,
and
lastly
closing
the
feedback
loop
by
resolving
reported
to
them
from
the
public
(Galtung,
2016:
15).
Local
community
members
are
chosen
to
become
monitors
who
undergo
training
to
be
able
run
community
project
assessments
and
facilitate
consultations
with
project
stakeholders.
These
consultations
are
especially
important
as
they
provide
crucial

28. DV410
Page
28
of
40
41743
information
relevant
to
assessing
if
a
project
has
successfully
achieved
all
of
its
objectives
and
most
importantly
whether
beneficiaries
are
satisfied
and
perceive
the
project
to
have
brought
improvements
to
their
lives.
Integrity
Action
has
developed
an
online
and
mobile
app
called
DevelopmentCheck,
which
allows
monitors
to
upload
monitoring
data
and
information
including
photos
and
official
documents
onto
each
project’s
“profile
page”.
Published
information,
including
scores
of
the
three
key
development
indicators
–
access
to
information,
community
engagement,
and
project
effectiveness
–
can
be
accessed
by
any
member
of
the
public
to
be
used
as
leverage
to
motivate
implementing
agents
to
uphold
their
responsibilities
and
address
reported
problems
in
a
sufficient
and
effective
manner.
IV.2.1
The
Fix-­‐Rate
DevelopmentCheck
uses
a
key
performance
indicator
(KPI),
known
as
the
Fix-­‐Rate,
which
places
an
emphasis
on
the
monitoring
of
programmes
to
be
results-­‐oriented,
in
a
manner
that
goes
beyond
simply
identifying
or
reporting
problems
(Galtung,
2014).
The
Fix-­‐Rate
measures
‘the
incidence
with
which
transparency
and
accountability
problems
are
resolved
to
the
satisfaction
of
key
stakeholders’
and
act
as
a
benchmark
and
tracking
indicator
for
organisations
and
government
agencies
to
assess
their
performance,
address
any
problems
they
may
have
with
transparency
and
accountability,
and
work
towards
satisfying
programme
stakeholders
particularly
beneficiaries
(Galtung,
2016:
1-­‐
3).
The
Fix-­‐Rate’s
design
follows
the
example
of
Transparency
International’s
Corruption
Perceptions
Index
(CPI),
which
is
grounded
in
the
results
of
opinion
surveys
completed
by
business
people
and
assessments
of
a
country’s
performance
based
on
the
findings
of
a
group
of
reputable
‘country/risk/expert
analysts’
(Transparency
International,
2010).
In
a
similar
strategy
of
focusing
on
the
“perceptions”
of
key
stakeholders
as
indicators,
the
Fix-­‐Rate
is
a
tool
for
measuring
these
particular
outputs
of
a
project:
‘resolution
of
citizen
complaints,
or
improvements
in
public
service
delivery
based
on
problems
identified
by
the
stakeholders
of
the
service’
(Galtung,
2016,
4).
In
reference
to
Integrity
Action’s
systems,
then,
the
inputs
necessary
to
achieve
this
outputs
come
in

29. DV410
Page
29
of
40
41743
the
form
of
activities
or
policy
changes
such
as
community
meetings,
public
hearings,
or
social
audits
(Galtung,
2016:
4).
The
Fix-­‐Rate
is,
in
turn,
a
holistic
measure
of
whether
or
not
‘inputs
empower
citizens
and
public
office
holders,
individually
or
collectively,
to
achieve
a
specific
fix,
and
therefore
an
improved
outcome
that
is
in
the
public
interest’
(Galtung,
2016:
4).
IV.2.2
Obstacles
to
Mainstreaming
the
Fix-­‐Rate
as
a
Key
Performance
Indicator
for
International
Development
Through
personal
communication
with
managers
and
employees
at
Integrity
Action,
the
researcher
gained
an
understanding
of
some
of
the
constraints
faced
by
the
organisation
to
mainstream
DevelopmentCheck
and
the
Fix-­‐Rate.
Integrity
Action
takes
a
collaborative
approach
to
encouraging
service
providers
and
development
organisations
to
take
corrective
action
against
problems,
rather
than
simply
“naming
and
shaming”
them
for
their
bad
performance.
This
makes
it
more
difficult
to
gain
buy-­‐
in
from
implementing
organisations,
as
they
are
less
likely
to
believe
that
the
assessments
produced
are
likely
to
impact
their
“brand
image”.
By
trying
to
stay
true
to
their
collaborative
approach
to
the
identification
of
issues
and
formulation
of
solutions,
the
organisation
faces
extreme
barriers
as
they
try
to
make
these
practical
tools
of
performance
monitoring
operational
across
the
development
sector.
Another
obstacle
that
the
organisation
faces
is
that
the
effectiveness
of
the
system
still
relies
on
an
organised
presence
on
the
ground
to
ensure
that
the
information
collected
is
used
to
inform
the
management
decisions
of
implementing
organisations
to
improve
their
operational
practices.
In
other
words,
unlike
how
user
ratings
and
comments
on
Trip
Advisor
are
enough
of
a
motivational
factor
for
hotels
to
improve
areas
of
the
services
that
they
provide,
the
effectiveness
of
DevelopmentCheck
and
the
Fix-­‐Rate
still
rely
upon
the
assistance
of
human
intermediaries
to
directly
engage
with
those
actors
that
they
are
assessing.
Without
on-­‐the-­‐ground
assistance,
UGC
on
DevelopmentCheck
would
otherwise
be
left
unused.

30. DV410
Page
30
of
40
41743
V.
Discussion
The
present
study
sheds
light
on
the
effectiveness
and
limitations
of
current
participatory
monitoring
strategies
and
BFS
used
in
development.
Noting
the
limitations
and
performance
gaps
of
these
current
systems,
the
majority
of
evidence
suggests
that
a
radical
reform
is
needed
to
completely
deconstruct
and
overhaul
the
development
sector’s
current
reliance
on
‘supply-­‐led
performance
monitoring’
(Jacobs,
2010:
62).
The
researcher
suggests
that
user-­‐owned
and
initiated
feedback
systems
–
analogous
to
commercial
sector
models
of
open
consumer
feedback
systems
–
should
be
adapted
to
the
monitoring
of
development
projects
in
order
to
address
the
shortcomings
of
principal-­‐
and
manager-­‐owned
feedback
systems.
From
an
analysis
of
the
few
stand-­‐
alone
BFS
in
operation
in
development,
the
researcher’s
primary
objective
is
to
assess
their
utility,
limitations,
as
well
as
the
added
benefits
that
they
bring
to
the
monitoring
of
development
projects
and
public
service
initiatives.
Within
the
commercial
sector,
dedicated
review
sites
which
allow
members
of
the
public
to
review
products
and
services
have
massively
reshaped
the
way
in
which
consumers
make
purchase
decisions.
UGC
is
now
viewed
by
customers
as
‘unbiased
and
trustful
output’
(Barreda
&
Bilgihan,
2013:
263),
placing
companies
in
a
position
where
they
cannot
ignore
customer
reviews
since
doing
so
will
negatively
impact
their
brand
image.
While
companies
cannot
control
the
content
being
uploaded
onto
review
sites,
they
can
still
manage
it
by
engaging
in
more
proactive
dialogue
with
consumers
and
work
towards
making
customer-­‐oriented
management
decisions
that
result
in
improvements
to
their
service
operations
or
production
(O’Connor,
2008:
57;
Barreda
&
Bilgihan,
2013:
267).
If
most
shortcomings
of
type
A
and
B
systems
arise
from
the
‘incentives
that
shape
management
and
organisational
behaviour’
(Jacobs,
2010:
62),
then
a
plausible
solution
is
to
introduce
type
C
systems
into
development,
as
these
can
be
initiated
without
the
authorisation
of
principals
and
managers.
With
feedback
systems
being
increasingly
equated
to
better
field
work
processes
and
project
outcomes
(Jacobs
et
al.,
2010:
41),
project
managers
may
become
further
incentivised
to
focus
on
the
priorities
of
beneficiaries
rather
than
those
of
other
internal
and
external
stakeholders
(Jacobs,
2010:
56).

31. DV410
Page
31
of
40
41743
Before
commercial
sector
companies
began
to
acknowledge
the
credibility
of
consumer
feedback
and
the
implications
that
it
has
on
brand-­‐image,
review
sites
needed
to
gain
a
substantial
amount
of
buy-­‐in
from
citizens
who
recognised
the
utility
of
UGC
(Litvin
et
al.,
2008:
1).
However,
based
on
the
case
evidence,
it
would
appear
that
the
success
of
BFS
within
the
development
sector
requires
a
radical
restructuring
of
attitudes
towards
participatory
monitoring
and
the
‘incentives
that
shape
management
and
organisational
behaviour’
(Jacobs,
2010:
62).
Building
off
of
the
Fix-­‐Rate
deployed
by
Integrity
Action
as
a
KPI,
there
should
be
much
more
of
a
concerted
effort
within
the
sector
to
collectively
re-­‐evaluate
‘how
performance
and
impact
are
defined
and
measured,
and
as
a
result
how
funds
are
distributed’
(Jacobs,
2010:
57).
These
development
indicators
should
be
built
in
a
way
that
attaches
the
‘value
that
their
intended
beneficiaries
perceive
from
their
work’
to
the
amount
of
external
funding
one
would
receive
(Jacobs,
2010:
52).
Within
the
commercial
sector,
it
is
more
common
to
see
companies
valuing
the
satisfaction
of
customers
gained
from
the
products
or
services
that
they
have
purchased.
This
is
simply
because
consumer
satisfaction
is
directly
linked
to
the
bottom
line
of
companies
–
net
earnings
and
profits
(Jacobs
et
al.,
2010:
43).
At
the
present
moment
there
are
no
real
sector-­‐wide
mechanisms
that
would
truly
empower
beneficiaries
to
be
able
to
sanction
organisations
if
they
fail
to
deliver
the
intended
outcomes
of
a
project
(Brett,
2003:
23).
This
power
still
lies
with
senior
managers
and
donors
(Jacobs
et
al.,
2010:
40).
Therefore,
to
be
able
to
deploy
a
beneficiary-­‐oriented
measure
of
programme
effectiveness
like
the
Fix-­‐Rate
across
the
development
sector,
the
system
would
need
to
incrementally
gain
endorsements
from
managing
stakeholders
in
order
to
build
its
legitimacy.
As
displayed
in
the
case
of
CMS,
the
reliance
on
endorsements
from
those
which
the
application
seeks
to
review,
exemplifies
the
fact
that
it
is
harder
to
establish
a
stand-­‐alone
BFS
in
development
than
in
the
private
sector.
If
perceived
as
a
threat
to
their
“brand
image”,
it
remains
highly
unlikely
that
development
organisations
would
endorse
a
platform
that
would
use
unfiltered
and
unsolicited
information
from
beneficiaries
to
develop
a
sector-­‐wide
benchmark
for
performance,
without
strong
incentives
in
place.
The
development
of

32. DV410
Page
32
of
40
41743
incentives
strong
enough
to
entice
development
organisations,
particularly
the
larger
ones,
to
make
use
of
the
DevelopmentCheck
as
a
tool
to
monitor
their
performance
as
would
a
hotel
monitoring
their
reviews
on
Trip
Advisor,
remains
to
be
the
largest
hurdle
faced
by
Integrity
Action.
Technological
tools
for
gathering
feedback
have
become
ubiquitous
in
the
field
of
development.
However,
as
these
cases
outline,
ICT-­‐enhanced
tools
and
mechanisms
should
not
be
employed
as
replacements
to
traditional
systems
that
rely
upon
in-­‐person
gathering
and
dissemination
of
feedback
‘particularly
in
places
where
relationships
and
trust
are
essential
for
confidence
in
the
feedback
process’
(CDA,
2011:
22).
This
point
is
especially
true
for
the
case
of
Integrity
Action.
The
data
that
is
uploaded
onto
the
DevelopmentCheck
platform
is
not
used
to
“name
and
shame”,
but
rather
allow
external
monitoring
agents
and
the
local
community
members
to
engage
with
implementing
agents
in
a
collaborative
manner
to
encourage
them
to
resolve
problems
associated
with
a
project’s
implementation.
It
is
clear
from
this
that
having
an
‘organized
presence
on
the
ground’
is
still
a
necessary
element
for
achieving
the
successful
deployment
of
beneficiary
feedback
systems
with
technological
components
(Shkabatur,
2014b:
viii).
Within
developing
countries,
there
is
still
a
significant
‘need
for
human
intermediaries
to
bridge
both
the
overt
and
the
social
science
endowment
gaps
between
what
the
poor
have
and
what
they
need
in
order
to
use
ICT’
(Rajalekshmi,
2007:
22).
These
cases
show
that
current
type
C
systems
in
development
are
not
able
to
“stand
alone”
like
their
counterparts
in
the
private
sector.
From
this,
one
should
argue
that
multi-­‐channel
approaches
to
monitoring
are
still
most
effective
at
producing
information
that
can
guide
programmes
into
achieving
better
outcomes
(Jacobs
et
al.,
2010:
43).
Further,
it
is
important
to
recognise
that
context
matters,
and
therefore
tools
for
monitoring
a
programme
should
always
be
chosen
based
on
the
capacities
of
the
local
population
to
effectively
utilise
them.
Information
collected
via
participatory
monitoring
cannot
serve
as
a
substitute
for
the
rigorous
statistical
surveys
and
analysis
produced
through
conventional
monitoring
approaches.
However,
it
is
important
for
development
practitioners
to
recognise
the

33. DV410
Page
33
of
40
41743
robustness
and
complementarity
that
beneficiary
feedback
can
provide
to
conventional
monitoring
practices
(CARE
International,
2016:
23).
As
with
the
cases
described,
data
being
captured
from
ordinary
citizens
has
proven
to
be
a
powerful
tool
for
holding
public
officials
and
development
assistance
providers
accountable
for
the
services
they
offer.
UGC,
in
these
cases,
act
as
benchmarks
of
performance
for
development
projects
and
service
delivery
initiatives,
which,
without
the
BFS,
lacked
few
if
any
quality
standards
and
indicators
that
would
have
‘driven
internal
reflection’
or
provided
‘incentives
for
operating
units
to
improve’
(Jacobs
et
al.,
2010:
42).
The
case
of
CMS
exemplifies
the
benefits
that
can
be
derived
from
providing
citizens
with
feedback-­‐based
tools
to
monitor
programmes
and
public
services.
By
making
the
perceptions
of
beneficiaries
more
readily
available
for
the
assessment
by
others,
beneficiaries
are
inadvertently
given
‘more
influence
over
the
institutions
which
affect
their
lives’
(Jacobs
et
al.,
2010:
41).
CMS
provides
an
important
example
of
how
influential
and
useful
a
BFS
can
be
in
helping
monitor
the
performance
of
an
institution
as
large
as
the
Philippine
DepEd.
It
demonstrates
how
BFS
can
work
to
‘validate,
refute,
or
contextualize
official
data,
enriching
the
content
of
the
SDGs
and
providing
local,
national,
and
international
officials
with
more
reliable
and
actionable
information’
(CARE
International,
2016:
23).
This
is
because
it
is
the
beneficiary
who
is
best
able
to
assess
how
closely
an
intervention
meets
local
priorities
or
the
standards
of
implementation
at
which
they
believe
a
project
would
best
serve
at
producing
the
most
positive
outcomes
and
impacts
(Chambers,
2007,
cited
in
Jacobs
et
al.,
2010:
42).
Compared
to
its
predecessor,
the
SDG
agenda
is
composed
of
a
much
broader
set
of
goals,
each
made
up
of
significantly
more
detailed
and
fine-­‐tuned
targets
that
are
more
representative
of
the
the
multi-­‐dimensional
nature
of
each
set
of
poverty
alleviation
dimensions.
Being
able
to
provide
more
granular
and
micro-­‐level
information,
BFS
can
fill
in
the
gap
for
a
monitoring
system
that
will
assist
the
SDGs
further
place
‘perspectives
of
marginalised
groups
at
the
centre
of
analysis’
(Jacobs,
2010:
40).
Granular
data
gained
through
BFS
equip
implementing
staff
and
organisations
with
the
ability
to
understand
and
therefore
tackle
the
root
causes
of
the
problems,
which
the
SDGs
seek
to
address
(Theis,
2003:
1).

34. DV410
Page
34
of
40
41743
VI.
Conclusion
Overall,
this
research
was
conducted
to
assess
whether
it
would
be
possible
to
translate
commercial
sector
concepts
such
as
open
consumer
feedback
loops
and
UGC
into
development
to
improve
monitoring
strategies
used
to
assess
the
performance
of
implementing
organisations
and
the
effectiveness
of
projects.
Few
attempts
have
been
made
in
the
development
sector
to
establish
stand-­‐alone
user-­‐initiated
and
owned
feedback
systems
that
could
be
used
to
assess
the
performance
of
development
organisations
in
relation
to
sector-­‐wide
benchmarks.
Therefore,
far
fewer
attempts
have
been
made
to
shed
light
on
the
successes
or
the
challenges
faced
when
trying
to
make
such
a
system
operational
within
the
development
sector.
The
main
findings
of
this
dissertation
are
as
follows.
Having
a
fully
operational
stand-­‐
alone
BFS
in
the
development
sector
ultimately
relies
upon
a
complete
overhaul
of
the
reliance
on
‘supply-­‐led
performance
monitoring’
(Jacobs,
2010:
62).
Additionally,
alterations
must
be
made
to
what
incentivises
development
organisations
to
buy
into
such
a
system,
and
further
use
beneficiary
feedback
to
assess
their
own
performance.
Another
finding
is
that
the
effectiveness
of
type
C
systems
still
depends
on
the
active
engagement
of
human
intermediaries.
This
further
proves
the
point
that
ICT-­‐enhanced
mechanisms
should
not
be
employed
as
complete
replacements
to
traditional
systems
of
monitoring
that
require
in-­‐person
collection
and
dissemination
of
feedback.
A
multi-­‐
channel
approach
to
monitoring
projects
remains
to
be
recommended.
One
should
always
assess
whether
the
context
in
which
the
tools
are
deployed
is
conducive
to
their
effective
utilisation.
Finally,
though
current
type
C
systems
are
not
without
their
faults,
they
have
proven
to
be
effective
in
empowering
citizens
with
the
ability
to
guide
programme
implementation
and
public
service
delivery
in
a
manner
that
is
better
able
at
meeting
local
priorities.
The
capability
of
UGC
and
BFS
to
provide
managers
with
robust
and
actionable
information,
necessary
for
more
effective
and
successful
programme
outcomes,
will
show
to
be
useful
in
the
efforts
of
the
international
community
to
reach
the
targets
outlined
in
the
SDGs
by
2030.

35. DV410
Page
35
of
40
41743
VI.1
Further
Research
As
a
whole,
there
needs
to
be
a
radical
reshaping
of
attitudes
and
the
incentives
that
drive
development
organisations
to
adopt
participatory
approaches
to
monitoring,
which
in
recent
years
have
come
to
include
user-­‐initiated
and
owned
feedback
systems.
Further
research
is
needed
to
determine
the
right
“carrot
and
stick”
approach
–
combination
of
rewards
and
punishments—that
would
be
most
effective
in
encouraging
development
organisations
and
government
agencies
to
endorse
and
buy
into
a
type
C
system
capable
of
creating
sector-­‐wide
benchmarks
for
performance.

36. DV410
Page
36
of
40
41743
Bibliography
Anderson,
M.,
Brown,
D.,
Jean,
I.,
2012.
Time
to
Listen:
Hearing
People
on
the
Receiving
End
of
International
Aid.
CDA
Collaborative
Learning
Projects,
Cambridge,
Mass.
Barreda,
A.,
Bilgihan,
A.,
2013.
An
analysis
of
user-­‐generated
content
for
hotel
experiences.
JHTT
4,
263–280.
Bernardi,
R.,
Chiara,
F.D.,
2011.
ICTs
and
monitoring
of
MDGs:
a
case
study
of
Kenya
HIV/AIDS
monitoring
and
evaluation
in
a
donor
multi-­‐agency
context.
Information
Technology
for
Development
17,
24–41.
Brett,
E.A.,
2003.
Participation
and
accountability
in
development
management.
The
Journal
of
Development
Studies
40,
1–29.
Burkey,
S.,
1993.
People
First:
A
Guide
to
Self-­‐Reliant,
Participatory
Rural
Development.
Zed
Books,
London.
Callendar,
T.,
2014.
Participatory
Monitoring
and
Accountability:
Conceptualizations
and
Implementation,
and
the
Implications
for
Post-­‐2015
(Literature
Review).
United
Nations
Development
Group.
CARE
International,
2016.
The
Power
of
Participatory
Monitoring
in
Making
the
Sustainable
Development
Goals
a
Reality.
CARE
International.
CDA,
2011.
Feedback
Mechanisms
in
International
Assistance
Organizations.
CDA,
Cambridge,
Mass.
CheckMySchool
[WWW
Document],
n.d.
CheckMySchool.
URL
http://www.checkmyschool.org/
(accessed
8.16.16).
Chevalier,
J.,
Mayzlin,
D.,
2006.
The
Effect
of
Word
of
Mouth
on
Sales:
Online
Book
Reviews.
Journal
of
Marketing
Research
43.
Coyne,
B.,
Godinot,
X.,
Tran,
Q.,
Viard,
T.
(Eds.),
2014.
Challenge
2015:
Towards
Sustainable
Development
that
Leaves
No
One
Behind.
International
Movement
ATD
(All
Together
in
Dignity)
Fourth
World.
Daugherty,
T.,
Eastin,
M.S.,
Bright,
L.,
2008.
Exploring
Consumer
Motivations
for
Creating
User-­‐Generated
Content.
Journal
of
Interactive
Advertising
8,
16–25.
de
Vaus,
D.,
2001.
Research
Design
in
Social
Research.
SAGE
Publications
Ltd,
London.

37. DV410
Page
37
of
40
41743
Elliot,
J.,
2013.
An
Introduction
to
Sustainable
Development,
Fourth.
ed.
Routledge,
Oxon.
Galtung,
F.,
2016.
The
Fix-­‐Rate:
A
Key
Metric
for
Transparency
and
Accountability
(Working
Paper
No.
2).
Integrity
Action.
Galtung,
F.,
2014.
The
World
Bank’s
100%
citizen
feedback
agenda:
a
daunting
challenge
and
an
amazing
opportunity.
Feedback
Labs.
Gigler,
B.-­‐S.,
Bailur,
S.
(Eds.),
2014.
Closing
the
Feedback
Loop:
Can
Technology
Bridge
the
Accountability
Gap?,
Directions
in
Development
-­‐-­‐
Public
Sector
Governance.
Work
Bank,
Washington,
D.C.
Gigler,
B.-­‐S.,
Custer,
S.,
Bailur,
S.,
Dodds,
E.,
Asad,
S.,
Gagieva-­‐Petrova,
E.,
2014.
Closing
the
Feedback
Loop:
Can
Technology
Amplify
Citizen
Voices?,
in:
Gigler,
B.-­‐S.,
Bailur,
S.
(Eds.),
Closing
the
Feedback
Loop:
Can
Technology
Bridge
the
Accountability
Gap?
The
World
Bank,
Washington,
D.C.,
pp.
211–278.
Hancké,
B.,
2009.
Intelligent
Research
Design:
A
Guide
for
Beginning
Researchers
in
the
Social
Sciences.
Oxford
University
Press,
Oxford.
Hilhorst,
T.,
Guijt,
I.,
2006.
Participatory
monitoring
and
evaluation:
a
process
to
support
governance
and
empowerment
at
the
local
level
(A
Guidance
Paper).
Royal
Tropical
Institute.
Integrity
Action
[WWW
Document],
n.d.
Integrity
Action.
URL
http://integrityaction.org/
(accessed
8.16.16).
Jacobs,
A.,
2010.
Creating
the
Missing
Feedback
Loop.
IDS
Bulletin
41,
56–64.
Jacobs,
A.,
Barnett,
C.,
Ponsford,
R.,
2010.
Three
Approaches
to
Monitoring:
Feedback
Systems,
Participatory
Monitoring
and
Evaluation
and
Logical
Frameworks.
IDS
Bulletin
41,
36–44.
Jump,
L.,
2013.
Beneficiary
Feedback
Mechanisms:
A
Literature
Review.
Development
Initiatives.
Kusek,
J.Z.,
Rist,
R.C.,
2004.
Ten
Steps
to
a
Results-­‐based
Monitoring
and
Evaluation
System:
A
Handbook
for
Development
Practitioners.
World
Bank
Publications.
Leadership
Council
of
the
Sustainable
Development
Solutions
Network,
2009.
Indicators
and
Monitoring
Framework
for
Sustainable
Development
Goals.

38. DV410
Page
38
of
40
41743
Leavy,
J.,
Howard,
J.,
2013.
What
Matters
Most?
Evidence
from
84
Participatory
Studies
with
Those
Living
with
Extreme
Poverty
and
Marginalisation.
Litvin,
S.W.,
Goldsmith,
R.E.,
Pan,
B.,
2008.
Electronic
word-­‐of-­‐mouth
in
hospitality
and
tourism
management.
Tourism
Management
29,
458–468.
Manning,
R.,
Harland
Scott,
C.,
Haddad,
L.,
n.d.
Whose
Goals
Count?
Lessons
for
Setting
the
Next
Development
Goals
(IDS
Bulletin
No.
44.5-­‐6).
Institute
of
Development
Studies,
Oxford.
Masset,
E.,
White,
H.,
2004.
Are
Chronically
Poor
People
being
Left
Out
of
Progress
towards
the
Millennium
Development
Goals?
A
quantitative
analysis
of
older
people,
disabled
people
and
orphans.
Journal
of
Human
Development
5,
279–297.
O’Connor,
P.,
2010.
Managing
a
Hotel’s
Image
on
TripAdvisor.
Journal
of
Hospitality
Marketing
&
Management
19,
754–772.
O’Connor,
P.,
2008.
User-­‐Generated
Content
and
Travel:
A
Case
Study
on
Tripadvisor.Com,
in:
O’Connor,
P.,
Hopken,
W.,
Gretzel,
U.
(Eds.),
Information
and
Communication
Technologies
in
Tourism.
Springer
Wien,
New
York.
Rajalekshmi,
K.G.,
2007.
E-­‐Governance
Services
Through
Telecenters:
The
Role
of
Human
Intermediary
and
Issues
of
Trust.
Information
Technologies
and
International
Development
4,
19–35.
Sanga,
D.,
2011.
The
Challenges
of
Monitoring
and
Reporting
on
the
Millennium
Development
Goals
in
Africa
by
2015
and
Beyond.
The
African
Statistical
Journal
12.
Shkabatur,
J.,
2014a.
Check
my
School:
A
Case
Study
on
Citizens’
Monitoring
of
the
Education
Sector
in
the
Philippines,
in:
Gigler,
B.-­‐S.,
Bailur,
S.
(Eds.),
Closing
the
Feedback
Loop:
Can
Technology
Bridge
the
Accountability
Gap?,
Directions
in
Development
-­‐-­‐
Public
Sector
Governance.
Work
Bank,
Washington,
D.C.
Shkabatur,
J.,
2014b.
Check
My
School:
A
Case
Study
on
Citizen’s
Monitoring
of
the
Education
Sector
in
the
Philippines
(Case
Study).
The
World
Bank,
Washington,
D.C.
Suter,
S.,
Fishman,
A.,
2015.
Lessons
on
monitoring
and
reporting
from
the
MDGs
and
other
major
processes
(Supporting
Document).
World
Resources
Institute
(WRI).
Theis,
J.,
2003.
Rights-­‐based
monitoring
and
evaluation:
A
discussion
paper
(Discussion
Paper).
Save
the
Children.
Tranparency
International,
2010.
Long
Methodological
Brief
(Brief).
Transparency
International.

39. DV410
Page
39
of
40
41743
United
Nations,
2003.
Indicators
for
Monitoring
the
Millennium
Development
Goals:
Definitions,
Rationale,
Concepts
and
Sources
(Handbook).
United
Nations
Development
Group,
New
York.
United
Nations
Development
Group,
2011.
Results-­‐Based
Management
Handbook
(Handbook).
United
Nations
Development
Group.
United
Nations
Development
Program,
2013.
Reflections
on
Social
Accountability:
Catalyzing
Democratic
Governance
to
Accelerate
Progress
towards
the
Millennium
Development
Goals.
New
York.
Van
Wicklin
III,
W.,
Gurkan,
A.,
2013.
Participatory
and
Third
Party
Monitoring
in
World
Bank-­‐Financed
Projects:
What
Can
Non-­‐State
Actors
Do?,
How-­‐To
Notes.
World
Bank,
Washington,
D.C.
Whittle,
D.,
2013.
How
Feedback
Loops
Can
Improve
Aid
(and
Maybe
Governance)
(Essay).
Center
for
Global
Development.
Williams,
R.,
van
der
Wiele,
T.,
va
Iwaarden,
J.,
Eldridge,
S.,
2010.
The
importance
of
user-­‐generated
content:
the
case
of
hotels.
The
TQM
Journal
22,
117–128.
World
Bank,
2015.
Check
My
School
in
the
Philippines
[WWW
Document].
World
Bank.
URL
http://www.worldbank.org/en/topic/governance/brief/check-­‐my-­‐school-­‐in-­‐the-­‐
philippines
(accessed
8.17.16).
World
Bank,
2013.
Participatory
Development
and
the
World
Bank
(World
Bank
Discussion
Papers
No.
183).
Washington,
D.C.

40. DV410
Page
40
of
40
41743
Appendix
Appendix
A.
List
of
Interviewees
Name Affiliated Organisation Position
Date
Interviewed
Medium
Carla Benham World Vision UK
Accountability
Advisor
August 5, 2016 Skype
Alfredo Briseno World Bank
Governance
Consultant
July 29, 2016 Skype
Hayley Capp CARE International
Inclusive
Governance
Officer
July 5, 2016 Face-to-Face
Mariéta Fall
World Bank Governance
Analyst
August 2, 2016 Skype
Fredrik Galtung
Integrity
Action/DevelopmentCheck
President
& Co-
Founder
August 16,
2016
Face-to-Face
Edward Irby
Integrity
Action/DevelopmentCheck
Programme
Manager
August 8, 2016 Face-to-Face
Ann-Sofie Jepersen World Bank
Officer
(Operational
Policy and
Country
Services)
August 4, 2016 Skype
Tim Kelly World Bank
Lead ICT
Policy
Specialist
August 4, 2016 Skype
Laura McDonald SIMLab CEO July 25, 2016 Skype
Keith McLean World Bank
Lead Social
Development
Specialist
July 25, 2017 Skype
Soledad Muniz InsightShare
Head of
Participatory
Video for M&E
July 26, 2018 Skype
Utpal Raj Misra World Bank
Governance
Specialist
August 8, 2016
Skype
Annalisa Renna
Integrity
Action/DevelopmentCheck
Grants
Manager
August 3, 2016 Face-to-Face