The document details a workshop conducted by 4th Wheel Social Impact and the Swedish Institute aimed at enhancing social program design and evaluation in India by identifying and measuring contextualized indicators for Sustainable Development Goals (SDGs). It discusses the importance of data collection, theory of change (TOC) development, and best practices for indicator identification through practical group exercises. The workshop brought together development practitioners to share experiences and improve capacities in utilizing data and creating effective theories of change to track progress towards achieving global goals.

1. IDENTIFYING CONTEXTUALISED
INDICATORS TO MEASURE
IN INDIA
SDGS
SHOWCASING BEST PRACTICES ON INDICATOR
IDENTIFICATION AND IMPACT EVALUATION
15th December, 2018
Powai, Mumbai

2. Introduction
Who we are
Keeping social impact management at the centre, 4th Wheel Social
Impact is committed to strengthening social programs in India by
improving the way they are designed, implemented, monitored and
evaluated. The organization believes the integration of data, technology
and partnerships will enable the achievement of the Sustainable
Development Goals (SDGs).
the workshop.
The Swedish Institute is a public agency
that encourages interest and trust in
Sweden around the world. They
operate in the fields of culture,
education, science and
business by seeking to
strengthen ties and promote
development. To encourage
dialogues around SDGs,
Swedish Institute supported
Scope and Purpose of the Workshop
Theory on the data
revolution, indicator
identification and
designing Theory of
Change (ToC) for
programs
Application of
theory and practice
learnt through
group work
Practical examples
on ToC development
in different social
contexts
The workshop was spread across three components –
Agenda
1. THEORY
1.1
1.2
1.3
2.1
2.2
2.3
3.1
3.2
3.3
The Data
Revolution
Indicator
Development
Theory of
Change
Saathi
EcoSolutions
Magic Bus
Medha
Skilling
Services
Develop
TOCs
Identify
Indicators
Linkage to
SDG Targets
2. PRACTICE
3. APPLICATION
1 2 3

3. Establish the need for collecting
high quality data at the local level
to track progress being made
towards achievement of the global
goals
Share best practices in identifying
and defining indicators to measure
key elements of a program
Improve capacities of
stakeholders on designing
theories of change (ToCs) with
well-established causal
pathways (external factors,
assumptions, program
design)
Develop a list of
contextualised indicators
based on the global indicator
framework by the Inter-
Agency and Expert Group on
SDG indicators (IAEG-SDGs)
What we aimed to achieve
The UN World Data Forum advocates the important role data and
statistics will play in the implementation of the 2030 Agenda for
Sustainable Development. It is believed that shared goals and metrics
among stakeholders will provide a coherent narrative for action,
mobilize all actors involved in a particular area, galvanize the
community to develop clear strategies for implementation, raise the
levels of available finance and spur innovations and technologies
needed to implement them.
Only 33% of the workshop participants had developed ToCs for their
organisations while 29% stated that they face challenges in indicator
development. Limited experiences of participants in ToC and indicator
development substantiated the need for the workshop to contribute
to the global goals at the national level.
The workshop facilitated this macro-level goal by bringing together
development practitioners from across India involved in planning,
implementing, monitoring and evaluating social programs in India.
Why
Sectors of focus
Rationale of the workshop
1
2
3
4

4. Workshop
proceedings
1. Contributing
to the data
revolution
““Timely and usable data is
critical for informed
decision-making,
monitoring of progress
and evaluation of
outcomes, especially in
the context of the post-
2015 development
1
agenda.”
The workshop began with a
discussion on the need and
benefits of collecting and
producing high quality data to
contribute to the data
revolution. Techniques
ranging from geo-referencing
and satellite imagery to
collect big data as well as
using smartphones and
tablets for surveys at the local
level were explored.
Management buy-in is
essential to ensure
regular data collection
and using of data to
manage program
impact
Resource allocation
for comprehensive
and regular data
collection is
essential
Collection of reliable
data requires
equipped and
capacitated
enumerators
Structured and defined
program goals and
outcomes are
important to collect
usable and multi-
functional data
Key takeaways
Overcoming budget and time constraints in
collecting good data
Rationalise your data needs and accordingly modify the
evaluation design
Look for reliable secondary data sources
Use technology to reduce the time and resources spent
in data entry and cleaning
Select a sample size suitable to your time and budget
1
The UN Secretary General's Independent Expert Advisory Group
on a Data Revolution for Sustainable Development
Using technology to
ensure quality and reliable
data
Collecting data that is usable for measuring and
tracking program goals
Train your data collection team to
ensure that they understand the value
of data collection and how they are
contributing to broader program goals
Have a well-trained and prepared team
member(s) to monitor your data
collection process
Ensure that all data points fit into at least one of the
outcome indicators in your ToC
Ensure a multi-stakeholder approach in data
collection to establish triangulation and validity of
data
Questions and discussions
1 2 3 4

5. ““An indicator is a variable
that measures key elements
of a program. Indicators serve
as a scientific and researched
method to highlight
problems, identify trends, and
contribute to the process of
priority-setting, policy
formulation and evaluating
2
and monitoring progress.”
2. Identifying the
right indicators
Types of indicators
Key components of
defining indicators
Best practices in
indicator identification
A well-defined and
measurable indicator should
include four components –
the population/stakeholder
for whom change is being
measured and tracked, the
target for change expected to
be brought about, the
timeline in which change will
be brought about, and the
threshold/lower limit to
consider impact outcomes as
'good enough’
Indicators should be SMART by
focusing on the 4 Ws – what,
where, who, when and 1 H – how
much
All stakeholders – their realities
and views - should be taken into
account while developing
indicators
Select the most suitable
indicators that capture the
essence of the program to
ensure cost and time
effectiveness
Involve subject experts to
validate indicators
Ensure that all indicators relate
to the program targets
Participants were
oriented on three key
aspects of indicator
development –
Key takeaways
2
Health in Sustainable Development
Planning: The Role of Indicators,
World Health Organisation (2002)
1
2
3
1 2 4
5
6
3
Parameters for defining targets and thresholds
Keeping information from existing databases, define
your targets and thresholds to suit the context of your
program's reach and scale
Refer to reliable sources of existing data and literature
to gauge already existing frameworks and previous
research
Ideal timeline for testing and defining indicators
Program design, its scale and reach, management buy-
in and funder requirements are aspects that should
shape indicator development and timelines
There is no universal timeline for testing and defining
indicators
Ensuring involvement of all stakeholders
All stakeholders representative should be
consulted at different stages of project
planning, design and implementation
Multi-stakeholder views and opinions should
be included with a focus on intersectionality
Questions and discussions

6. ““The first step in designing a right-fit data collection strategy is to
create a solid theory of change. A theory of change is a clear visual map
that represents how a program will make an impact on the world. It
illustrates what goes into a program, what gets done, and how the
3
world is expected to change as a result.”
A ToC should start with
the program goal and
then sequentially move
to the outcomes and
required pre-conditions
to reach the program
goal (backward
mapping/back-casting)
Define indicators that
will measure progress
for every expected
outcome
A ToC should not
remain as just a
linear
chart/graphic. It
should tell a story
about how your
program will
progress
Assumptions
should be put in
at each layer of
the ToC to
account for
external factors
that can impact
program success
Key takeaways
3
Theory of Change: Laying the Foundation for Right-Fit Data Collection,
Innovations for Poverty Action (2016)
3. Developing a
comprehensive ToC
Developing a comprehensive and all-
encompassing ToC
Theory of Change vs Log-frame
Analysis (LFA) - the difference
Do not look at ToCs as a linear
representation. Make multiple
connections between and across
different outcomes
Build in as many assumptions as
possible. They are foundational for
program successes and
shortcomings
Design the ToC keeping in mind
contextual realities of your
program area and beneficiaries
Be flexible and ready to adapt your
ToC across the program cycle
ToCs start with the goal and then
progress downwards i.e. backward
mapping/back-casting as against
LFAs which start with the entry
point of the program and then
progress upwards to the goal
ToCs, through assumptions, take
into account external influencers.
LFAs do not leave scope for
assumptions
Interventions (inputs) are featured
in an LFA. In ToC, the focus in only
on outcomes and goals
1 3
2 4
Questions and discussions

7. ““Assumptions are a critical
aspect of any ToC. They can
determine the potential
shortcomings and
successes of your program.”
““A ToC cannot be looked at
as a linear representation
of a program. While this
may be possible in theory,
during on-ground
implementation there can
be multiple connections
and relations between pre-
conditions.”
““A strong and
comprehensive ToC must
have a multi-stakeholder
perspective. Along with
outcomes and pre-
conditions, assumptions
must carefully account for
the voices of all
stakeholders. “
CONTEXTUAL
EXAMPLES
Theory of
Change –

8. Participants
developed
their own
organisation’s
Theory of
Change
They
identified
indicators
and created
an indicator
list
Participants
mapped their
indicators to
the IAEG-SDG
indicator list
Participants
collated
indicators
within the
group
The three
groups
presented
their
indicator
list
Practical
application of
workshop
learnings -
GROUP ACTIVITY
Based on their
interest, participants
were divided into
three groups
EDUCATION
HEALTH
SKILLING
Flow of practical exercises
and group activities
1
2
3

9. DEVELOPING A
THEORY OF CHANGE
PRESENTING
AND SHARING
IDENTIFYING
AND COLLATING
INDICATORS
During this practical exercise,
participants tackled thought-
provoking challenges as they
attempted the backward
mapping method instead of the
straightforward approach used in
logic model frameworks. They
contended in distinguishing
between activities and outcomes.
Participants also faced difficulties
in choosing a single theme for
their TOCs. This was because the
selected domains – health,
education and skilling – are vast
and in many situations even a
single program covered multiple
areas within the domain.
Participants selected a single
program of their organization and
outlined its goal. Based on the
established goal, they added
essential pre-conditions, in the
form of outcomes, which the
initiative would need to fulfill, in
order to achieve their goal.
Participants then came up with
an extensive indicator list which
would enable them to measure
their outcomes. They also
referenced a comprehensive list of
SDG indicators and linked their
indicators to co-relating SDGs.
Afterwards, participants shared
their program's TOC and indicator
list within the group and in-depth
discussions resulted in an
extensive list of all outcome
indicators within the themes of
education, health and skilling.
Using various materials, the three
groups designed an indicator
overview and each group
presented their indicator
overview.
Programs for which ToCs were
developed spanned the education
cycle of a student – from early
childhood education to higher
education. The indicators defined
for corresponding outcomes had a
multi-stakeholder approach. Apart
from students, educators/teachers
and the community, especially
parents, were considered.
Indicators were listed that could
potentially measure program
impact from the lens of different
stakeholders. Lastly, different
indicators related to infrastructure
at educational institutions were
also listed.
Education
The group working on health
looked at this broad domain from
different lenses - mental health,
menstrual health, water and its
impact on community health,
gender and its underlying
implications for good health.
Keeping in mind this broad range
of sub-themes, the group defined
indicators for each of these
emerging themes. This was
presented in terms of indicators
that should be prioritised across
these themes while designing a
ToC and indicator framework.
The group working in the area of
skill development traced
indicators across the life cycle of a
skill development program – from
mobilisation to job placements
and retention. Further, the group
came up with indicators on how
skill development can improve the
socio-economic status and overall
standard of living for trainees and
their families. Indicators were also
listed on how skill development
improves individual capacities in
terms of aspirations and control
on their own life decisions.
Health
Skilling

10. Feedback
and insights
Gained insights into development
of ToC
Will apply workshop learning
Felt that the workshop was
relevant to the development sector
Gained insights into development
of indicators
Perception Scores
81.71%
91.42%
89.14%
84%
“
“The orientation to ToCs was beneficial –
specifically the causal pathway and how it is a
better tool than LFA for program planning,
design, implementation and evaluation”
“The inputs on indicator development,
specifically on integrating them with the ToC
outcomes was highly beneficial.”
“Data revolution was a great concept. It could
serve as a good base for future workshops,
specifically the different statistical and
mathematical techniques to enable a data
revolution.”

11. Participating
organisations
ampUz
Atul Foundation
Bansidha & Ila Panda
Foundation
Antarang Foundation
Apnalaya
Arvind Foundation
Baker Tilly DHC
Dobility India (SurveyCTO)
Grameen Capital India Pvt.
Ltd.
IIT Kanpur
Dream a Dream
Godrej
IDOBRO Media and
Marketing services Private
Limited
Barefoot Edu Foundation
INREM Foundation
Cognify Labs
Impact India Foundation
Indian Metals & Ferro Alloys
Ltd
Learning Links Foundation
Medha
Omkar Foundation
One planet rating
Piramal Water Private Ltd.
Rizwan Adatia Foundation
Saathi EcoInnovations
JPPL
Magic Bus
One Future Collective
Sangli Mission Society
Setco Foundation
WNS
Toybank - The Opentree
Foundation
United Sisters Foundation
YUVA
UnLtd India

12. +91 22 62214191
+91 98258 85044
www.the4thwheel.com