This document provides a strategic advocacy framework to help organizations like Chintan monitor and evaluate their advocacy efforts. It recommends that Chintan develop a theory of change to integrate its programs, goals, and mission. The framework includes defining goals and interim outcomes and tracking activities. Monitoring and evaluation can help Chintan understand what is effective, adapt strategies, and demonstrate progress. However, advocacy can be difficult to evaluate due to shifting timelines and strategies. The document provides recommendations for Chintan to plan advocacy in its organizational context and become a learning organization that regularly reviews lessons from its work.

1. Thinking Strategically:
Applying a Strategic Framework to
Advocacy Activities
Prepared by:
Stephen Rabent, Yan Qu, Faisal Hassan, and Shannon Kennedy

2. Table of Contents
Executive Summary 1
Section One: Developing a Strategic Advocacy Framework 2
Importance and Use of Monitoring and Evaluation 3
The Challenges of Monitoring and Evaluating Advocacy 4
Overview of a Strategic Advocacy Framework
Creating a Theory of Change 5
Components of the Framework 7
The Framework in Action 15
Review 18
Recommendations and Action Items 19
Section Two: Social Impact Evaluation 20
Why Measure Social Impact 21
Introduction to Methodology 23
Frameworks
Longitudinal Survey 25
Key Informant Interviews 28
Focus Group Discussions 31
Recommendations 34
Appendix i

3. 1
Executive Summary
Introduction:
The following report provides a tool, and a way of thinking, for Chintan to engage in the
monitoring and evaluation of its advocacy efforts. We layout a Strategic Advocacy Framework
for Chintan that integrates its programs, activities, and goals, allowing you to see the
interconnection that exists between them. Within the model, monitoring and evaluation
techniques play a key role in allowing you to track your efforts. Perhaps most importantly, it
provides a body of knowledge and evidence for you to draw upon as you make decisions
regarding your advocacy strategies, activities, and goals. We also present a section on the use of
social impact evaluation and techniques to conduct them. However, we would like to make clear
that Chintan is not yet ready to engage in formal social impact evaluation. Once the Strategic
Advocacy Framework is implemented, Chintan will begin to develop the knowledge necessary to
engage in these activities.
Key Recommendations and Action Items:
1) Plan your Advocacy Work Within the Larger Organizational Context
Based on our interactions and discussions with Chintan, your organization would benefit
from periodically taking a step-back from your everyday, essential programmatic and
advocacy work and review how it fits within your broader mission. This exercise can be
beneficial for keeping your organization focused on its mission and reinforcing how its
activities lead to the changes you seek.
Action Item: Chintan should create a theory of change diagram that models how its
various activities, interim outcomes, goals, and mission interact.
2) Become a “Learning Organization”
If Chintan wants to grow and expand its reach while remaining a lean, efficient
organization, you must focus on incorporating the learnings from your activities into
future efforts. Simply identifying the components of the framework and filling in the
details of your work is not enough. Constant review of information, observations, and
lessons learned must be integrated into the fabric of Chintan’s everyday activities.
Action Item: Schedule time during monthly meetings for formal discussion of the
Strategic Advocacy Framework and commit to one quarterly meeting on the topic.
3) Engage in Conversations on Social Impact’s Role at Chintan
Chintan has expressed the desire to engage in efforts to measure its social impact.
However, you are currently not in the position to immediately begin to do so. These
techniques can be time and resource intensive, and require data Chintan does not have.
Action Item: Hold organization-wide discussions on how impact evaluation is, or is not,
an important tool for Chintan to utilize. Keep in mind that impact evaluations can be
done internally, or by external consultant teams.

4. 2
Section One: Developing a Strategic Advocacy Framework
Introduction
Section One of this report provides a Strategic Advocacy Framework that incorporates
monitoring and evaluation tools. Presented within a “theory of change” concept, it provides a
broad overview of the elements of a successful monitoring and evaluation program as well as the
challenges inherent to monitoring and evaluating advocacy efforts. This section specifically
focuses on: (1) How Chintan’s current programmatic work and advocacy efforts can be
incorporated into a broader, organization-wide strategy; (2) The creation of a formal monitoring
and evaluation tool; And (3) how the results can be used to inform policy strategies and move the
organization closer to meeting its overall goals.
Overview
A “Theory of Change” provides the context in which your programs, campaigns, and goals
operate. It models how activities are expected to result in the desired changes to policy and
behavior. Using a strategic approach to your work allows you to see the connections between
your efforts, track your progress, and measure the achievement of your ultimate mission as an
organization. As you read through this report, it is important to note that while it is presented as
a smooth process, the steps involved most often occur simultaneously or even out of order.
Within this Strategic Advocacy Framework, monitoring and evaluation tools provide Chintan
with the information necessary to guide your decisions and measure your progress. The advocacy
framework we propose draws upon best practices from numerous organizations and foundations.
It contains three main components that are necessary to identify and track in order to make full
use of a theory of change:
 Goal Creation is the centerpiece of any program or policy campaign. Identifying realistic
and meaningful outcomes sets the context for future organizational efforts.
 Interim Outcomes are the intermediate steps necessary to achieve your overall goals and
are useful in tracking progress towards achieving them. Chintan already engages in a
small amount of measuring, however these efforts should be expanded as well as inform
your decisions in relation to the achievement of Chintan’s overall mission.
 Activities are the actions you take to measure progress toward reaching goals and
outcomes. They track the programmatic work (inputs) that Chintan does on an everyday
basis, as well as your advocacy efforts. Currently Chintan does engage in activity
tracking (output tracking) and has a system to collect and aggregate this information.
Tying Chintan’s activities to the achievement of interim outcomes is the next in
evaluating progress towards your mission.

5. 3
The Importance and Uses of Monitoring and Evaluation
There is a growing external demand on organizations, particularly for non-governmental
organizations (NGO’s), to monitor and evaluate their programs and activities. Additionally, these
techniques can be extremely beneficial to Chintan internally. This technique can be beneficial
through:1
1. Tracking campaign efforts while allowing for comparison across activities and
programs.
2. Communicating to partners and stakeholders the resources required to achieve your
desired outcomes.
3. Creating an understanding of what activities have been most effective, what
circumstances and actions may have contributed to their success, and adapting your
approaches to focus on these efforts.
4. Communicating to stakeholders what you have learned about your activities
through more detailed quarterly and annual reports.
5. Tracking adjustments to your strategy, actions, and desired outcomes over time.
6. Assessing progress versus only focusing on the achievement of policy impacts.
The importance of a robust monitoring and evaluation structure is thus two-fold, informing both
external and internal actors. This information can lead to greater accountability towards donors
who help to fund Chintan’s work through its incorporation into quarterly and annual reports.
Furthermore, it will allow you to better understand whether you have made progress towards
accomplishing long-term policy or social change goals. Monitoring and evaluation as an
assessment tool helps to ensure that Chintan’s resources are being used effectively while having
the desired impact.
However, monitoring and evaluation techniques do not exist in a vacuum and alone they cannot
reach their full potential in informing Chintan’s decision making. We recommend that to
maximize its effect on the organization, these techniques are incorporated into a broader vision
of how Chintan creates policy change and meets its mission. Moreover, developing this broad
framework is often the starting point in designing a monitoring and evaluation scheme.2
It
provides the context and impetus behind decisions (the “10,000 meter high” view of the
organization’s actions) on how the monitoring and evaluation system is implemented (the “on
the ground” details of the organizations operations).This system provides key insight into how
Chintan approaches its work and the interconnections that exist between its programmatic
efforts, advocacy strategies, and ultimate policy goals.
1
“Advocacy Evaluation Mini-Toolkit: Tips and Tools for Busy Organizations,” 3.
2
“A Guide to Measuring Advocacy and Policy,” 11.

6. 4
The Challenges of Monitoring and Evaluating Advocacy
Advocacy efforts are often difficult to measure and there is not one traditional format for doing
so. While the principals of designing monitoring and evaluation tools and the purpose of them
remain nearly identical between advocacy and service evaluation, certain challenges are unique
to their application and must be confronted before the process is undertaken.3
Challenge One: Constantly Shifting Time Frames, Strategies, and Milestones
Achieving policy goals can take years of work. Oftentimes, progress is made in fits and starts,
creating periods of high activity followed by lulls. Moreover, strategies to achieve outcomes can
shift substantially over these long times horizons in response to changes in the social and
political context surrounding an issue area. As such, milestones and interim targets for the policy
campaign will shift as well.4
It is critical that monitoring and evaluation efforts also adjust to
ensure that the data and metrics collected are still useful for informing the policy advocacy
process.
Challenge Two: Contribution versus Attribution
The role of external events and concurrent factors that occur throughout these long time horizons
makes the attribution of one specific activity towards the achievement of an outcome very
difficult. The political and social environment that Chintan exists within is incredibly
complicated and crowded with actors. As a result, linking efforts to direct effects can be difficult
due to the presence of a vast amount of confounding variables. In most cases, efforts can only be
argued to have contributed to bringing about an interim outcome or goal, rather than directly
attributable to an effect.
Challenge Three: Extreme Time and Resource Constraints
Chintan is a highly successful and continuously busy organization with few excess staff and
resources to devote to monitoring and evaluation. As such, thorough internal monitoring and
evaluation techniques or contracting for external evaluation services for each program and policy
are most likely beyond the everyday capabilities or the organization. Instead, simplified
techniques for a select portfolio of activities and data must be developed to ensure that the
outcomes of the process can inform your work.
Despite these challenges, engaging in monitoring and evaluation is central to your continued and
future success. We have developed a strategic advocacy framework with these challenges and
your organizational capacity in mind, allowing you to implement and advance your work in an
effective manner.
3
Adapted from “Monitoring and Evaluating Advocacy: Companion to the Advocacy Toolkit,” and “A Guide to Measuring
Advocacy and Policy.”
4
“A Guide to Measuring Policy and Advocacy,” 8.

7. 5
Overview of the Proposed Framework
We recommend that Chintan adopt a Strategic Advocacy Framework that incorporates
monitoring and evaluation mechanisms that will aid in the strategic development and
management of programs intended to achieve policy goals. The monitoring and evaluation tool
will help you understand if your activities are having an impact on reaching your interim
outcomes and ultimate goals. These lessons learned and information can be used by Chintan to
better manage its efforts as well as build a case for incorporating your desired programs into the
meaningful policy.
Creating a “Theory of Change”
A “Theory of Change” is a conceptual model utilized in the mapping of a vision for achieving
your policy goal. It links together the activities, interim outcomes, and goals that support the
achievement of the broader mission of Chintan. It is an articulation of what changes Chintan
wants to see and how your work will contribute to these changes.5
This conceptual model can be
most easily expressed in a visual diagram, mapping how your organization translates its activities
into goal achievement.6
We have provided an example of this model applied to the E-Waste
campaign in the Appendix.
Developing a theory of change is intended to be a collaborative process, where key players
within the organization can come to an agreement about the desired impact of their work and
collectively create a clear picture of what success looks like. An important component of this
process is incorporating both interim outcomes and goals into mapping the theory of change.
This process can be instrumental in building an understanding of goals, setting expectations, and
evaluating success.7
Discussions around the How, What, and Why of Chintan’s policy and advocacy work can be
useful in framing discussions about a theory of change. The development of activities, interim
outcomes, and goals can clarify How Chintan’s work contributes to What policy and advocacy
goals you have. Chintan’s mission and the ultimate impact it seeks to achieve help to frame Why
it engages in the preceding activities. Discussing these questions are essential for internally
designing an evaluation and monitoring program:8
1. What activities and interim outcomes should be measured? What information is most
important for you to know?
2. How often, and for how long, should you collect this data?
3. How rigorous should your process be and how much of your resources should you
devote to collecting and analyzing our data? Will you be collecting quantitative or
qualitative data?
4. How will your results be used?
5
“Developing Monitoring and Evaluation Framework for Budget Work Projects: A How-To Guide For Practitioners,” 13.
6
“A Guide to Measuring Advocacy and Policy,” 11.
7
“A Guide to Measuring Advocacy and Policy,” 15-16.
8
“A Guide to Measuring Advocacy and Policy,” 16.

8. 6
In addition to answering these questions about future action, you can also place your current
programs and policy advocacy campaigns within a theory of change framework to better
understand the linkages between activities and outcomes.
In order to work within this framework, you must refine your thinking on how Chintan creates
and manages its projects and advocacy strategies. Instead of an ad-hoc approach to your work,
focusing on responding to events and new information, Chintan can begin to be more strategic
with its activities and aim to shape policy. Rather than focusing on questions of ‘what are we
doing’, the organization should begin to ask ‘what have we and can we achieve’. In this model,
you will gain a clear sense of the purpose and expected results of your programs or policy
strategies. Then you can modify activities and approaches where necessary to ensure desired
results.9
A theory of change allows you to view your programs and advocacy efforts from a high level
and reveals the causal relationships that exist among your activities. It is a management tool,
steering programs and efforts towards the achievement of Chintan’s central mission.10
Once
established, Chintan can delve more deeply into its many components, of which monitoring and
evaluation plays a crucial role. Exhibits One and Two in the Appendix provide an example of
how Chintan may model a theory of change.
9
“Developing Monitoring and Evaluation Framework for Budget Work Projects: A How-To Guide For Practitioners,” 20-21.
10
“Developing Monitoring and Evaluation Framework for Budget Work Projects: A How-To Guide For Practitioners,” 28.

9. 7
Components of the Monitoring and Evaluation Framework:
Within the theory of change, three central elements are present that connect your programmatic
and advocacy efforts to the mission Chintan is attempting to achieve: 1. Goals, 2. Interim
Outcomes, and 3. Activities.
1. Goals
It is important to develop a clear goal for a project or advocacy campaign. This identified main
objective can be accomplished through a set of policy changes that often take years of work and
strategic planning. When discussing goals for a program and policy advocacy strategy, particular
attention should be paid to how they fit with the overall mission of Chintan and how they
contribute toward it. For example, Chintan’s policy advocacy on e-waste management directly
relates to its mission of achieving a sustainable municipal waste system that enhances the
livelihoods of those working in the sector.
Goals should be developed both at the programmatic as well as the advocacy strategy level to
guide Chintan’s decisions.11
It can be useful to define goals as the inverse of the problem that
being addressed. For example, if the core problem is ‘waste segregation is not occurring in
residential households’, the goal could be ‘increasing the percentage of residential households
segregating waste’. The following five attributes are often presented as hallmarks of a well-
defined goal:12
 Specific
 Measureable
 Attainable
 Realistic
 Time-bound
Once they are clearly defined, Chintan can begin to unpack each goal to reveal the underlying
components that are necessary for their achievement.
Below is a sample of how Chintan could have used the strategic framework in its E-Waste
advocacy campaign.
Expected Results
Which indicators
will be used to
measure success?
What is the
current status of
the indicator?
How far do you
want the indicator
to move?
How will
indicator data be
collected and who
is responsible?
Goal: New
National Policy on
E-Waste
Management
Revision of draft E-
Waste rules
The government has
drafted E-Waste
regulations that do
not address the
informal sector.
The revision of a
National Policy on
E-Waste that
includes the
informal sector
Policy Tracking of
the issue by the
Advocacy Staff.
11
“A Guide to Measuring Advocacy and Policy,” 17.
12
“Developing Monitoring and Evaluation Framework for Budget Work Projects: A How-To Guide For Practitioners,” 13.

10. 8
One possible goal for the E-Waste campaign may have been the development of a new national
policy on E-Waste management. The campaign could be considered a success if the draft E-
Waste regulations that were issued by the government were revised. The current status of the
policy can refer to recent activity surrounding the issue area. The next box details the extent to
which you want the current status to shift, in this case the revision of a draft policy. The final box
ensures that someone is assigned to collect the necessary information.
2. Interim Outcomes
Interim outcomes act as the bridge between achieving goals and the activities undertaken to
contribute to their achievement. They are important to illustrate progress towards goals, which
may be long-term or difficult to achieve. They are measured through changes that happen within
a specific target audience and are often the direct results of activities and tactics taken to move
toward the policy goal and often must be achieved in order to reach that goal. Measuring
interim outcomes will allows you to:13
 Examine if there is evidence of progress towards your ultimate policy goal.
 Assess which milestones you are reaching and which you are not, allowing you to adjust
your activities accordingly.
 Determine which efforts are working best and how to better allocate resources to those
that have the most impact.
 Examine if the interim outcomes you are achieving demonstrate success, even if the
policy goal has yet to be achieved.
In advocating for policy changes it is particularly important to identify and track progress
towards the achievement of interim outcomes. Even with policy change as your end goal, other
more immediate outcomes can be just as important for tracking Chintan’s overall organizational
strength, its impact on societal conditions, and can help build momentum towards policy change.
Moreover, by assessing a range of interim outcomes through monitoring and evaluation,
Chintan’s efforts can be viewed in light of their contribution to multiple impact categories even
if the policy goal is not achieved.14
Framing interim outcomes as the necessary conditions to create a policy or behavioral change
may be more conducive to discussing what to track in a given effort. Organizational Research
Services distilled six categories of outcomes, five of which we present as a strong basis for you
to begin to think about and identify interim outcomes:15
13
“Advocacy Evaluation Mini-Toolkit: Tips and Tools for Busy Organizations,” 5.
14
“Advocacy and Policy Change Evaluation: A Brief Overview,” 2.
15
“A Guide to Measuring Advocacy and Policy,” Harvard Research Project, 22.

11. 9
Tracking progress towards interim outcomes can be both a formal and informal exercise in data
gathering. Formal measures of tracking progress in interim outcomes include focus groups,
polling, media tracking, surveys, and interviews. Informal methods of measuring progress
towards interim outcomes can also be very useful in providing information on the impact of your
effort. Integrating data collection into organizational routines is a simplified and introspective
way to assess progress. This approach may include:16
 Discussion of progress during program and staff meetings,
 Written record of impressions after engaging in activities,
 Reflection sessions on progress.
.
Chintan currently only engages in a basic level of interim outcomes tracking and evaluation, and
solely does so at the programmatic level. The types of information already evaluated that fall
under the category of interim outcomes include:
 The number of children “mainstreamed” into government schools whom no longer work
as wastepickers.
 Improvement in numeracy and literacy levels in children attending Learning Centers
over time.
 Public awareness and satisfaction with Chintan’s work in coordinating household waste
collection from waste-pickers.
16
A full list of indicators can be viewed in Figure 3 in the Appendix.
Example Interim Outcome Categories
1. Shift in Social Norms: How Chintan’s advocacy and policy goals are aligned with
social values and behaviors in society.
2. Strengthened Organizational Capacity: How well Chintan’s organizational structures,
staffing, leadership, and strategic planning are aiding in its efforts.
3. Strengthened Alliances: If Chintan’s alliances and partners are aiding momentum
towards its goals through coordination, collaboration, and mission alignment.
4. Strengthened Base of Support: The level of support for policy changes across the
general public, interest groups, opinion leaders, and politicians.
5. Improved Policies: Chintan’s ability to articulate evidenced based policies and ability
to help policies progress through the policy process (policy formally proposed, levels
of support, adoption, etc.).

12. 10
Chintan should similarly begin establishing interim outcomes related to its advocacy campaigns
to assess its efforts and impact. Furthermore, it must address these topics on a regular basis,
incorporating the information gathered from monitoring and tracking into its decision-making
process. Expanding the organization’s focus to include metrics on its advocacy work will be
instrumental in tying Chintan’s programmatic activities to the advocacy work it engages in while
in pursuit of its mission.
Below is an example of how Chintan could have used the strategic framework to identify interim
outcomes in its E-Waste campaign:
Expected Results
Which indicators
will be used to
measure success?
What is the
current status of
the indicator?
How far do you
want the indicator
to move?
How will
indicator data be
collected and who
is responsible?
Interim Outcome:
Increased Public
Awareness of E-
Waste Recycling
Positive responses
among survey
respondents.
Unknown. Need to
collect baseline
data.
60% of target
respondents convey
information related
to Chintan’s E-
Waste messaging.
Survey of target
audience by
Advocacy Staff and
volunteers.
Interim Outcome:
Increased Media
Coverage of E-
Waste Regulation
Number of stories
on E-Waste that
contain Chintan’s
messaging on E-
Waste management.
To be obtained
through counts of
media coverage on
the topic from 3 top
media outlets over
previous year.
50% increase of
media coverage of
Chintan’s desired
policies over the
next year.
Media tracking by
Advocacy Staff and
volunteers.
Interim Outcome:
Build political will
for a revision of
draft E-Waste
rules
Number of elected
officials who
publicly support the
inclusion of the
informal sector in
revised E-Waste
regulations.
Unknown. To be
obtained through
preliminary
conversations with
politicians.
5 political
supporters
throughout the
government.
Tracking of
interactions with
and actions taken by
politicians by
Advocacy Staff.
Three interim outcomes would be important to track to evaluate progress towards the goal of a
new national policy on E-Waste management. Several indicators were identified during a
brainstorming session, which were then narrowed down to one key indicator for each, which
could be used to evaluate its progress. At the beginning of a new campaign, the status of an
indicator may be unknown, particularly if it is a new issue for Chintan or the campaign’s target
audience. Chintan must then rely on its own institutional knowledge and past campaign
experience to set realistic expectations for how far they believe it is possible to move the metric
and make progress on their interim outcomes. Once again, it is important to identify who will be
responsible for tracking the interim outcome, and where the data will come from.

13. 11
3. Activities
Activities consist of what is actually being done to achieve both interim outcomes and goals.
Inputs, such as the target of your advocacy effort, the financial resources available, and staff,
shape the direction of these activities. Outputs, or measures of effort, are the results of activities
and quantify what or how much is produced from a given activity.17
This is the very bottom level
of the theory of change and therefore is critical information to collect and analyze. It is at this
level in which Monitoring mainly takes place, where the work that Chintan does is constantly
reviewed to ensure it is being carried out according to plan.18
Activity tracking can take on a variety of forms and is highly specific to the program or
advocacy effort Chintan is interested in monitoring and evaluating. A complete list of possible
activities to track can be found in Exhibit 3 in the Appendix, but some of these include tracking:
 Public information distribution
 Policymaker education
 Network building
Due to organizational constrains such as time and resource availability, it is crucial that Chintan
is able to identify and prioritize the outcomes that are most meaningful to revealing progress
towards achieving your interim outcomes and policy goals. In helping to guide your thinking in
this respect, we offer the following questions you can consider as you prioritize the resources
allocated to monitoring and evaluating a given program or advocacy campaign:19
17
“Monitoring and Evaluating Advocacy: A Companion to the Advocacy Toolkit,” 13.
18
“Developing Monitoring and Evaluation Framework for Budget Work Projects: A How-To Guide For Practitioners,” 57.
19
Adapted from “Monitoring and Evaluating Advocacy: A Companion to the Advocacy Toolkit,” (19-20) and “Advocacy
Evaluation Mini-Toolkit: Tips and Tools for Busy Organizations,” (1-2).

14. 12
Chintan already has a simple, yet robust, system for tracking many of its programmatic activities
and their outputs. The organization delegates the collection of its data to Field Officers who
input information to pre-formatted excel sheets. This system conforms to accepted best practices,
as those doing the monitoring are individuals who are directly involved in their implementation
and are in the best position to gauge whether they are on track.20
The tracking sheets are
submitted to a central location where they are used to inform quarterly donor reports. Currently,
this information is available for output metrics such as:
 The amount, in kg/day, of different wastes and recyclables collected by
wastepickers on a given contract.
 The number of children attending its Learning Centers.
 The number of trainings and workshops held.
Nearly all of the metrics that Chintan tracks lie on the programmatic side of their efforts, and
relate to outputs (activity tracking) without connection to any interim outcomes or policy goals.
Furthermore, there is a lack of monitoring their advocacy activities. By expanding the
foundation already in place, Chintan will be able to more effectively use the information it
collects, and have a broader insight into assessing its progress towards its interim outcomes
their effect on moving policy.
20
“Developing Monitoring and Evaluation Framework for Budget Work Projects: A How-To Guide For
Practitioners,” 57.
Questions to Identify Activities to Monitor
1. What will users of the monitoring and evaluation want to know? Are some metrics
more important than others? Is there a new strategy that you are particularly curious
about whether it is effective?
2. Does Chintan have a unique contribution to the interim outcome or policy goal? Due
to the prevalence of external and contextual factors in long-term policy advocacy
strategies, it may be more appropriate to focus on outcomes that are more directly
connected with the advocacy effort.
3. Who will conduct the monitoring? It is important to consider who within the
organization, or who external to Chintan, will conduct the evaluation. This will be
determined by both the resources available, and the interim outcomes and policy
strategies you are pursuing, as some are better suited to informal tracking and others
to the expertise and perspective of outside evaluators.
4. What will we do with the information once we collect it? Does it have a clear
connection to informing progress towards interim outcomes or goals? Will it serve
another purpose?

15. 13
Below is an example of how Chintan could have used the strategic framework to identify
activities that would contribute to progress in your interim outcomes in its E-Waste campaign:
What must
be done to
achieve
interim
outcomes?
What interim
outcome will
success on the
activity
contribute to?
Which indicators will
monitor progress?
Where is
the
output
now?
How far do you
want to move the
output?
How will
data be
collected
and who is
responsible?
Activity:
Public
Information
Campaign
 Increased
Public
Awareness
of E-Waste
Recycling.
 Increased
Media
Coverage of
E-Waste
Regulations.
 Number of
Facebook posts.
 Number of houses
face-to-face contact
and information
distributed at.
 Number of Earned
Media mentions of
Chintan’s desired
E-waste policy.
Started at
zero. No
baseline
informatio
n.
 3 Facebook
posts per week
on E-Waste.
 30% of
households
contacted given
information.
 3 earned media
mentions per
month.
The
Advocacy
Staff, with
the help of
interns, will
monitor and
track the
public
information
campaign.
Activity:
Policymaker
Education
Build political
will for a
revision of draft
E-Waste rules.
 Number of face-to-
face meetings with
politicians
discussing E-Waste
Regulations.
 Distribution of
report on E-Waste
and the informal
sector.
Started at
zero.
Campaign
has not
launched.
 Contact 10
politicians and
secure face-to-
face meetings
with 7.
 Distribute
report to
politicians
partner and
organizations;
press release on
website and
social media
accounts.
The
Advocacy
Staff will
identify
politicians
and track
interactions.
Activity:
Network
Building
 Build
political
will for a
revision of
draft E-
Waste rules.
 Increased
Public
Awareness
of E-Waste
Recycling.
 Number of
organizations
identified and
contacted.
 Types of
constituencies
represented.
 Number of
meetings among
members.
Currently
we have
one
partner,
Safai
Sena, on
our E-
Waste
campaign.
 Identify and
contact 8
organizations
working in this
issue area.
 Meet in-person
with 5
organizations.
Interns and
volunteers
will identify
possible
organizations,
the Advocacy
Staff will
track contact
with
organizations.
Three activity categories were identified as the most important to monitor while assessing their
achievement of interim outcomes. At the beginning of a new campaign, many of these indicators
may not have any baseline information. Once again, Chintan should rely on its own experiences
with previous campaigns, and seek out best practices from other organizations, to create
achievable targets in each indicator area. As for the other two components, it is essential to
identify who will collect this information and where it will come from.

16. 14
Feedback Loops
All the components of the system interact in an iterative process, constantly informing decisions.
Measuring digital outreach and social media activities, and their contribution to interim
outcomes such as issue visibility, can play a direct role in shaping the advocacy strategy that
Chintan pursues for a particular policy goal. The feedback mechanism works in the opposite
direction as well, with information gathered throughout the evaluation of advocacy efforts used
to guide the programmatic work of Chintan. Ultimately, the information exchange should occur
seamlessly and provide invaluable information and context to how Chintan is approaching and
achieving its mission.

17. 15
The Framework in Action
This strategic advocacy strategy will ultimately be tailored by Chintan to fit its needs. However,
the framework will follow the same process no matter how it is adapted by Chintan. It is
necessary to build the framework within the theory of change in order to maximize its use in
tying together the many activities Chintan engages in to its ultimate mission. Once established,
Chintan can delve into the three component parts and begin creating an exhaustive list of
possibilities for each for each in relation to its programs and policy advocacy campaigns. The
following matrix provides a complete view of the strategic advocacy plan for the E-Waste
campaign, which has been presented throughout this section:21
Strategic Results
Results Indicators Baseline Targets Means of
Verification
Expected Results
Which indicators
will be used to
measure success?
What is the
current status of
the indicator?
How far do you
want the
indicator to
move?
How will
indicator data be
collected and
who is
responsible?
Goal: New
National Policy on
E-Waste
Management
Revision of draft E-
Waste rules.
The government has
drafted E-Waste
regulations that do
not address the
informal sector.
The revision of a
National Policy on
E-Waste that
includes the
informal sector.
Policy Tracking of
the issue by the
Advocacy Staff.
Interim Outcome:
Increased Public
Awareness of E-
Waste Recycling
Positive responses
among survey
respondents.
Unknown. Need to
collect baseline
data.
60% of target
respondents convey
information related
to Chintan’s E-
Waste messaging.
Survey of target
audience by
Advocacy Staff and
volunteers.
Interim Outcome:
Increased Media
Coverage of E-
Waste Regulation
Number of stories
on E-Waste that
contain Chintan’s
messaging on E-
Waste management.
To be obtained
through counts of
media coverage on
the topic from 3 top
media outlets over
previous year.
50% increase of
media coverage of
Chintan’s desired
policies over the
next year.
Media tracking by
Advocacy Staff and
volunteers.
Interim Outcome:
Build political will
for a revision of
draft E-Waste
rules
Number of elected
officials who
publicly support the
inclusion of the
informal sector in
revised E-Waste
regulations.
Unknown. To be
obtained through
preliminary
conversations with
politicians.
5 political
supporters
throughout the
government.
Tracking of
interactions with
and actions taken by
politicians by
Advocacy Staff.
21
Adapted from “Monitoring and Evaluating Advocacy: A Companion to the Advocacy Toolkit.” and “Developing
Monitoring and Evaluation Framework for Budget Work Projects: A How-To Guide For Practitioners.”

18. 16
Course of Action
Results Indicators Baseline Targets Means of
Verification
What must
be done to
achieve
interim
outcomes
What interim
outcome will
success on the
activity
contribute to?
Which
indicators will
monitor
progress?
Where is the
output now?
How far do you
want to move
the output?
How will
data be
collected and
who is
responsible?
Activity:
Public
Information
Campaign
 Increased
Public
Awareness
of E-Waste
Recycling.
 Increased
Media
Coverage of
E-Waste
Regulations.
 Number of
Facebook
posts.
Number of
houses face-
to-face
contact and
information
distributed at.
 Number of
Earned Media
mentions of
Chintan’s
desired E-
waste policy
Started at zero.
No baseline
information
 3 Facebook
posts per
week on E-
Waste.
 30% of
households
contacted
given
information.
 3 earned
media
mentions per
month.
The Advocacy
Staff, with the
help of interns,
will monitor
and track the
public
information
campaign.
Activity:
Policymaker
Education
 Build
political
will for a
revision of
draft E-
Waste rules.
 Number of
face-to-face
meetings with
politicians
discussing E-
Waste
Regulations.
 Distribution
of report on
E-Waste and
the informal
sector.
Started at zero.
Campaign has
not launched
 Contact 10
politicians
and secure
face-to-face
meetings with
7.
 Distribute
report to
politicians
partner and
organizations;
press release
on website
and social
media
accounts.
The Advocacy
Staff will
identify
politicians and
track
interactions.
Activity:
Network
Building
 Build
political
will for a
revision of
draft E-
Waste rules.
 Increased
Public
Awareness
of E-Waste
Recycling.
 Number of
organizations
identified and
contacted.
 Types of
constituencies
represented.
 Number of
meetings
among
members.
Currently we
have one
partner, Safai
Sena, on our
E-Waste
campaign.
 Identify and
contact 8
organizations
working in
this issue
area.
 Meet in-
person with 5
organizations.
Interns and
volunteers will
identify
possible
organizations,
the Advocacy
Staff will track
contact with
organizations.

19. 17
Five steps can be applied to each component of the framework to build them out and tie them
together:22
Chintan works in a burgeoning issue area with many actors influencing the policy arena. As
such, it is particularly susceptible to external influences and events shaping its work. Consistent
review of programs, activities, and interim outcomes will help Chintan ensure its efforts are
responding to the external environment and having their intended impact.
The process of developing a strategic monitoring and evaluation framework for your programs or
your advocacy efforts should be a collaborative endeavor. Ideally, these discussions should take
place on a regular basis, checking in on progress and including new programs, advocacy
strategies, and campaigns into your current framework.
22
Adapted from “Monitoring and Evaluating Advocacy: A Companion to the Advocacy Toolkit.” and “Developing Monitoring
and Evaluation Framework for Budget Work Projects: A How-To Guide For Practitioners.”
Five Steps to Filling in the Framework
1. Results: A description of the component (goal, interim outcome, or activity). These
should already be defined while developing a theory of change.
2. Indicators: How the result will be measured and what success looks like.
3. Baseline: Where the indicator or output currently is at.
4. Targets: How far Chintan would like to see the indicator or output move from the
baseline state (what is success for the factor?).
5. Means of Verification: How the data will be collected and who is responsible.

20. 18
Review
After Chintan develops a theory of change, formalizes its components, and begins the monitoring
and evaluation process, it must ensure the knowledge gleaned from the exercise is used and
informs its decision-making process. While the proposed framework aims to simplify as much as
possible the elements involved in this strategic thinking process, it inevitably requires time and
resources. Using a framework helps focus your attention on the key aspects of your programs
and advocacy efforts that are most relevant to achieving interim outcomes and organization-wide
goals. This review can occur in the following stages: 23
1. Informally at monthly meetings among program managers. Chintan can build on its
current discussions by incorporating more explicit language on its activities and progress
towards interim outcomes, drawing observations and data from its monitoring and
evaluation activities.
2. Formally at quarterly progress meetings. It may be useful for Chintan to devote an entire
meeting, once per quarter, to discuss its progress and strategies that may or may not be
working.
3. Annual Review Session. An annual review session can be very useful for Chintan to take
a step back and view its activities and efforts from a broad level. In this space, the
organization can revisit its theory of change and examine how its components have
changed throughout the year. It may be necessary to update interim outcomes, revise
activities, and update it with new programs and advocacy efforts. This review will help
Chintan ensure its approaches and efforts are effective and efficient in reaching its overall
mission.
23
“Developing Monitoring and Evaluation Framework for Budget Work Projects: A How-To Guide For Practitioners,” 73-75.

21. 19
Recommendations:
Chintan has many of the necessary elements of a monitoring and evaluation programs to
implement a strategic advocacy framework. However, it has not engaged in efforts specific to its
advocacy campaigns. We present the following recommendations for your consideration as you
work to develop a formal strategic advocacy framework:
1. Place Your Advocacy Work in a Larger Context
Chintan should work to create a theory of change diagram that models how its various
activities, interim outcomes, goals, and mission interact. This exercise will be
instrumental in helping you begin the process of creating a strategic advocacy framework
and incorporating monitoring and evaluation techniques to track your progress.
2. Develop Interim Outcomes
Chintan does not engage in any formal development of interim outcomes. This
component of the strategic advocacy framework is crucial to tying your activities to your
goals, and ultimately to Chintan’s mission. We recommend that Chintan begin to
formally establish interim outcome markers for its advocacy goals.
3. Become a Learning Organization
Even if fully implemented, the strategic advocacy framework that we propose will end up
useless without continuous review by Chintan. Information and data that you will collect
throughout the process should be reviewed and discussed at regular progress meetings,
and informally among staff. Working to develop a culture of self-learning will engrain
the belief in the value of this information and the necessity of its incorporation in the
decision-making process.
Actions:
To implement the recommendations, we suggest Chintan take the following actions:
 Immediate Actions:
o Each program should meet to discuss how the framework could be applied to their
program.
o Program Managers meet to discuss the results of their meetings with each other.
 Medium-Term Actions:
o A small number of Interim Outcomes are created on a trial basis in each program
and are monitored and tracked internally.
o Chintan begins holding a quarterly meeting where the results of the Interim
Outcome tracking is discussed. Challenges and successes are highlighted and
areas of cross-organization collaboration are identified.

22. 20
Section Two: Social Impact Measurement
Introduction
This section provides a brief introduction on how to undertake a social impact assessment of
Chintan’s ongoing program and advocacy campaign and gives detailed instruction on data
collection methods. These methods work within the strategic advocacy framework to develop an
information baseline, allowing you to perform comprehensive and in-depth social impact
evaluation in the future. We also provide detailed data collection samples based on Chintan’s
current programs to show how these methods are operated and applied in actual research. We do
caution that a thorough impact evaluation is a time-consuming and resource-intensive process.
The report outlines both qualitative and quantitative data collection and analysis techniques that
can be useful in measuring the impact of your programmatic work. However, even in their
simplified form, they can be both time and resource intensive. Moreover they rely on baseline
data collection that does not yet exist. Once Chintan develops a sound baseline and
longitudinal data through robust monitoring and evaluation, you may be able to engage in
more in-depth program impact measurement activities.
What is Social Impact?
Social impact is the consequence of certain public or private actions on particular individuals,
groups or overall society. Social impact measurement tries to identify and materialize the change
brought about by public actions, such as programs, projects or certain policy implementation.
Based on Chintan’s mission and current programs, we have identified five major social impact
categories associated with Chintan’s objectivity and ongoing programs:
1. Cultural impacts: The impact on social norms, customs, awareness and public opinion.
Example: Measuring the change of people’s knowledge of the deleterious effect of E-
waste on individual health and environment.
2. Socio-economic impacts: The impact on socio-economic indicators, such as individual
income, employment, tax and investment.
Example: Measuring the improvement of the income levels of wastepickers after
formalizing their job.
3. Community impact: The impact on infrastructure, services, volunteer organizations and
activity networks.
Example: Measuring the increased number of sorting trash bins in different communities.
4. Health impact: The impact on mental, physical and social well-being.
Example: Measuring improved health conditions of children within wastepicker
communities.
5. Life style impact: The impact on people’s living habits, behavior and interaction with
other people.

23. 21
Example: Changes in people’s purchase and use of electronic products that contain less
hazardous components.24
Why Measure Social Impact?
As Chintan develops and expands as an organization, measuring social impact will play a crucial
role in evaluating interim outcomes that link Chintan’s program activities with their ultimate
mission. It tracks the direct implementation impacts from important programs on social
population groups to show the incremental progress toward the overall change. Identifying,
measuring and communicating social impact enables Chintan to track their achievements and
make improvements or adjustments on current strategies and programs.
In addition, this measurement will allow you to:
 Identify and set agendas based on social issues associated with the program.
 Enhance sustainable, positive effects on beneficiaries (citizens, communities).
 Build up a trust relationship between Chintan, the municipal government and community
residents that is beneficial for future program implementation.25
Social impact Identification
Before conducting any data collection and analysis work, it is always important to identify what
social change Chintan primarily wants to enable. This could be long-term change due to a
specific policy adoption or an intermediate outcome through certain activities. We recommend
Chintan identify the key impact based on the scope of program and policy as the first step to
establishing an effective assessment design. This can most easily be identified through
exercising the theory of change model. While thinking about your potential impact, it is useful
to ask questions such as these:26
 What is the ultimate impact and intermediate outcomes that you would like to see for
citizens, government officials and city environment?
 How large is the scope of your influence based on organizational scale and capacity?
 Are there any potential unexpected outcomes of our programs, either positive or negative,
you should consider?
Tracing Impact
There are many confounding factors that may also lead to an observed outcome after policy
implementation. In order to establish a more convincing causal relationship between program
implementation and the social outcome, you could establish a counter-factual: a comparison
group that is very similar to the observed people or community, to test what would happen if
24
Adapted from “A Comprehensive Guide for Social Impact Assessment,” 5.
25
“A Comprehensive Guide for Social Impact Assessment,” 8.
26
“Social Impact Assessment,” 1.

24. 22
beneficiaries had not experienced the program or policy. Due to time and resources constraints,
we do not recommend Chintan adopt this method. However, it is still important for you to
“scope” your impact, trying to identify the most obvious impact that could be directly associated
with the implementation of program.
The diagram below displays an impact value chain that allows you to better understand the
relationship between program activities, impact and the expected goals and where the role of
social impact evaluation fits in. 27
To review, Inputs are the resources necessary for the operation of program, such as money,
human capital and time. These are directed towards the major activity conducted in the program,
such as trainings, meetings, and grassroots events. Outputs are the measure results from the
organization’s activities, e.g. the weight of waste that has been collected, the number of students
that have been admitted to learning center, the number people that signed the pledge etc.
Outcome are the actual change that occurred in the social system. In a formal program
evaluation, the overall change compared to the changes in a counter-factual group determines
how large the social impact is. Finally, goal alignment involves the evaluation of the difference
between the expected outcomes and what impact has actually been achieved. All together, these
measures provide insight into the refinements of future program implementation.
27
“Double bottom line project report: assessing social impact in double bottom line ventures.”
Input Activity Output Outcome
Goal
Alignment

25. 23
Introduction to Methodology
Data Collection Methods
There are both quantitative and qualitative methodologies that could be applied to meet different
measurement needs. Quantitative methodologies, such as systematic surveys, are research
methods that involve numerical tabulations and statistical comparison. 28
Qualitative
methodologies aim to understand, report and evaluate the meaning of events for people in
particular situations. They focus on participants’ own interpretations of their experience and
attitudes and construction of reality. Most frequently used methods include key informant
interviews, focus group, case studies and participant observation.
There are several useful data collection approaches that can be easily operated by Chintan
independently, including: (1) longitudinal surveys, (2) key informant interviews, and (3)
focus group discussions. These methods will help you establish baseline data for future
comprehensive and in-depth impact evaluation give you an overall understanding about to what
extent the social change has been made by Chintan’s program or advocacy campaign.
Selecting Impact Variables
The first step to begin an impact assessment is to identify the domain of impact and select
variables to be measured. Impact variables are the measures of hypothesized outcomes and
impact that could be obtained either by qualitative or quantitative methodologies. Table 1
provides an example of a framework for researching impact in the E-waste campaign. The value
chain of impact is elaborated in stakeholders, domains of impact, an associated series of impact
variables and the appropriate data collection methods for measurement.
28
“Collecting and Using Data for Impact Assessment,” 4.

26. 24
Table 1. Framework for Researching Impact of E-waste Campaign29
Stakeholders Domains of Impact Impact Variables Source of Information
Community resident
Increased awareness of
E-waste recycling
Increased knowledge of
E-waste category
Increased knowledge of
E-waste deleterious effect
of environment and
individual
Survey
Case Study
Improved health
condition
Decreased health issues
of residents
Survey
Behavior change Donate electronic for
reuse
Consumption of
electronic products with
less toxic constitutes
Survey
Key informant Interviews
Case Study
E-waste pickers
Enhanced skill of
handling E-waste
Increased knowledge of
E-waste category
Use of scientific methods
to collect E-waste
Focus Group Discussion
Survey
Municipal officials
Enhanced political will of
E-waste management
Increased knowledge
about E-waste
management
Increased expression of
support for building E-
waste recycling sites
Key informant interview
Focus group discussion
29
Adapted from “Collecting and Using Data for Impact Assessment,” 3.

27. 25
Frameworks
1. Longitudinal Survey
A longitudinal survey is a primary quantitative method that tracks the development and changes
of the targeted group due to specific interventions and generates primary data for analysis.
Longitudinal surveys enable you to track changes of impact overtime and make a stronger case
for impact on certain populations.30
Longitudinal surveys usually involve one baseline survey and several follow-up surveys that
relate the change back to the baseline at earlier stage of program implementation. The time-
period can span for several years and the interval of surveys can either be six months, one year or
two years to allow the impact to be reflected in the sample group. A long time horizon survey
caters exactly to Chintan’s evaluation needs and is the recommended data collection tool.
Advantage and limitations
Longitudinal surveys allow you to keep record of the development and change happening over
time and thus the more effectively collect strong evidence of the impact of program. However, it
is usually very costly because of the requirement of a large sample size for representativeness
and a long time commitment. It is also difficult to maintain the same people in the study that
were interviewed for the baseline.
Survey questions
The design of survey questions plays a determinant role in obtaining sufficient and high quality
data. The questions should be clear, understandable and elicit a meaningful response. A pilot
test survey is helpful in evaluating the value of the questions. Closed-ended questions allow for
easier data analysis by using coded responses. However, open-ended questions collect larger
diversity in responses.31
Survey Design
The survey questionnaire must be rigorously structured. It should have a strong logic
organization and coherent section arrangements. We recommend you use shorter surveys in
length rather than longer surveys to solicit more meaningful responses.32
A survey should not
take more than 30 minutes to finish to ensure respondents give useful responses while not being
burdensome.
Survey sampling
The sample for longitudinal survey would ideally be a random sample drawn from the population
that is divided between a participant group and non-participant group (frequently withholding
30
“Collecting and Using Data for Impact Assessment,” 4.
31
Adapted from “Collecting and Using Data for Impact Assessment,” 4.
32
“Collecting and Using Data for Impact Assessment,” 5.

28. 26
intervention from the non-participant group until a later date). However, it is difficult and can be
costly to engage in this type of rigorous sampling. Using a Simple Random Sample (SRS) or
Stratified Random Sample among the targeted group or beneficiaries is a simpler technique and
can still provide useful data. SRS is made of randomly selected individuals, with each having the
same probability of being selected. Stratified random sample is several SRS drawn from several
subgroups of population according to demographic characteristics (gender, ethnicity, education,
living areas), allowing differential impacts on different populations to be explored As a rule,
each sample must include 32 valid responses to generate statistically significant results.
Exhibit 4 in the Appendix provides a sample questionnaire for surveying households’ awareness
and knowledge about E-waste and behavior of disposing E-waste after Chintan conducting E-
waste drive in Delhi. This survey target is a random sample of households drawn from major
communities in Delhi. It includes both a baseline survey and follow-up surveys (with an interval
of six months after then program ends).
Analyzing Survey Results33
Analyzing survey results centers on what is the best way is to summarize and present the data
collected from the questionnaires. Here we introduce the most three common types of data that
result from longitudinal surveys and show how each of them could be analyzed in a simple and
understandable way.
1. Categorical Data: This type of data uses specific names or labels as the possible set of
answers. It is usually referred to as “nominal data”. Each piece of categorical data cannot
be ranked “higher” or “lower” than another piece, each simply represents different
categories. For example:
How do you dispose your electronic products?
o Donation
o Sold to scrap dealer
o Disposed with household wastes
o Put on the street
Categorical data is usually the easiest data to analyze because you only need to calculate
the share of responses in each category. To analyze, you can simply calculate the number
of responses in each category and divide them by the total number of responses. This is
referred to as the relative frequency in statistical analysis. The relative frequencies should
sum up to 100%. Below is an example of a (relative) frequency table for the above
question.
33
“How to Design and Analyze a Survey”

29. 27
Table Two: Relative Frequency Table with Categorical Data
Answers Responses Share
Donation 30 30/100=30%
Sold to scrap dealer 40 40/100=40%
Disposed with household wastes 16 16/100=16%
Put on the street 14 14/100=14%
Total 100 100%
2. Ordinal Data: Ordinal data can help you to ask “how much” questions. Ordinal response
type data presents answers that make sense in an order. Two examples of a series of
ordinal data may be: “Never, Rarely, Sometimes, Often, Always” and “Strongly Agree,
Agree, Neutral, Disagree, Strongly Disagree.”
When analyzing ordinal data, you should present the data in a frequency table to see the
distribution of results directly. Do not convert the responses to numbers and calculate the
average of those numbers, this could lead to misleading interpretations of the data.
Never Rarely Sometimes Often Always
3% (3) 60% (60) 5% (5) 2% (2) 30% (30)
3. Interval data: Interval data has two main characteristics: (1) the data needs to be ordered
and (2) there needs to be a meaningful range between the values. Interval data is useful
when combined with segmented data from other questions. For example, if you want to
know about people’s awareness of electronic waste from different income levels, you
could segment this question based on a previous interval-style question about their
income. You can use a contingency table to present your finding. Below is an example
about how people at different income levels dispose their electronic waste.
Income Level
(Total/Share)
Donation Sold to scrap
dealer
Disposed
with
household
wastes
Put on the
street
Total
<10000 Rs 8%(2/25) 32%(8/25) 24%(6/25) 36% (9/25) 12.5% (25)
10000-20000 Rs 37% (28/75) 43% (32/75) 12% (9/75) 8%(6/75) 37.5% (75)
20000-30000 Rs 26% (13/50) 48% (24/50) 12% (6/50) 14% (7/50) 25% (50)
30000-40000 Rs 36% (18/50) 24% (12/50) 30%(15/50) 10% (5/50) 25% (50)
Total 30.5% (61) 38% (76) 18% (36) 13.5% (27) 100% (200)
If the interval is even, you can also treat interval type of data as ordinal data and directly
display it in the frequency table.

30. 28
2. Key Informant Interviews
Key informant interviews are a qualitative data collection method that involves holding in-depth
interviews of 15 to 35 people to gain first-hand knowledge. With a loose structure, key informant
interviews aim to foster the free flow of information among interviewees by identifying a list of
key issues to be discussed. Chintan can utilize this method when you want to understand the
motivation, behaviors and perspectives of different stakeholders or when you need to further
interpret quantitative data analysis results.34
Advantage and limitations
The data collected through key informant interviews usually comes from those directly impacted
by the program or policy. Chintan can gain unexpected ideas, insights, and a new understanding
of stakeholder’s experiences with a relatively easy to use method. However, interviews can
easily generate bias if the informants are not representative of the target group or if interviewer
or interpreters bias is strong.35
Steps to conduct a key informant interview 36
1. Formulate survey questions: The first step of conducting key informant interview is to
precisely define the study by listing relevant study questions. The study questions should
be kept minimum with no more than five and should be specific; for example, “Do waste-
pickers enjoy a better livelihood after the training program?” “Does the program have a
positive impact on the health condition of the community?”
2. Prepare a short interview guide: Interviewers must be aware of what topics need to be
covered in the interviews and what main study questions need to be asked. Interviewers
should develop a short but clear interview guide listing the major issues and topics to be
covered in each interview.
34
“Conducting Key Informant Interviews,”1.
35
“Collecting and Using Data for Impact Assessment,” 1
36
Adapted from “Collecting and Using Data for Impact Assessment,” 2-4.
1. Formulate
Survey
Questions
2. Prepare an
Interview
Guide
3. Select Key
Informants
4. Conduct
Interviews
5. Analyze
Interview Data

31. 29
Table 2. An example of interview guide for “Project Armaan” 37
3. Selecting key informants: The key informants (30-35 people) should be selected based on
their specialized knowledge and perspectives of the selected topic of interest. First, the
interviewers should identify the groups and organizations from which interviewees be
drawn. It is better to cover all stakeholder groups to capture a diverse range of opinions
and interests. For example, groups with potential interviewees for the e-waste campaign
could include municipal officials, project staff, waste pickers, affected households, and
corporate representatives.
4. Conduct interviews: There are several steps within conducting the interviews that will
help secure unbiased, effective results:
a. Establish rapport with interviewees by explain the purpose of the interview,
intended uses of the results and guarantee anonymity. Additionally, you should
minimize the use of jargon and technical terms.
b. Sequence the questions with factual questions first and questions requiring
opinion and perspectives second.
c. Phrase questions to elicit more meaningful and detailed answers. Avoid simple
“yes” or “no” questions. For example, ask questions like “Can you tell me what
you know about E-waste drive?” instead of “Do you know about the E-waste
drive in this area?”
d. Use probing techniques to seek more details and elaboration through follow-up
questions.
e. Maintain a neutral attitude. Interviewers should avoid giving off a strong bias
towards one position or another.
37
“Conducting key informant interviews in developing countries,” 8.
Learning Quality at
Education Centers
• Curriculum
• Learning materials
• Attendance
Effects of the Program
• Positive changes in
children
• Evidence of changes
• Challenges and areas
for improvement
General Questions
• Long-term
sustainability of the
program?
• Suggestions for the
growth of the
program?
• Current challenges?

32. 30
f. Create as detailed records as possible to ensure accuracy. You can also use a tape
recorder or mobile phone recorder if available.
5. Data Analysis: To reduce the large amounts of information you collect into manageable
themes for review and examination, you can create a one to two page summary sheet.
This summary should include information on key informants, the reasons for their
inclusion, major observations and their implications. Abbreviations are useful as a
descriptive code to help organize the responses. These codes label data under appropriate
categories and cover main concepts, ideas and key themes such as students’ learning
outcome. A short sheet is then prepared that lists page numbers devoted to particular
items, which later become subheadings in the text. A storage system for the information,
such as an excel sheet or word document can help organize and store the final results.
Finally, tables, figures, or charts of the results are useful to present findings that can then
be communicated to policymakers and other stakeholders.
In Exhibits 5, 6, and 7 in the Appendix we provide a sample templates for conducting key
informant interviews concerning the social impact of children participating in Project Armaan
and relevant data analysis methods.

33. 31
3. Focus Group Discussions
Focus group discussions are a qualitative data collection method that gathers people together
from similar experiences and backgrounds to discuss a specific topic of interest. The group
discussion is guided by a group facilitator (moderator) who gives an introduction of the
discussion topic then helps facilitate an interactive discussion among the focus group
participants.38
Focus group discussions can be useful for Chintan to better understand
stakeholders’ attitudes, opinions and needs while learning barriers to program implementation.
Advantages and limitations39
Focus group discussions are low cost and provides almost immediate results. They can generate
rich and detailed information regarding a group’s idea, experiences and opinion about an issue
and minimizes false or extreme opinions through interactions within the group setting. However,
the flexible format makes it susceptible to facilitator’s bias and thus undermines the reliability of
the results. There is no quantitative data generated to make general conclusion of the population.
The information may be difficult to analyze and must be interpreted under group context.
Steps to conduct a focus group discussion40
1. Select the Team: A qualified team plays a key role in conducting a successful focus group
discussion. The team should be small with a facilitator to guide and record the discussion.
The facilitator should have adequate knowledge of the topic and be skillful at holding
group discussions. Backgrounds in sociology or public policy would be helpful for the
group.
2. Select the Participants: First, identify the groups and institutions that should be
represented in the focus group based on what information you hope to get out of the
discussion. Separate focus groups should be held for each group identified. Consultations
with key informants can help guide the selection of each focus group. It is also advisable
to consult several key informants to eliminate bias of personal preference. The focus
group under each category should be 7-11 people to allow for a free flow of discussion.
All participants should share a similar background and traits related to the discussion
topic to allow participants to freely express their opinions without being subject to group
sentiment. Ideally people should not know each other.
3. Develop a Discussion Guide: A clear outline with the topics and issues to be covered is
useful for facilitator to explore, ask and probe questions and makes data collection more
38
“Conducting Focus group Interviews,” 1.
39
“Conducting Focus group Interviews,” 1.
40
“Conducting Focus group Interviews,” 2-4.
1. Select the
Team
2. Select the
Participants
3. Develop a
Discussion
Guide
4. Conduct the
Interview
5. Analyze
Results

34. 32
efficient. The guide should be flexible to allow for unanticipated but relevant issues that
may be brought up. Use several carefully designed questions to steer each topic to keep
the discussion more focused. Table 3 is an example of discussion guide to assess waste-
picker’s door-to-door collection of neighborhood.
Table 3. Example of Doorstep Waste Collection Discussion Guide41
Question Key Points Notes
How do you think the waste
pickers’ doorstep waste collection
in your neighborhood?
 Waste-pickers’ professionalism
 Collecting time
 Outcome and result
 Waste-pickers’ efficiency
 Provide-user relations
 Attitudes toward waste-pickers
4. Conduct the Interview: There are several steps within conducting the interviews that will
help secure unbiased, effective results:
a. Establish rapport: The facilitator must create a thoughtful, permissive
atmosphere, provide ground rules, and set the tone of the discussion in the first
few moments of the discussion.
b. Phrase questions: Adopt questions that elicit more meaningful information rather
than impede the discussion. Open-ended questions are preferable to stimulate
detailed stories and perspectives. Facilitators can then probe participants further
by expanding the discussion through “when, where, why, how” questions.42
Table
4 has sample questions regarding attitudes towards wastepicker livelihoods:
Table 4. Example of focus group questions regarding attitudes toward waste
pickers’ livelihood
Question Notes
What do you perceive as the greatest challenge for
waste pickers to earn a consistent and safe
livelihood now?
What do you think the government’s role should be
in relation to waste pickers? (Probe: should it be
responsible for formalize their job?)
Do you think current government set priority to
waste pickers in waste management process?
c. Control the discussion: The facilitator is responsible for maintaining an even and
participatory flow of discussion among all participants. To balance the
participation, the facilitator can try to address questions towards reluctant
participants or give non-verbal hints when someone talks for an extended period
(stop taking notes, look in other direction) or politely intervene the talk and
41
“Conducting Focus group Interviews,” 2
42
Adapted from “A Handbook of Data Collection Tools: Companion to ‘A Guide to Measure Advocacy and Policy’,” 4.

35. 33
refocus the discussion.
d. Minimize group pressure: Facilitator should be cautious when a general idea has
been adopted under group pressure without any general discussion or agreement.
It is useful to probe alternative views to relieve group pressure and elicit different
ideas.
e. Record the discussion: Immediately after the discussion, the team should draw a
diagram of the seating arrangement, conduct facilitator and assistant facilitator
debriefing, label and file notes, tape and other materials. The team should also
summarize the information, the team’s impressions, and implications of the
information for the study.
5. Analysis of results: Qualitative assessment of responses should be conducted soon after
the focus group discussion to identify any potential trends, patterns, and majorly held or
most frequently aired opinion. Each transcript should be dissected while segregating
useful and non-useful parts of the discussion. Each focus group question can also be
broken out separately and accompanied with a summary statement that describes the
general response.
Combine Data Sources and Review
Strategically using and integrating relevant data sources into the monitoring and evaluation of
Chintan’s program can enrich future programs while informing the impact analysis. There are no
perfect means or sequence of collecting background, conducting surveys, and completing focus
groups or key informant interviews that must be followed. These methods should overlap and
complement each other.
Conducting qualitative analyses throughout the assessment provides rich, explanatory
information that provides evidence or refutes research and secondary data. Some of the questions
developed for interviews or focus group discussions may also be adapted into questionnaires for
surveys. Qualitative studies also allow for strategic adjustments and building upon original
surveys as circumstances change. The original survey remains valid for future comparison with
follow-up surveys or changes in hypotheses and survey designs.43
43
“Collecting and Using Data for Impact Assessment,” 9.

36. 34
Recommendations
Chintan has already implemented a data collecting system that regularly documents field work.
However, Chintan needs to expand their data collection scope and move to more rigorous
methods in order to inform their advocacy strategies and eventually measure social impact.
Obtaining first-hand knowledge on social change impacts allows you to appropriately evaluate
programs or advocacy campaigns, identifying the successful elements and make adjustments.
Here we provide the following suggestions for appropriately using these approaches to better
conduct social impact measurement.
1. Identify Key Impact Categories
Conducting longitudinal surveys are time consuming and resource intensive. Chintan
should focus on researching one or two key social impact outcomes that have a strong
causal relationship with your ultimate mission. We recommend you pay special attention
to the positive change of key beneficiaries of the program instead of conducting
comprehensive assessments on the overall environmental change at this stage. Some
considerations include; the improvement of waste pickers’ livelihoods, increased
awareness of zerowaste, increased opportunities for children of waste-pickers to attend
formal school and more.
2. Begin Baseline Quantitative Data Collection
If Chintan is going to move forward in conducting post-impact evaluation, it needs
baseline data. Much of this data will be developed through a robust monitoring and
evaluation framework, presented in the previous section. By strategically combining
quantitative and qualitative methodologies, Chintan will have a more in-depth
understanding of their social impact. Collecting qualitative data would give you more
detailed information about perspectives, emotions and feedback.
3. Identify Technical Partners
Chintan should work to set up good cooperative relations with consulting companies,
research institutes or universities to make social impact measurements more formal and
professional. With the help of outside resources, Chintan can take a step towards building
experimental designs to test the actual causal relationships between programs and social
impact.

37. i
Appendix

38. ii
Exhibit One: Theory of Change Framework
Programmatic
Activities
1
2
3
4
Interim
Outcomes
1
2
3
Policy Advocacy
Activities
1
2
3
4
Interim
Outcomes
1
2
3
Chintan's
Program
Work
Impact
Policy
Strategy
Policy Goal
Measurement
Evaluation
Measurement
Evaluation

39. iii
Exhibit Two: Theory of Change Example for the E-Waste Campaign
Programmatic
Activities
Waste-picker
Trainings
E-Waste
Collection Drives
E-Waste
Utilization
Workshops
Interim Outcomes
Increased Waste-
picker Awareness
Increased Public
Awareness
Increased E-
Waste Collection
Policy Advocacy
Activities
Public
Information
Campaign
Network
Building
Policymaker
Education
Interim Outcomes
Increased Public
Awareness of E-
Waste Recycling
Increased Media
Coverage of E-
Waste Regulation
Build political will
for a revision of
draft E-Waste
rules
Chintan's E-
Waste
Program
Work
Impact of
Activities
Advocacy
Strategy for E-
Waste
Campaign
New National
E-Waste
Policy
Measurement
Evaluation
Measurement
Evaluation

40. iv
Exhibit Three: Overview of Strategic Advocacy Framework Components44
Activities Interim Outcomes Goals
Public Information Campaign Organizational Capacity Policy Adoption
Earned Media Expansion (Newspaper,
TV, Radio)
New Partnerships Policy Change
Network Building New Political Partners
Policy
Implementation
Research and Reports
Increased Issue or Organizational
Visibility
Behavioral Change
Policymaker Education Public Will Shift
44
Adapted from “Monitoring and Evaluating Advocacy: A Companion to the Advocacy Toolkit.” and “Developing Monitoring
and Evaluation Framework for Budget Work Projects: A How-To Guide For Practitioners.”

41. v
Activities
Activity Definition Possible Indicator
Public Information
Campaign
Using traditional media,
social media, and stakeholder
engagement to expand the
audience.
 New section on the
webpage
 New facebook post
 Article on website
 Number of grassroots
events held
Earned Media Expansion
(Newspaper, TV, Radio)
Reaching out to print, radio,
or television media to gain
visibility on the issue.
 Number of outreach
attempts to reporters.
 Number of different
types of media
contacted
Network Building
Gaining more visibility and
strength by bringing together
outside groups or
organizations to partner on an
issue
 Number of coalition
groups
 Types of
constituencies
represented
 Number of meetings
held between
members
Research and Reports
Report that relates to a
particular policy area or
program undertaken by
Chintan
 Item Produced
 Number of downloads
Policymaker Education
Meet with politicians to
explain desired policy, its
expected impact, or desired
program and its expected
impact.
 Number of politicians
contacted, met with
 Commitments from
politicians
Presentations
Making a case for a policy or
project in-person to an
organization or individual
 Number of
presentations held
 Types of audience for
each (politician,
corporation, school)
 Number in attendance

42. vi
Interim Outcomes
Outcome Description Definition Possible Indicators
Organizational
Capacity
The ability of the organization
to lead, manage, and
implement programs and
strategies
 Increased knowledge
about advocacy and
organizing
 Improved media contacts
 Improved data collection
and analysis skills
New Partnerships
Mutually beneficial
relationships with
organizations and individuals
who participate in an
advocacy strategy
 Number or new
relationships
 Strength of current
relationships
 Amount of collaboration
between organizations
 Improved alignment of
partnership efforts
New Political Partners
Politicians who adopt an issue
and support it
 Number of politicians
recruited
 New geographic regions
represented
 New levels of government
represented
 Number of supporting
actions of politicians
Increased Issue or
Organizational
Visibility
Recognition of the problem,
familiarity with proposed
program, or familiarity with
the organization
 Number of media stories
 Percentage of audience
with knowledge of policy
or program (through
polling)
Public Will Shift
Willingness of public to
support an issue or take part
in a program
 Percentage of audience
willing to participate in a
program
 Attendance at an
advocacy event (training,
presentation, meeting)
Political Will Shift
Willingness of politicians to
act to support an issue or
program
 Number of officials who
support the policy or
program
 Votes for or against a
policy

43. vii
Goals
Outcome Description Definition Possible Indicators
Policy Adoption
Passage of a policy or
adoption of a program
 Policies or programs
formally adopted or
begun
Policy Change
Revision of a current policy
to address a new aspect of a
problem.
 Policy formally
amended
Policy Implementation
Proper implementation of a
policy or program.
 Level or funding or
resources allocated
 Expected outcome
experienced in
practice
Behavioral Change
Changes in behavior of target
audience in accordance with
policy or program
 Number participating
in program
 Results of policy seen
in practice

44. viii
Exhibit 4: Sample Survey
Chintan’s E-waste Assessment of New Delhi
Questionnaire for Households45
Date:____________
Location:_____________
Interviewer:_____________
Interviewed Person:
Name
Telephone
E-mail
Ward/Suburb
Town
Introduction
Chintan is a not-for-profit organization based in New Delhi, in collaboration with *****, is
collecting data on e-waste generation and management in Delhi in order to know the current e-
waste generation and management in the City. The study will enable the authorities to determine
the necessary steps required for handling e-waste. We would be grateful if you could spare some
time to answer a few questions:
45
Adapted from “E-waste Assessment Tanzania,” 65.

45. ix
Questions about Awareness and Behavior
Questions Response
2.1 Yes No
a) Do you know what e-waste or
waste of electrical and electronic
equipment is?
b) Are you aware that components
of e-waste need a special treatment
in order to be safely disposed of?
c) Do you have waste collectors in
your community?
2.2 Yes, Everything Yes, But No E-waste No
Do waste collectors come and pick-
up waste at your door? Do they
collection e-waste too?
2.3 Yes No I don’t know
Is the way e-waste is currently
collected convenient to you? How
could it be improved?
Comments
Product Tracing of Household Electronics
Products
Where was it bought
and in what
condition? (new/ used
& working/ broken)
Number of
years used
Number of
years stored
In what condition was
the product at the end
of life? (Working –W;
Broken – B; Broken
but Fixable - F )
Refrigerator
Washing Machine
Microwave
Personal Computer
Mobile phone
Laptop
TV
Radio

46. x
Usual Method of Disposal of Electronic Products (please mark with x)
Donation
Sold to
second-
hand dealer
Sold to
scrap
dealer
Disposed with
household waste
Put on the
street
Other
Fridge
Washing Machine
Microwave
Personal
Computer
Mobile phone
Laptop
TV
Radio
Number of person in the household (please tick appropriate box)
1 2 3-4 5-8 More than 8
Total Household Income per Month (Rupee) (please tick appropriate box)
Less than
10,000
10,000-30,000 30,000-50,000 50,000-70,000 70,000-100,000 More than
100,000

47. xi
Exhibit 5: Key Informant Interview Sample Questions of Assessment of Project
Armaan
1. (Questions for representative of children) How has participating this project been of benefit to you?
What specific knowledge have you learned through participating the project? What other. Are there
any extracurricular activities that you find interesting and benefit from them?
2. (Questions for representative of parents) How is your thought of sending child to informal study
center? Has children’s participation of the project brought any change to your family? What are the
factors that you hesitate to allow children participate this project? (probing economic burden, sexual
bias)
3. (Questions for teachers) How is students’ learning quality in the study center (probing improvement
of numeracy and literacy skill). What are the challenges now for improving children’s learning
outcome or mainstreaming more children into formal school (probing facility, learning materials,
children attendance etc.)
4. (Questions for project staff) How do you see the sustainability of this project in the future? (Probe
funding, government support etc.)

48. xii
Exhibit 6: Example of a Coding System for the Evaluation of Project Armaan46
General Description Code Transcripts Page Number
Learning outcome LEA-OUT
Literacy skill LEA-OUT-LIT 7,9,33,44
Numeracy skill LEA-OUT-NUM 15.54.176
Computer skill LEA-OUT-COM 28,37,104,135,170
Extracurricular knowledge LEA-OUT-EXTR 3,35,62,78
Project Effects PRO-EFF
Children life PRO-EFF-LIFE 11,26,33,55,100
Family PRO-EFF-FAM 5,33-4,45
Project Sustainability PRO-SUS
Funding PRO-SUS-FUN 17,86-7,187-3
Government Support PRO-SUS-GOV 139-41,198
Exhibit 7. Example of Data Analysis for the Key Informant Interviews47
Recommendations for Improving Mainstream Rate
Recommendation Number of Respondents
Improving parental involvement and awareness 23
Conducting teacher training programs 21
Adding other subjects to the curriculum 16
46
Adapted from “E-waste Assessment Tanzania,” 26.
47
Adapted from “E-waste Assessment Tanzania,” 29.

49. 1
References
Agency for International Development. (1989). Conducting Key Informant Interviews in
Developing Countries. Washington DC: Kumar, Krishna.
Center for Good Governance. (2006). A Comprehensive Guide for Social Impact Assessment.
Chandurkar, D. and Sen N. (2014). Developing Monitoring and Evaluation Framework
for Budget Work Projects: A How-To Guide for Practitioners.
National Foundation for India.
Chintan Environmental Research and Action Group. (2014). Annual Report: 2013-14.
New Delhi.
Chintan Environmental Research and Action Group. (March 2013). Project Armaan:
An Initiative to Help Children Stop Picking Trash and Get and Education. New Delhi.
Clark, C., Rosenzweig, W. (2004). Double Bottom Line Project Report: Assessing Social Impact
in Double Bottom Line Ventures.
“The Evaluation Exchange.” Harvard Family Research Project, Harvard Graduate School of
Education, Vol.13, Num.1, (2007).
Learning for Action. (2013). Advocacy Evaluation Mini-Toolkit: Tips and Tools for Busy
Organizations. San Francisco, CA.
Magashi, A., Schluep, M. (2011). E-Waste Assessment Tanzania.
Cleaner Production Centre of Tanzania & Empa Switzerland.
Organizational Research Services. (2007). A Guide to Measuring Advocacy and Policy.
Seattle, WA: Gienapp, A., Reisman, J., and Stachowiak, S.
Organizational Research Services. (2007). A Handbook of Data Collection Tools: Companion to
‘A Guide to Measuring Advocacy and Policy’.
Seattle, WA: Gienapp, A., Reisman, J., and Stachowiak, S.
Overseas Development Institute (2011). Background Note: A Guide to Monitoring and
Evaluating Policy Influence. London: Jones, H.
Red Ochre. (2011). Social Impact Assessment.
Accessed via:
http://www.vai.org.uk/wp-content/uploads/2011/07/110707-Social-Impact-Toolkit.pdf

50. 2
United Nations Children’s Fund. (2010). Monitoring and Evaluating Advocacy:
Companion to the Advocacy Toolkit. New York: Coffman, J.
USAID. (2006). Collecting and Using Data for Impact Assessment.
USAID. (1996). Conducting Focus Group Interviews.
USAID. (1996). Conducting Key Informant Interviews.
Zapier. The Ultimate Guide to Forms & Surveys. Peters, C., Schreiber, D., Guay, M.,
and Baedell, S.
Accessed via: https://zapier.com/learn/ultimate-guide-to-forms-and-surveys/#toc.