This document discusses evaluating CSR projects and programs. It begins by differentiating evaluation from appraisal, monitoring, and impact assessment. It then covers various types of evaluation including formative and summative, as well as tools like logical framework analysis and Bennett's hierarchy. Examples are provided of evaluating integrated contract broiler farming and e-learning materials. Challenges of evaluation like establishing controls and measuring long-term impacts are also discussed. The document emphasizes that evaluation is important for accountability, learning, and improving future CSR efforts.

1. How to Evaluate CSR Projects and
Programmes
Prof. P.V.K.Sasidhar
School of Extension and Development Studies
Indira Gandhi National Open University
New Delhi-110068 ( www.ignou.ac.in)
Teleconference Session
PGDCSR and MACSR Programmes
(11th April 2022)

2. Relevance of the Session & Outline
• To understand the meaning of evaluation &
differentiate it from appraisal, monitoring & impact
assessment.
• Discuss different types, tools, techniques, designs,
approaches & challenges of evaluation.
Topic Programmes Courses Covered
How to
Evaluate CSR
Projects and
Programmes
PGDCSR
(July 2021 & January 2022
Sessions)
&
MACSR
(January 2022 Session)
MEDS-54
MEDSP-051
No. of Learners: 600

3. CSR Programme
Objective (s)
Evaluation
Indicator(s)
Data Collection
Technique (s)
Time and Place
of Data Collection
1.
2.
3.
What will
happen as a
result of the
CSR
programme?
What will you
measure that
will provide
evidence of
change ?
• Questionnaire
• Telephone
survey
• Mail survey
• Observation
• FGD etc
• Pre & post tests
• Follow-up: 3-6 months after
programme
• Baseline data - During
planning stage
Card Exercise on ‘Good Practices in CSR
Evaluation’
Have you conducted any CSR evaluation study?
If Yes, fill in the boxes for 1 to 3 CSR programme objectives

4. • CSR professionals responsibility:
Implementing CSR Programme
Planning Cycle with:
– Efficiency
– Accountability
– Proper Resource Allocation
• CSR Departments look for
experts to appraise, monitor,
evaluate & assess the impact of
CSR interventions.
Programme
Formulation
Delivery
Evaluation
Why Research on
Appraisal – Monitoring - Evaluation - Impact Assessment ?
CSR Programme
Planning Cycle

5. Appraisal – Monitoring - Evaluation - Impact
Assessment
Appraisal - Critical examination of a CSR programme proposal, normally before
implementation & funding (economic viability, technical feasibility, social
desirability etc).
Monitoring Evaluation
Continuous: Starts and ends with a CSR
programme.
One shot operation: At a point of time
(usually at completion or mid way of CSR
programme)
Required for immediate use and mid-
course correction
Used for future planning/ replication/
expansion
Done by CSR implementing personnel Usually by outside agency
Quick but covers all units In-depth; covers a sample (with Criteria,
Indicators and Time)
Correcting / Managing Learning process
Symptomatic, early warning system Diagnostic ( Inputs, Outputs and Outcomes)
AME focus on the processes & direct results of CSR programme.
Building on these, ‘Impact Assessment’ focuses on long-term & wider-ranging changes
beyond the immediate results of the CSR programme.

6. Why Evaluation Competencies are
Essential for CSR Professionals?
To be Relevant, CSR Professionals
Need to Have / Acquire M&E
Competencies
MEAL Questions by CSR Funding Agencies
• Why should we continue to fund CSR
programmes? (ITC: 353 Crores in 2020-21)
• Are the CSR programmes effective?
• Who benefited & what impacts were
made?
• How would you improve / terminate
ineffective CSR programmes? etc

7. Value of M&E - Research Opportunities
Summary of Key
Competencies
• Integration of
evaluation in CSR
programme planning
& implementation
• Management of the
evaluation process
• Facilitation of
knowledge building &
sharing evaluation
findings

8. Evaluation of CSR Programmes
(e.g. Criteria & Indicators)
Assessing CSR Project as systematically & objectively as
possible about:
1. Best-fit approaches
2. Embracing pluralism
3. Increasing accountability to clients
4. Developing human resources
5. Ensuring sustainability
(Inputs, Outputs & Outcomes)
• Stated Criteria &
Indicators
• Time Reference
Key Evaluation Indicators - Examples
1. Gender-sensitive CSR projects, promoting women participation & ownership
(e.g. Shakti project of HUL empowered 72,000 women micro entrepreneurs)
2. Multiple service providers & collaborators for pooling resources (e.g Nand Ghars)
3. Stakeholders empowerment
4. Continuous HRD
5. CSR Programme aligning with SDGs

9. Evaluation of e-Learning Material
e.g. 1. Digisakshar - a CSR initiative of Vodafone Idea Ltd. & NASSCOM Foundation
e.g. 2. Nand Ghars – a CSR initiative of Vedanta
• Assessing e-learning material as
systematically & objectively as
possible about:
Content:
– Quality
– Effectiveness
– Ease of Use
(Inputs, Outputs & Outcomes)
• Stated Criteria &
Indicators
• Time Reference
Key Evaluation Indicators
Self explanatory, Self-contained,
Self-directed, Self-motivating &
Self-evaluating
Nand Ghars: E Learning
through 40 weeks of
scientific curriculum for
children

10. Evaluation
Formative Summative
Implementation Progress
Early Stage Later Stage
Timeline
Types of Evaluation

11. Types of Evaluation
Evaluation by focus
• Formative Evaluation: During programmes development
stages (Process evaluation - Implementation stage) /Mid
term appraisals – Progress stage)
• Summative Evaluation: Once the programme achieves a
stable state of operation (Outcome evaluation / Ex-post
evaluation)
Evaluation by agency – Internal / External
Evaluation by stage
– Ongoing : During the project
– Terminal: After completion of project immediately
– Ex-Post : After a time from completion of project
Longitudinal evaluation -repeat evaluation to study the
sustainability of results/outcomes

12. Evaluation Tools & Techniques by
Programme Stage
Programme
Stage
Evaluation
Type
Evaluation Questions Evaluation tools and
techniques
Planning Needs
assessment
Feasibility
study
Base line
study
 What are the felt and unfelt
development needs?
 Can the programme address these
needs?
 Is CSR programme feasible (socio-
economic & environmental ?)
Survey
FGD
Observation;
Content analysis
Economic analysis
B:C ratio
Implementation Formative
evaluation
 Is programme meeting its objectives
of indented outcomes?
 Are stakeholders satisfied with the
programme?
Annual monitoring
reports
KASA change
Satisfaction surveys
Conclusion Summative
evaluation
 Are the needs addressed?
 Are the desired outcomes achieved?
 What is the cost effectiveness of the
programme?
 Under what conditions? With what
training? At what cost? etc
Pre & post
programme data
Economic analysis

13. Evaluation Designs
1. "Pre - Project" Vs. "Post-Project" Evaluation
Survey on present situation and situation prior to CSR project
(suffers from 'memory or recall bias' of the respondents).
‘Baseline' or ‘Bench Mark’ study and ‘Post-Project’ study.
Unfortunately in most CSR projects, baseline studies are not
undertaken or when available it is inadequate.
2. With-Without Project
• CSR Beneficiary – CSR non beneficiary (Control Group)
Combination of both - most appropriate

14. How to Conduct Programme
Evaluation ?
1. Writing evaluation proposal:
• Need for evaluation - economic /technical / social
• Evaluation questions & objectives
• Evaluation methodology & tool
o LFA
o Bennett’s Hierarchy
o Inputs and Results
• Budget
2. Designing evaluation survey instrument - Ethics Committee
3. Coding, pre-testing, recoding & data collection
3. Data analysis
4. Evaluation report writing & communicating findings – Short / longer
reports
Inputs , Outputs and Outcomes

15. Logical Framework Approach
INPUTS OUTPUTS OUTCOMES
CSR Program
investments
Activities Participation Short Medium Long-
term
OUTCOMES What results for individuals, families, communities..… ( With
Objectively Verifiable Indicators and Means of Verification)
SHORT
Learning
Changes in
• KASA
• Opinion
• Motivation
• Behavioral intent
MEDIUM
Action
Changes in
•Behavior
•Decision-making
•Policies
•Social action
LONG-TERM
Conditions
Changes in
Conditions
Social (well-being)
Health, Economic
Civic , Environmental

16. Bennett’s Hierarchy of Evaluation
Provides a process by examining the chain
of means (what you do) and ends (the
result of your actions) through 7 levels
Example: Understanding Gap in
Development Scenario for TML - CSR
Intervention - Tribals Transform from
Forest Gatherers to Entrepreneurs
Means
• Collectivise the tribal communities to
start a social enterprise – FPO
• Capacity building and resource support
Ends
• Market linkage to products

17. Inputs & Results Evaluation
e.g. TML - CSR Intervention-Tribals Transform from Forest Gatherers to
Entrepreneurs
Output
(Immediate results
in one year):
e.g.,
• FPO sold 25 tonnes
of fruits worth Rs 50
lakhs.
• Income of Rs
20,000/- to each
family.
• FPO earned Rs 4.4
lakhs.
Inputs
• TML and Shashwat – NGO
• 400 Tribal Families -organized under 26 SHGs & an FPO.
• Training of Tribals & Revolving Fund
Effect (Intermediate
results / outcomes):
(Consequence of the
use of the output)
e.g., Strengthened
ability to sustain
through the lockdown
and Scaling upto 600
families
Impact (Long term):
(Consequence of the
effect)
Impact is outcomes for
Tribals as a whole than
on individuals.
e.g., Improving overall
livelihood security.
Results

18. Outcome Indicators
(Social Media Engagement – Nand Ghar)
KPIs to Measure the Outcomes of Knowledge
Management (Social Media Engagement)
Social Media KPIs for Reach
 Follower Count
 Impressions
 Web Traffic
o No. of Visitors (% of Total)
o Percent of New Visitors (Average /
view)
o Bounce Rate
o Pages per session
o Average session duration
o Conversion Rate (CR)
o Cost of Acquiring Stakeholders (CAS)
o Cost Per Acquisition (CPA)
 Share of Voice
Social Media KPIs for
Engagement:
 Clicks
 Likes
 Shares
 Comments
 Mentions
Rebekah Carter, 2018

19. Example 1: Integrated Contract Broiler Farming:
An Evaluation Case Study in India
Evaluation Questions
1. Do contract and non-contract farmers incur
significantly different production and marketing
costs and earn different marketing margins?
2. Does the provision of EASs by private CBF
companies enable contract farmers to make
better profits than non-contract farmers?
3. Have assured markets, competitive price and
guarantee against risk resulted in successful
value chain development through CBF?
4. Are the value chain developments and provision
of EASs by private CBF companies really win-
win situations for both integrators and farmers,
or are they socially acceptable ways of
exploiting the farmers?

20. Evaluation
hierarchy
Measurement Indicators Measurement
Level 7 : End
results
Socio-economic
changes and
impacts
 SWOT parameters
 FGD on: selection of contract farmers; terms and conditions
applicable in CBF
Open-ended
Level 6 :
Practice change
Technical advices
adoption
 Non-adoption, discontinuation, partial adoption and full
adoption of technical advices
Scale on four-
point continuum
Level 5: KASA Farmers’
perceptions
 Perceptions on inputs (chicks, feed, medicines and EAS) and
outputs (broiler birds, manure value and payment system)
Scale on five-
point continuum
Level 4 :
Reactions
Farmers’
feedback
 Factors of motivation to do CBF and NCBF
 Reasons to change integrator(s) or input providers in the past
two years
Open-ended
Level 3 :
Outputs
Technical and
economic
performance
 Broiler birds (flock size, mortality number, birds sold, sale age,
sales rate and birds lifting days)
 Productivity (mortality percentage, birds sold, feed
consumption and body weight)
 Efficiency (FCR, sale age, weight gain/day)
 Economics of inputs and outputs
 EAS (frequency of information from various sources)
Technical and
economic
performance
index
Level 2 :
Activities
Activities in
CBF and NCBF
 Physical and human resource activities in CBF and NCBF Survey
Level 1: Inputs Investments and
Demographics
 Fixed and variable costs
 Age, gender, education, social category, family and size, poultry
occupation and experience
Survey
Bennett's Hierarchy Applied in Evaluation

21. Provides a process
by examining the
chain of means
(what we do in
e-LM preparation)
and ends
(the result of e-LM
development )
through 7 levels
Level 7 (End Results) : Economic, Social, Nutritional,
Gender Empowerment etc.
Level 6 (Practice Change) : Change in Behavior, Level of
Practice Adoption, Current Competencies, On-Job
Performance etc.
Level 5 (KOSA) : Change in Awareness, Opinion,
Knowledge, Attitude, Skills etc.
Level 4 (Reactions of Students) : Content quality, Ease of
use, Self Directed Learning, Liking of Formats, Time
taken to read / listen & Reactions on Usefulness of e-LM
etc.
Level 3 (Outputs) : e-LM developed (A/V/ Online etc), No.
of learners completed etc.
Level 2 : (Activities): Prescribed activities of key
stakeholders, Operational procedures etc.
Level 1 (Inputs): Financial & Human Resources, Target
Group – Profile, Need Assessment Report etc.
Example 2: e-LM Evaluation

22. Key Evaluation Questions on Quality of
Content
Questions Low High
Is the material appropriate?
Are the learning objectives clear?
Is the content current, relevant, and
accurate?
Is the content appropriate to the needs of
the target group or delivery sector?

23. Quality of Content - Are the learning
objectives clear?
High Rating
• If learning goals were
clearly stated in eLM
• Connection existed
between the eLM and
gains learners could make
• Learners could
understand and state
what they were expected
to learn
Low Rating
• Unclear about what
learners were supposed to
be learning and why
• eLM had no reasonable
pattern or flow
Question Low High
Are the
learning
objectives
clear?
Evaluation Findings
• Learners are interested in the
subject
• Topic is relevant
• Lessons have immediate
application etc.

24. Needs
Assessment
(Specific
Programme)

25. M&E Capacity Building
Continuing
Education
Programmes

26. Challenges in Evaluation
a) Establishing a scientific knowledge base and generate evidence is a
challenge
Example: Do Tribals who attended the ‘TML-Shaswat Training
Programmes’ become better entrepreneur than those who do not
attend?
b) It is difficult to establish a control group, maintain uniform treatments
and measure long-term impacts.
Example: Do Anganwadi children who read ‘Nand Ghar’s E-Learning
Material’ will follow the practices better than those who do not?
c) Non-availability / inadequate baseline data on impact indicators to see
expanded impacts over time.
Example: Non availability / inadequate baseline data on impact
indicators.

27. Evaluation
• Neglected & mistakenly seen as a 'fault finding'
mechanism
• Learning process & past experiences guide us to a better
future
Conclusion
Questions / Comments ?
Thank you
pvksasidhar@ignou.ac.in