Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Monitoring & Evaluation Framework - Fiinovation


Published on

Monitoring and evaluation is a vital component that determines the effectiveness of a corporation's assistance by establishing clear links between past, present and future initiatives and results. The process helps in improving the programme performance and achieving desired results. It provides opportunities for fine-tuning, re-orientation and planning of the programme effectively, without which it becomes impossible to measure the success and impact of the programme even if the approach is right.

Published in: Business
  • I like this service ⇒ ⇐ from Academic Writers. I don't have enough time write it by myself.
    Are you sure you want to  Yes  No
    Your message goes here
  • I think you need a perfect and 100% unique academic essays papers have a look once this site i hope you will get valuable papers, ⇒ ⇐
    Are you sure you want to  Yes  No
    Your message goes here
  • You might get some help from ⇒ ⇐ Success and best regards!
    Are you sure you want to  Yes  No
    Your message goes here

Monitoring & Evaluation Framework - Fiinovation

  1. 1. fiinovationOpportunity Solutions, Simplified. MONITORING & EVALUATION FRAMEWORK
  2. 2. IndexMonitoring and Evaluation & it’s importance1 Monitoring and Evaluation framework in the Project Program Cycle02 Monitoring and the inclusion of RBM03 Types of Monitoring04 Tailor-Made Evaluation Studies06 Evaluation Questions & RBM08 Six key steps for M&E Planning09 Minimizing Bias and Errors10 CSR in India - Industry Overview11 Fiinovation Expertise12 Some of our Initiatives13 Organization Brief14 Awards & Recognition15
  3. 3. What Does Monitoring and Evaluation Mean? Monitoring Evaluation Why is Monitoring and Evaluation Important? Organisations use monitoring and evaluation for two key purposes: 01 Does your project have an M&E plan? Does your project have an M&E binder? 1 Have you developed all monitoring forms? 3 Are monitoring data regularly used during M&E meetings or other events? 8 Are staff and partners trained on data entry and analysis? 7 Are staff and partners trained on using the monitoring forms? 4 Have you created a monitoring database? 6 Have you conducted a quality check of the monitoring data? 5 If your answer is to any of the questions above, the project team should work to complete the‘No' step(s). This review tool has not been designed to be used in isolation and refers to the package of practices being offered at Fiinovation, and this handbook is for further guidance at each review stage. To learn about their own activities and results, and to support internal planning and development To be accountable to their stakeholders. Monitoring is a routine and systematic collection of information against a plan. The information might be about activities, products or services, users, or about outside factors affecting the organisation or project. Evaluation is about making judgements about the value of any component part of an organisation or its products, services or benets, or about the organisation as a whole. Organisations need evidence of their efciency and effectiveness for funders, commissioners and investors. They also need to communicate achievements to a wider public. Charities are now specically required to report the achievements of the charity over the year. However, the role of monitoring and evaluation is providing learning and improving the organisation is of equal importance. The following eight questions shall help in creating a better understanding on the need of monitoring and evaluation: 2 Monitoring & Evaluation
  4. 4. M&E Framework in the Project Program Cycle 02 Initial Needs Assessment Log frame and indicators M&E planning Baseline assessment Midterm evaluation and/or reviews Final evaluation Dissemination and use of lessons This is done to determine whether a project/program is needed and, if so, to inform its planning. This involves the operational design of the project/program and its objectives, indicators, means of verification and assumptions This is the practical planning for the project/program to monitor and evaluate the log frame's objectives and indicators. This is the measurement of the initial conditions (appropriate indicators) before the start of a project/program.These are important reflection events to assess and inform ongoing project/program implementation. This occurs after project/program completion to assess how well the project/program achieved its intended objectives and what difference this has made. This informs ongoing programming. Although this has been put in the last section of the diagram but, reporting, reflection and learning should occur throughout the whole project/program cycle. There is no magic 10-percent sampling rule. It is important to note that the sample size is not related to the size of the popula on being sampled. A frequent mistake is to conduct surveys among 10 percent of a given popula on; in fact, it is likely that 10 percent of the popula on is either too many or too few households. With too many households, the survey is using excessive resources and me; with too few households, the sample willnotadequatelyrepresentthepopula on. Monitoring & Evaluation
  5. 5. Monitoring and the Inclusion of RBM Impact Ÿ Measuring changes at impact level requires a longer time frame, and is therefore dealt with by evaluation and not monitoring. Outcomes Ÿ Are the outputs leading to the achievement of the outcomes? Ÿ Is there anything happening that should lead management to modify the operations implementation plan? Output Activities Ÿ Are activities leading to expected outputs? Ÿ Are activities being implemented on schedule and within the budget? Inputs Monitoring Questions and RBM 03 A mistake common for M&E systems is to rely solely on either observa on data or par cipant responses. Observa on data alone does not provide an explana on of prac ces or behaviors and o en requires large assump ons on the part of the M&E team. Focus group data (an example of par cipant responses) may not capture important prac ces that par cipants do not see as relevant and may record instead what par cipants thinkdatacollec onteamswanttohear. Result Based Management (RBM) is an approach to project/program management based on clearly dened results, and the methodologies and tools to measure and achieve them. RBM supports better performance and greater accountability by applying a clear, logical framework to plan, manage and measure an intervention with a focus on the results you want to achieve. By identifying in advance the intended results of a project/program and how we can measure their progress, we can better manage a project/program and determine whether a difference has genuinely been made for the people concerned. Monitoring and evaluation (M&E) is a critical part of RBM. It forms the basis for clear and accurate reporting on the results achieved by an intervention (projector program). In this way, information reporting is no longer a headache, but becomes an opportunity for critical analysis and organizational learning, informing decision-making and impact assessment. Ÿ How do beneciaries feel about the work/intervention? Ÿ What is causing delays or unexpected results? Ÿ Are nances, personnel and materials available on time and in the right quantities and quality? Monitoring & Evaluation
  6. 6. It tracks the use of inputs and resources, the progress of activities and the delivery of outputs. It examines how activities are delivered – the efciency in time and resources. It is often conducted in conjunction with compliance monitoring and feeds into the evaluation of impact. For example, a water and sanitation project may monitor that targeted households receive septic systems according to schedule. Types of Monitoring It tracks effects and impacts. This is where monitoring merges with evaluation to determine if the project/program is on target towards its intended results (outputs, outcomes, impact). It measures whether there may be any unintended impact (positive or negative). For example, a psychosocial project may monitor that its community activities achieve the outputs that contribute to community resilience and ability to recover from a disaster. 1. Results Monitoring 2. Process (Activity) Monitoring 3. Compliance Monitoring 4. Context (Situation) Monitoring 04 It ensures compliance with donor regulations and expected results, grant and contract requirements, local governmental regulations and laws, and ethical standards. For example, a shelter project may monitor that shelters adhere to agreed national and international safety standards in construction. It tracks the , especially as it affects identied riskssetting in which the project/program operates and assumptions, but also any unexpected considerations that may arise. It includes the eld as well as the larger political, institutional, funding, and policy context that affect the project/program. For example, a project in a conict-prone area may monitor potential ghting that could not only affect project’s success but, endanger the project’s staff and volunteers. Monitoring & Evaluation
  7. 7. It tracks beneciary perceptions of a project/program. It includes beneciary satisfaction or complaints with the project/program, including their participation, treatment, access to resources and their overall experience of change. Sometimes referred to as it often includes a stakeholderBeneciary Contact Monitoring (BCM), complaints and feedback mechanism. It should take account of different population groups, as well as the perceptions of indirect beneciaries (e.g. community members not directly receiving a good or service). For example, a cash-for work program assisting community members after a natural disaster may monitor how they feel about the selection of program participants, the payment of participants and the contribution the program is making to the community (e.g. are these equitable?). 6. Financial Monitoring 05 5. Beneficiary Monitoring It tracks the sustainability, institutional development and capacity building in the project/program and with its partners. It is often done in conjunction with the monitoring processes of the larger, implementing organization. For example, a National Society's headquarters may use organizational monitoring to track communication and collaboration in project implementation among its branches and chapters. 7. Organizational Monitoring It accounts for costs by input and activity within predened categories of expenditure. It is often conducted in conjunction with compliance and process monitoring. For example, a livelihood project implementing a series of micro-enterprises may monitor the money awarded and repaid, and ensure implementation is according to the budget and time frame. Monitoring & Evaluation
  8. 8. Tailor-made Evaluation Studies Evaluations conducted during the life of the project should provide actionable recommendations to improve the quality of the project in the time remaining. The final evaluation findings must be incorporated into the strategy design for subsequent programming. The evaluation exercises can be categorized under three broad heads as enumerated below. According to evaluation timing According to who conducts the evaluation According to evaluation technicality or methodology Formative evaluations occur d u r i n g p r o j e c t / p r o g r a m implementation to improve performance and assess compliance. Internal or self-evaluations are conducted by those responsible f o r i m p l e m e n t i n g a project/program. They can be less expensive than external evaluations and help build staff capacity and ownership. H o w e v e r, t h e y m a y l a c k c r e d i b i l i t y w i t h c e r t a i n stakeholders, such as donors, as they are perceived as more subjective (biased or one-sided). These tend to be focused on learning lessons rather than demonstrating accountability. Real-time evaluations (RTEs) a r e u n d e r t a k e n d u r i n g p r o j e c t / p r o g r a m implementation to provide i m m e d i a t e f e e d b a c k f o r modifications to improve ongoing implementation. Emphasis is on immediate lesson learning over impact evaluation or accountability. Summative evaluations occur at the end of project/program implementation to assess effectiveness and impact. External or independent evaluations are conducted by evaluator(s) outside of the implementing team, lending it a degree of objectivity and often technical expertise. These tend to focus on accountability. Meta-evaluations are used to assess the evaluation process itself. Some key uses of meta- evaluations include: take inventory of evaluations to inform the selection of future evaluations; combine evaluation results; check compliance with evaluation policy and good practices; assess how well evaluations are disseminated and utilized for organizational learning and change, etc. Midterm evaluations are formative in purpose and o c c u r m i d w a y t h r o u g h implementation. Participatory evaluations are c o n d u c t e d w i t h t h e beneficiaries and other key stakeholders, and can be empowering, building their capacity, ownership and support. Thematic evaluations focus on one theme, such as gender or environment, typically across a number of projects, programs or the whole organization. 06Monitoring & Evaluation
  9. 9. According to evaluation timing According to who conducts the evaluation According to evaluation technicality or methodology Final evaluations are summative in purpose and are conducted (often externally) at the completion of project/program implementation to assess how well the project/ program a c h i e v e d i t s i n t e n d e d objectives. Joint evaluations are conducted collaboratively by more than one implementing partner, and can help build consensus at different levels, credibility and joint support. Cluster/sector evaluations focus on a set of related a c t i v i t i e s , p r o j e c t s o r programs, typically across sites and implemented by multiple organizations (e.g. National Societies, the United Nations and NGOs). E x - p o s t e v a l u a t i o n s a r e conducted some time after implementation to assess long- term impact and sustainability. Impact evaluations focus on the effect of a project/ program, rather than on its management and delivery. Therefore, they t y p i c a l l y o c c u r a f t e r project/program completion during a final evaluation or an ex- post evaluation. However, impact may be measured during p r o j e c t / p r o g r a m implementation during longer projects/programs and when feasible. It highlights some of the challenges in measuring impact. Proper management of an evaluation is a critical element for its success. There are multiple resources to support evaluation management which identify the key criteria and standards that guide how we plan, commission, conduct, report on and utilize evaluations. The framework mentioned in the beginning is to be applied to all evaluation activities. It draws upon the best practices to ensure accurate and reliable evaluations that are credible with stakeholders. 07Monitoring & Evaluation
  10. 10. Evaluation Questions and RBM 08 Impact Outcomes Output Ÿ Were the operations objective achieved? (effectiveness) Ÿ Did the outputs lead to intended outcome? Activities Inputs Ÿ Are the benets likely to be maintained for an extended period after assistance ends? (sustainability) Ÿ What changes did the project bring about? Ÿ Were the operation’s objectives consistent with beneciaries’ needs and with agency policies? (relevance) Ÿ Were there any unplanned or unintended changes.? Ÿ Were activities implemented on schedule and within budget? (efciency) Ÿ Were outputs delivered economically? Ÿ Were stocks of items available on time and in the right quantities and quality? Monitoring & Evaluation
  11. 11. Six Key Steps for M&E Planning Review the project/program's operational design (RBM/LFA) Identify key stakeholder informational needs and expectations Identify any M&E requirements Scope of major M&E events and functions Identify the purpose and scope of the M&E system 1. Develop an M&E plan table Assess the availability of secondary data Determine the balance of quantitative and qualitative data Triangulate data collection sources and methods Determine sampling requirements Prepare for any surveys Prepare specic data collection methods/tools Establish stakeholder complaints and feedback mechanisms Establish project/program staff/volunteer review mechanisms Plan for data management Use an indicator tracking table (ITT) Use a risk log (table) Plan for data collection and management 2. Develop a data analysis plan, identifying the: i. Purpose of data analysis ii. Frequency of data analysis iii. Responsibility for data analysis iv. Process for data analysis. Follow the key data analysis stages: i. Data preparation ii. Data analysis (ndings and conclusions) iii. Data validation iv. Data presentation v. Recommendations and action planning. Plan for data analysis 3. Assess the projects/program's human resources capacity for M&E Determine the extent of local participation Determine the extent of outside expertise Dene roles and responsibilities at each level of the M&E system. Plan to manage project/program team's M&E activities Identify M&E capacity-building requirements and opportunities Plan for M&E human resources and capacity building 5. Itemize M&E budget needs i.Human Resource ii.Capital Expenses Incorporate M&E costs into the project/program budget Review any donor budget requirements and contributions Plan for cost contingency Prepare the M&E budget 6. Anticipate and plan for reporting: i. Needs/audience ii. Frequency iii. Formats iv. People responsible. Plan for information utilization: i. Information dissemination ii. Decision-making and planning Plan for information reporting and utilization 4. 09Monitoring & Evaluation
  12. 12. Minimizing Bias and Errors Minimizing bias helps to increase accuracy and precision. Accuracy means that the data measures what it is intended to measure. For example, if you are trying to measure knowledge change following a training session, you would not just measure how many people were trained but also include some type of test of any knowledge change. As much as we would like to eliminate bias and error in our measurements and information reporting, no research is completely without bias. Nevertheless, there are precautions that can be taken, and the first is to we encounter in our work:be familiar with the major types of bias 1. Selection Bias Selection bias results from poor selection of the sample population to measure/study. Also called design bias or sample error, it occurs when the people, place or time period measured is not representative of the larger population or condition being studied. It is a very important concept to understand because there is a tendency to study the most successful and/or convenient sites or populations to reach (which are often the same). For example, if data collection is done during a convenient time of the day, during the dry season or targets communities easily accessible near paved roads, it may not accurately represent the conditions being studied for the whole population. 2. Measurement Bias Measurement bias results from poor data measurement – either due to a fault in the data measurement instrument or the data collector. Sometimes the direct measurement may be done incorrectly, or the attitudes of the interviewer may influence how questions are asked and responses are recorded. For instance, household occupancy in a disaster response operation may be calculated incorrectly, or survey questions may be written in a way that biases the response, e.g. “Why do you like this project?” (rather than “What do you think of this project?”). 3. Processing Error Processing error results from the poor management of data – miscoded data, incorrect data entry, incorrect computer programming and inadequate checking. This source of error is particularly common with the entry of quantitative (statistical) data, for which specific practices and checks have been developed. 4. Analytical Bias Analytical bias results from the poor analysis of collected data. Different approaches to data analysis generate varying results e.g. the statistical methods employed, or how the data is separated and interpreted. A good practice to help reduce analytical bias is to carefully identify the rationale for the data analysis methods. 10Monitoring & Evaluation
  13. 13. 1 CSR Expenditure Among 10 Major Industries HEALTH 22%RURAL DEVELOPMENT 33% EDUCATION 23% ENVIRONMENT 22% 2012-13 Expense on CSR in the Private and Public Sector 20 400 60 80 100 Slum Area Developent RD Projects Technical Incubators PM Relief Fund Sports Forced Army Veterans National Heritage Environmental Sustainability Reducing Inequalities Education, Vocational Skills Hunger, Poverty, Healthcare 95 5 65 35 18 82 85 15 30 70 95 5 58 42 42 58 63 37 75 25 22 78 PrivateSectorPublicSector 1 As per the annual reports of the top 200 firms for the year 2012-13 2 FICCI and Nextgen CSR in India - Industry Overview 11Monitoring & Evaluation
  14. 14. Fiinovation Expertise A vast bank of expertise gained over the years across a wide range of sectors and research methodology. And also, the flexibility to serve a wide range of projects in the sectors of: Livelihood, Education, Health and Environment. Defined M&E standards that help improve program quality and positively impact the people we serve. The standards followed at Fiinovation reflect the key characteristics of high-quality programs and agency culture that promotes better learning and strengthens accountability to stakeholders. These are critical elements of a high- performing, dynamic learning organization. Triangulation of data is another key step practiced at Fiinovation, which involves collecting data from multiple sources, sometimes using multiple tools, to identify and reduce bias. If triangulation of qualitative data is not carried out, a risk of biasing or distorting the collected data exists, resulting in incorrect or incomplete information which might result in skewed understanding of the circumstances. By collecting data from multiple sources or with multiple tools, we identify and address discrepancies or inconsistencies in the data. Triangulation often leads to additional questions or clarifications, which can be answered through follow-up interviews, discussions or exercises. Core Values We ensure excellent quality of the actionable reports which is intended to provide absolute and detailed insight into the markets, products, competition and the overall perspective at an affordable price. We ensure secrecy for all the research projects that we c a r ry o u t a n d m a i n t a i n confidentiality as per the code of conduct. We guarantee authenticity, reliability and credibility of the data, facts and figures that we gather and disseminate through primary and secondary research. Authenticity Quality Confidentiality 12Monitoring & Evaluation
  15. 15. 13Monitoring & Evaluation The Monitoring and Evaluation Team of Fiinovation has been involved in helping its partners to carry out eld research studies for monitoring, evaluating and assessing the impact of their proposed and ongoing/complete projects. A few of our successfully completed assignments in the current year are mentioned below: Fiinovation partnered with IL&FS Skills to carry out an integrated on-site assessment study for assessing the current status of the community in terms of their readiness and acceptability for skill based trainings. The study was carried out to explore the avenues that shall be available at the proposed site for Power Plant in Nana Layja (Mandvi), Gujarat. Based on the research ndings of Fiinovation, the project aims to develop skill, employment and entrepreneurial opportunities for the population of Mandvi Block in Kutch district with respect to the Nana Lajya Power Plant being put up by IL&FS Group. Fiinovation gathered adequate information through extensive research, both primary and secondary sources for the purpose of the study. The Need Assessment of the project was carried out for 3 months with assessment tools intrinsically focused on the need and acceptance of the community. Fiinovation has also conducted a summative evaluation study of the skill training institute (Cairn Enterprise Center - CEC) of Cairn India Limited (CIL) at Barmer (Rajasthan). The study was carried out for the CSR project of CIL being implemented by IL&FS Skills with a view to evaluate the program performance and its effectiveness by collecting information that can be used for further evaluation. Fiinovation developed tailor-made evaluation parameters for assessing the outcomes of the skill training intervention. The framework used empirical evaluations aimed at elucidating and substantiating the linkages between the trainings provided and the intended or observed benets. The project required unbigoted opinion along with technical expertise which was provided by Fiinovation for assessment and recommendations for future course of action. With its team of experts, Fiinovation has successfully conducted evaluation and assessment of Adult Literacy Centers (ALC) run by M/s JK Tyre & Industries Ltd. at Kankroli (Rajasthan), Morena( Madhya Pradesh) and Mysore( Karnataka). For this purpose, a detailed research was carried out to assess the efcacy and effectiveness of the ALCs. Some of our Initiatives
  16. 16. Afliations Established in 2008, Fiinovation (Innovative Financial Advisors Pvt. Ltd.) has created a niche as a multi disciplinary research organization in Corporate Social Responsibility (CSR) and Sustainability domain. Headquartered in New Delhi, it has been forging partnerships between credible NGOs and corporations that are looking to implement and initiate community based interventions and initiatives. With a pool of over 4000 credible NGOs spread across the country, the motive has been to create awareness, inspire innovative thinking and action with sustainable development at its core. These NGOs are empanelled post a thorough and stringent due diligence check through secondary research and on ground evaluation. Fiinovation is known in the development sector for enhancing quality across organizational value chain through its in-house Proposal Design & Research Laboratory for designing innovative CSR initiatives. Fiinovation also supports corporations through conducting need assessment, monitoring and evaluation as well as impact assessment of their programs. The distinct practices of Fiinovation include CSR Portfolio Management (CPM), CSR-CSO Partnership, Initiative Design, Initiative Management, Monitoring & Evaluation, Impact Assessment and CSR audit across verticals of health, environment, education and livelihood, which contribute to INR 15 crores in the top line of nancial statement. Fiinovation has been a knowledge partner with various industrial bodies such as Confederation of Indian Industries (CII) and Institute of Directors (IOD). With CII, the partnership emphasizes on research for promotion of afrmative action along with CSR and social enterprises. The organization has deliberated with various industrial leaders at numerous conferences conducted on corporate governance, environment management and CSR & sustainability during its path breaking partnership with IOD. Being a research organisation, Fiinovation also contributes to IOD's Master class for Directors that has a wide spectrum of subjects such as corporate governance, sustainability, HR practices, nance, corporate social responsibility, environment management and board effectiveness. Other than this, the organization has been closely associated with other forums such as NGO BOX and World CSR Congress. Organization Brief 14Monitoring & Evaluation
  17. 17. Awards & Recognitions Mr. S. Chakraborty along with Smt. Sheila Dixit at the 24th World Congress on Total Quality and Leadership 15 Fiinovation has been conferred with the “CSR Team of the Year” & “Caring Company Award” at the World CSR Day 2015. It was also the recipient of the 'Best Social Innovation: Product or Service' at the World CSR Day 2014. Apart from this, the company was bestowed with the "Manager of the Year" & "Best Enterprise of the Year - in the eld of Health, Education, Environment and Livelihood with a focus on CSR & Sustainability" Awards by the European Business Assembly at the Socrates Award Ceremony, Summit of Leaders in 2014 at Oxford Town Hall, United Kingdom. Mr. Soumitro Chakraborty addressing the gathering at CII conference - Bridging the Gap: Fostering growth through Education & Entrepreneurship Monitoring & Evaluation
  18. 18. Media Links fiinovation 24/30, Ground Floor, Okhla Industrial Estate, Phase III, New Delhi - 110020, India Phone: 011-42332200 | Fax : 011-42332205 Email : info@, | Website : www.