Impact assessment cb aapproach_kiis_jan2011_eng


Published on

Published in: Technology, Real Estate
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Impact assessment cb aapproach_kiis_jan2011_eng

  1. 1. COMMUNITY BASED DEVELOPMENT APPROACHTO LOCAL DEVELOPMENT IN UKRAINEThe results of sociological research«Evaluation of the impact of community based developmentapproach to local development implemented by the UNDP Projectsin Ukraine financed by the European Union and other donors»Kyiv – 2011
  2. 2. 2This publication was prepared with financial support from the Community BasedApproach Project which is financed by European Union, co-financed and implemented bythe United Nations Development Programme in Ukraine.All thoughts, conclusions and recommendations belong to the authors and editors ofthis publication and do not necessary reflect the opinions of international donors of theCommunity Based Approach Project. For more information on the Project activities AND ACKNOWLEDGEMENTSProject teamVolodymyr Paniotto, director general, KIISNatalya Kharchenko, executive director, KIISDmytro Khutkyy, research fellow, KIISAnton Grusheckyi, junior research fellow, KIISVitaliy Kisil, junior research fellow, KIISKateryna Skrypka, head of fieldwork department, KIISOlga Zhykhoruk, junior research fellow, KIISNatalia Sevekina, head of a control department, KIISOleksiy Gorbachyk, head of data processing department, KIISAndriy Androsiuk, junior research fellow, KIISAuthors of reportVolodymyr Paniotto, director general, KIISDmytro Khutkyy, research fellow, KIISAnton Grusheckyi, junior research fellow, KIISVitaliy Kisil, junior research fellow, KIISPartners and acknowledgementsWe would like to express gratitude to all participants of this research - members ofthe communities, representatives of bodies of local self-governance, local and centralauthorities, businessmen and external experts.We also want to express gratitude to the experts of the UNDP Projects, who offeredvaluable advice and consultations concerning the methodology of the community basedapproach to local development and specifics of its application in Ukraine, and assisted inorganisational work on the field stage: Kurtmolla Abdulganiyev, Dzvinka Kachur, OlgaOsaulenko, Oksana Remiga, Galyna Smirnova, Iryna Skaliy, Tetyana Diyeva, DenisPoltavets, Olena Ruditch, Jaysingh Sah, and Ganna Yatsyuk. We are also grateful to theUNDP experts who extended important contributions to the report: Danylo Bilak, AntoninaIshchenko, Tetiana Matiychyk, and Ayder Seytosmanov.
  3. 3. 3TERMS AND ABBREVIATIONSACMH – Association of Co-Owners of Multi-Apartment HouseBSP – Body of Self-Organisation of PopulationCBA – Community Based Approach (UNDP Project)CBDA – Community Based Development Approach (approach to local development)CC – City CouncilCDP – Community Development PlanCIDP – Crimea Integration and Development ProgrammeCM – Communities MembersCO – Community Organisation1CO-MT – Community Organisation Management TeamCRC –Community Resource CentreCRDP – Chornobyl Recovery and Development ProgrammeEU – European UnionFG – Focus-Group DiscussionII – In-depth interviewKIIS – Kiev International Institute of SociologyLDF – Local Development ForumMCSD – Municipal Council of Sustainable DevelopmentMCSDF – Municipal Council of Sustainable Development ForumMGSDP – Municipal Governance and Sustainable Development ProgrammeMSU – Municipal Support UnitOC – Oblast CouncilOCC – Oblast Coordination CouncilOSA – Oblast State AdministrationOIU – Oblast Implementation UnitRC – Rayon CouncilRCRC – Rayon Community Resource CentreRE – Regional ExpertsRSA – Rayon State AdministrationUNDP – United Nations Development ProgrammeVC – Village Council1According to CBA manual, Ukrainian law allows inhabitants of a given territory to assemble and discuss ondevelopment agenda pertaining to their community (territory). They may decide and declare the formation oftheir organization through a protocol. Such organisation is recognised as a CO. The inhabitants may registerthis CO under a particular legal framework (such as BSP, cooperative, ACMH, NGO etc.) and may acquire aspecific legal name/recognition as defined by the law of the country. Therefore the generic name “CO” will beused to designate all the forms mentioned above.
  4. 4. 4CONTENTSInformation about the research…………………………………………………………………...6Resume of research results……………………………………………………………………….8Section 1. Introduction………………………………………………………………………...111.1.Methodology of community based approach to local development………………...111.2.Community based development approach implemented by UNDP Projects inUkraine…………………………………………………………………………………….121.3.Expected results of approach impact and efficiency…………………………………151.4.Research goal and objectives …………………………………………………………151.5.Report structure ………………………………………………………………………….16Section 2. Research methodology…………………………………………………………..182.1. Spheres and components of evaluation of the approach effectiveness andimpact............................................................................................................................182.2. Focus groups discussions with citizens and local authorities…………………........192.3. In-depth interviews with regional experts……………………………………………..202.4. In-depth interviews with national experts……………………………………………..212.5. Survey of regional experts……………………………………………………………...212.6. Survey of community members ……………………………………………………….222.7. Methodology of data analysis…………………………………………………………..232.8. Research operational hypotheses…………………………………………………….24Section 3. Evaluation of the approach methodology effectiveness……………………273.1. Involvement of population……………………………………………………………...273.2. Pattern of priorities setting……………………………………………………………...293.3. Effectiveness of co-financing scheme…………………………………………………303.4. Conclusions about the effectiveness of the approach methodology……………...33Section 4. Evaluation of the approach impact on local self-governance……………..344.1. Appropriateness of support organisations created ………………………………….344.2. Level of transparency, accountability and equality…………………………………..434.3. Quality of the strategic planning, bottom-up planning……………………………….454.4. Access to information about activities of local authorities…………………………..464.5. Role of local business…………………………………………………………………..484.6. Citizen- authorities relationships……………………………………………………..484.7. Conclusions concerning influence of approach on local self-governance………54Section 5. Evaluation of approach impact on service delivery in sectors, which aresupported by the UNDP Projects……………………………………………………………..565.1. Cost of service creation or rehabilitation………………………………………………565.2. Cost of service delivery…………………………………………………………………565.3. Quality of service delivery………………………………………………………………575.4. Sustainability of social infrastructures created…………………………………….....605.5. Conclusions concerning services delivery in sectors, which are supported by theUNDP Projects………………………………………………………………………………..60Section 6. Evaluation of influence of approach on living quality of targetgroups................................................................................................................................616.1. Changes in material conditions………………………………………………………...636.2. Changes in economic conditions………………………………………………………646.3. Changes in health……………………………………………………………………….646.4. Changes in the psychological self-feeling……………………………………………656.5. Changes in the social cohesion………………………………………………………..666.6. Conclusions concerning impact of approach on quality of life of the targetgroups………………………………………………………………………………………….69Section 7. Evaluation of dissemination of experience, factors of success of theapproach, possibilities of improvement of public policy concerning localdevelopment……………………………………………………………………………………...707.1. Dissemination of the gained experience……………………………………………..70
  5. 5. 57.2. Factors of successful approach implementation…………………………………….707.3. Possibilities of perfection of public policy concerning local development…………757.4. Conclusions concerning dissemination of experience, factors of success approach,possibilities of improvement of public policy concerning local development…………..77Section 8. Conclusions, recommendations and discussion…………………………….798.1. Conclusions………………………………………………………………………………798.2. Recommendations concerning perfection of approach and public policy concerninglocal development…………………………………………………………………………….83Appendix A: Spheres and components of evaluation of efficiency and influence of theapproach.............................................................................................................................86Appendix B: Detailed information on research realisation……………………………………88Appendix C: Guide for realisation of focus groups discussions……………………………..92Appendix D: Guide for realisation of in-depth interview with regional experts……………..96Appendix E: Guide for realisation of in-depth interview with national experts……………100Appendix F: Questionnaire for survey of regional experts………………………………….103Appendix G: Questionnaire for survey of community members …………………………...107
  6. 6. 6INFORMATION ABOUT THE RESEARCHThe research «Evaluation of the impact of community based development approachto local development implemented by the UNDP Projects in Ukraine financed by theEuropean Union and other donors» lead by the UNDP/EU Project "Community BasedApproach" was conducted by the Kyiv International Institute of Sociology.The goal of the research was to assess the impact and evaluate the effectivenessof the community based development approach to local development that had beenimplemented in the four UNDP Projects in Ukraine: "Crimea Integration and DevelopmentProgramme", "Chornobyl Recovery and Development Programme", "MunicipalGovernance and Sustainable Development Programme" and "Community BasedApproach to Local Development Project".In order to attain this goal, the following objectives were determined:– To collect the opinions of the representatives of involved parties on the efficiency of themethodology of the community based development approach to local development;– To collect the opinions of the representatives of involved parties on the impact ofapproach on local self-governance;– To collect the opinions of the representatives of involved parties on the approach’simpact on the service delivery in the sectors, supported by the projects;– To collect the opinions of the representatives of involved parties on the approach’simpact on the living quality of target groups;– To detect the factors of effectiveness of the approach implementation and to developrecommendations concerning dissemination of the study results, improvement of theapproach and political initiatives.Initially, a literature review was made. It helped to form a stronger understanding ofthe community based development approach to local development and infer aboutprospective results of the approach implementation among the population by the UNDPProjects. The UNDP experts consulted with the KIIS team, whose assistance facilitatedqualitative research implementation. They explained the specifics of the functionality of theproject, advised how to form the sample and helped in the organisation of fieldwork. Aconceptual scheme of the research was developed, the structure of collection and analysisof sociological data determined, and the approach effectiveness and impact evaluated.11 Focus group discussions were conducted, lighting up different partners’ attitudestowards the impact of community based development approach to local development. 27In-depth interviews that were conducted with the regional experts (community members,village and rayon authorities, and oblast project staff) opened up possibilities to improvethe understanding of different aspects and local peculiarities of the approach realisation.206 Regional experts were interviewed: the data on these interviews represents the localrayon and oblast authorities involved in the UNDP Projects; they helped to adopt theapproach in the MGSDP and CBA projects. The survey of 855 communities membersprovided important information about the impact of the approach on the lines of directbeneficiaries. 9 In-depth thematic interviews conducted with the national experts were avaluable source of the information regarding the dissemination of experience gained bythese UNDP Projects, the integral vision of propositions concerning further improvement ofthe community based development approach to local development, and therecommendation about desirable political executive initiatives that might help to replicatethis approach in Ukraine.As a result, reports regarding the results if the research by each method wereprepared. The final published report accumulates and synthesizes the main results of theresearch and recommendations.
  7. 7. 7RESUME OF RESEARCH RESULTSWe have sufficient grounds to conclude that overall the methodology of communitybased development approach to local development can be evaluated as effective.Partners adhere to the requirements of approach methodology and are ready to assist inits dissemination.There is a high level of involvement of the population in activities of communityorganisations.Community members participate in activities of community organisations in anumber of ways. Almost all community organisation members (93.9% of surveyedcommunity organisation members) participated in community general meetings at leastonce. The majority of community organisation members (86.8% of surveyed communityorganisation members) are involved in the decision-making process of the communityorganisations on the basis of consensus, vote or questionnaire. Members of thecommunity organisations (88.5% of surveyed community organisation members) arewidely informed about the activities of community organisations. Almost all members ofcommunity organisations (95.1% to 100% of surveyed community organisation membersdepending on UNDP Project) pay their membership fees. The majority of communitycitizens (76.6% of surveyed community organisation members) reported that they know ofcases when non-members of community organisations participated in community projectsthrough voluntary contributions in the form of money or work. 29.9% of surveyedcommunity organisation members also reported that members of community organisationsmade additional contributions on voluntary basis.Priorities of community development are identified taking into account interests ofthe majority of community organisation members and are mainly independently frominfluence of authorities.The majority of community organisation members (63.2% of surveyed respondents)agree that priorities were set solely or mostly by the influence of community members.The current co-financing sample is generally considered by partners as an effectiveone, the stakeholders want and sometimes do invest more than is required.The majority of stakeholders recognise the co-financing scheme as rather or veryeffective (75.7% of regional experts and 81.8% of community organisation management).71.4% of regional experts recalled cases when community members, local business orlocal authorities made larger contributions than the specified minimal proportion.It was discovered that the community based development approach has a manifestpositive impact on local self-government in terms of self-organisation of communities andlocal authorities. This conclusion is supported by evidence regarding the generalappropriateness of the established support structures to the set objectives, theirtransparency, accountability and equity, working strategic bottom-up planning, increasedaccess to information on activities of local authorities, participation of local business assponsors and improved citizen-authorities relationships.The research demonstrates that the support structures established (CO, ACMH,RC, MSU, LDF, MCSD, MCSDF and OCC) fully correspond to the established objectives.They sometimes implement additional functionalities.The established support structures contribute to the local development andcooperation between citizens and authorities. 84.9% of all surveyed regional expertsassess community organisations as rather or very effective for the local communitydevelopment process. In CBA Project LDFs were valued by 92.7% of surveyed CBAregional experts as rather or very effective. RRCs, ORCs and OCC were qualified asrather or very effective by at least 80% of surveyed CBA experts. In MGSDP Project MSUswere characterized as rather or very effective by 91.1% of surveyed MGSDP regional
  8. 8. 8experts. While MCSDFs were rated as rather or very effective by 73.3% of surveyedMGSDP regional experts.Community organisations favour cooperation between communities and localauthorities – this is recognised by 92.7% of all surveyed regional experts. LDFs, OCCsand MCSDFs have been rated as rather or very effective by 93.3%, 85.3% and 69.6% ofregional experts respectively.These organisations are widely used beyond the UNDP Projects. The supportstructures disseminate information materials, approach methodology and acquiredexperience to other communities. 70% of all surveyed regional experts recalled caseswhen communities not participating in UNDP Projects utilized the available resourcecentres. The model of community based development approach is replicated by othercommunities. 75.4% of surveyed regional experts reported they were aware ofcommunities which self-organised following the examples of communities participating inUNDP Projects. 86% of surveyed members of the community organisations reported thattheir community organisations implement their initiatives outside the UNDP Projects attheir own expense. 71.4% of COs apply for other grants or competitions beyond UNDPProjects, and 60% of the COs which applied won at least one grant or competition.The stakeholders are eager to support established support institutions in the futureand have a positive outlook for high sustainability. 70% of all surveyed regional expertspredicted that the institutions will rather or very probably function after cooperation withUNDP Projects. Local authorities and UNDP Project staff expressed readiness to assistthem personally: 90% of all surveyed regional experts think it is rather or very probablethat they will support community organisations in the future.There is an explicit increase in quality of human resources and mutual learning ofcommunity leaders and representatives of local authorities. Accumulation of knowledgeand application of skills by community leaders and local officials has been promoted. Itshould be mentioned that at least 97% of all surveyed regional experts report growth oftheir knowledge of local governance and 98% – of skills in cooperation with communities.The established support structures are characterized by a high level oftransparency, accountability and equality in their activities.Activities of community organisations are virtually transparent. At least 59% ofsurveyed members of community organisations are rather or fully informed abouttendering procedures, public auditing, at least 76.2% – about priorities setting, reports onusage of CO funds, works on objects, and 90% – about decision making process. Only0.6-8.8% of surveyed members of community organisations are completely unaware of COactivities. Members of communities have relatively equal access to benefits created bycommunity projects. 95.7% of surveyed community members are confident that allcommunity members have potential access to the established or rehabilitated services.The model of strategic planning with the mechanism of bottom-up planning is widelydisseminated. The interests of communities are accounted in the strategic plans of rayondevelopment and interests of urban communities - in the strategic plans of urbandevelopment.75.7% of all surveyed regional experts confirm that the priorities of communitydevelopment are rather of fully accounted in a rayon or city development plan. Only 1.5%of the experts say such priorities are unaccounted. 67.8% of all surveyed regional expertsbelieve the community development priorities are literally implemented. Only 2.5% thinkthat they are not implemented.It was revealed that there was improvement in access to information on activities oflocal authorities, which indicates an increase in its transparency.At least 80% of all surveyed regional experts have noticed an increase in theamount and quality of informing of citizens regarding activities of local authorities. Among
  9. 9. 9community members from the main (beneficiary) group comparing to control (where UNDPProjects were not introduced) there are 26.5% more than those who have recognised anincrease in accessibility of necessary information about activities of local authorities. Theanalogues difference in awareness of activities of local authorities is 25.4% for the benefitof the beneficiary group.Representatives of local businesses have expressed a desire to sponsor localdevelopment by making contributions to the projects of community organisations.There is a manifest improvement of democratic character and efficiency of co-operation between citizens and authorities. As a whole the result of approachimplementation is a strong improvement of relationships between the citizens and localauthorities.There are positive dynamics of transparency of the local government activities. Atleast 89.8% of all surveyed regional experts note some or great increase in the level ofaccessibility of the local officials and openness to dialogue. Simultaneously, 50.6% of thecommunity members from the main (beneficiary) group, in comparison to the 23.3% fromcontrol group that believe that openness to dialogue has rather or very increased. Localauthorities in their activities more often take into account opinions of citizens. As much as91.7% of all surveyed regional experts remarked that there was some or great increase inconsideration of citizens’ opinions. At the same time, 46.7% of community members fromthe main (beneficiary) group, in contrast with the 20.8% from the control think that suchconsideration rather or very increased.There is a marked difference in the level of citizens’ trust towards local officialsamongst the communities who have participated in UNDP Projects and those which havenot. On a scale from 0 (“Absolutely do not trust”’) to 10 (“Completely trust”) the level oftrust towards local authorities differs from 6.4 in main (beneficiary) group to 4.9 in controlgroup. It can be seen that there are positive dynamics in citizens’ trust towards localofficials. It was found that in the main (beneficiary) group 41.2% admit their trust to localauthorities has rather or very increased, while in the control group only 20.8% respondedin such a manner. Furthermore, 71.3% of CO management feel confident incommunication with authority officials, while only 7.4% of them feel diffident.Clearly, there is an increase in cooperation between communities and localauthorities. 53.8% of community members from main (beneficiary) group, in contrast to19.5% from control group have noticed that cooperation between citizens and authoritieshas rather of very increased.The positive dynamics in satisfaction of citizens with work of local officials, shouldalso be mentioned. 47.9% of surveyed community members in main (beneficiary) grouphave noticed that they are more satisfied with the work of local authorities in comparisonwith 20.9% in the control group. Similarly, community members of the main group aremore satisfied with the current work of local authorities than the community members ofthe control group.From the study results it can be adequately stated that there was in increase insatisfaction with quality of services that were covered by the projects. The creation orrehabilitation of services is cost-effective. There is an increase of relative economy in theuse of these services. Moreover, the created or rehabilitated communal infrastructures arecertainly and potentially sustainable.Some national experts remarked that the creation and delivery of services using thecommunity based development approach is cheaper than that by most local authorities orother organisations without community involvement.Community projects have an ambiguous influence on relative cost of service anddelivery. From the one hand, the cost of services is smaller compared to communitieswhere there are no community projects with the community based approach. However,
  10. 10. 10while some community members see an increase in energy consumption, others seedecrease. The CO management claim that they apply efforts to save gas (77.9%), heat(84.7%), electricity (91%). 79.8% of CO managers reported their communities try to savewater.Community projects undoubtedly increase quality of community services. Anincrease in quality of heating observe 65.2% of surveyed members in communities whereenergy-saving projects of heating were introduced, compared to 36.3% in communitieswith other projects and 39.2% in control group. Similarly, it is 66.4% in contrast with 32.7%and 35% for street light projects. An analogues improvement was seen in the watersupply. An increase in quality of water supply observe that 75.9% of the surveyedmembers in communities where projects of water supply were introduced, compared to38.6% in communities with other projects and 38.5% in the control group. Similarly, it is67.5% in comparison with 45.8% and 36.9% respectively for waste management projects.The greatest difference in quality is for school transport and healthcare. For schooltransport the difference is 32.6-37.7%, and for healthcare it is 42-42.4%. Members ofcommunity organisations express eagerness to support established or rehabilitatedcommunal infrastructures.As a result of this data analysis it is relevant to admit an increase in economicconditions, psychological self-feeling and considerable increase of social cohesion ofcommunities.Positive changes in life during recent years acknowledge 32.3% of communitymembers in the main (beneficiary) group compared to 22.3% in control group. There aresome reasons to suppose improvement in material conditions, however because of thesmall sample, positive qualitative results are not supported by the quantitative data. Due tocommunity activities, the employment rate has increased in the main (beneficiary) group,whereas it has not in the control group. One might pose certain considerations to supposeimprovement in health, but because of the small sample, positive qualitative results are notsupported by the quantitative data. As a result of community self-organisation there is anincrease in self-confidence of members of community organisations implementingcommunity projects. There is a marked difference between main (beneficiary) and controlgroups in level of trust to members of their own community. On a scale from 0 (“absolutelydo not trust”) to 10 (“completely trust”) citizens from main group have a level of trust of 7.7while citizens from control group have one of 6.1. 50.9% of members of communitiesparticipating in UNDP Projects are sure that during the last years unity of theircommunities has increased when only 18.8% of communities not participating in UNDPProjects observe such an increase.Community members from the main (beneficiary) group are more satisfied with thesocial life in their village or town than from control group (on a scale from 1 (“do notsatisfied at all”) to 5 (“completely dissatisfied”) difference is 3.3 compared to 2.6respectively). Similarly, community members from the main (beneficiary) group are moresatisfied with future prospects of community development than from the control group (thedifference is 3.8 compared to 2.7 respectively).The impact of community based development approach differs in the four UNDPProjects. We might imply several factors which positively influence the result.Community based approach to local development has a stronger positive impact ifthere is: a longer duration of institutional support, more intense financial and humanresource inputs per territory or per community, more intense involvement of partner localauthorities, work in initially more coherent rural communities.
  11. 11. 11SECTION 1. INTRODUCTION1.1. Methodology of the community based development approach to localdevelopmentAccording to the projects’ documents of UNDP Projects, the overall objective of theapproach is to create an enabling environment for long-term self-sustaining socio-economic and community development at a local level by promoting local self-governanceand community-based initiatives that would contribute to overall human development andattainment of Ukraine’s Millennium Development Goals.By stimulating people’s participation in local debates about priority needs of theircommunities, and by helping them find and implement solutions to local problems, UNDPProjects aim to build a sense of confidence in citizens, enhance their role in local decision-making, and facilitate the dialogue between citizens and the Government. Communitymobilisation and improved dialogue and cooperation between citizens, their associations,donors and local government are aimed to lay the ground for local long-term developmentplanning.This provides opportunities to ensure consistency of national policies and the waythe EU implements its own local development policies concerning local development. TheUNDP and the EC play its role in disseminating the community-based developmentmethods at a time where it is most needed for (a) supporting the government’s efforts todeal anew with local development issues in an EU-compliant manner; and (b) preparingground by delivering immediate results for the longer-term regional development projects.Specific objectives of the approach are listed as the following:1. Improve living conditions in rural, semi-urban and urban communities throughoutUkraine by promoting sustainable rehabilitation, management and operation of basic socialand communal infrastructure and services through community-based self-help initiatives.Community organisations, relevant local (village and municipal), rayon and regionalauthorities carry out the rehabilitation of basic social infrastructure and municipal serviceswithin major national MDG-based priority:• Health (local health posts network) (UMDG 4)• Energy (energy conservation measures at local level, etc) (UMDG 3)• Environment (UMDG 3)• Water management (UMDG 3)• Local transport systems (UMDG 1)During the introduction of community projects, each participating community isguided through the following steps of participatory community development:• Sensitization and community self-assessment• Formation of community organisations• Community development planning• Project identification, prioritization and implementation• Follow-up (community progress review mechanism established so thatcommunity members can codify past achievements and build on them)2. Demonstrate effective participatory local governance and decentralizedmanagement mechanisms throughout Ukraine for public service delivery by promotinginclusive, self-governing community organisations undertaking self-help initiatives inpartnership with local authorities, private business entities and other stakeholders.The dialogue between community organisations and local authorities is formalizedthrough the establishment of Local Development Forums (in case of CBA Project) and thelike. Such forums are composed of representatives of local authorities and communityorganisations, private business, public utilities companies, and local NGOs.
  12. 12. 123. Enhance relevant professional skills and knowledge of community organisationsand local authorities to initiate and maintain participatory local development process onsocial economic development and public services delivery.UNDP Projects develop institutional capacities of community organisations andlocal authorities to identify community needs and priority, to manage and monitorparticipatory local process for a sustainable social-economic development and efficientpublic service delivery. UNDP Projects provide training and support to ensure that effortsare carried forward to implement community development plans. Various village,municipal, rayon and oblast resource centres are created for community mobilisation.For communities to become self-confident and raise their self-esteem, the approachprovides a transfer of previous positive achievements demonstrated by UNDP Projects ina significant number of settlements in 24 regions of Ukraine and in Autonomous Republicof Crimea.According to the approach methodology, the interested communities gather generalmeetings and create community organisations which might take various legal forms(NGOs, BSPs, ACMH etc.). To form a community organisation, it must be formed by atleast 80% of households of the corresponding community. The priorities of communitydevelopment are settled in a democratic way (by vote or survey).1.2. Community based development approach implemented by UNDP Projects inUkraineIn response to the acute challenges that Ukraine is facing, with the objective toachieve sustainable development, UNDP applies the community-based approach to thelocal development in Ukraine. UNDP supports the sustainable social, economic andenvironmental development mainly through the introduction of the four projects in Ukrainein close cooperation with international organisations and development agencies. Thecommunity based development approach to local development mobilizes communities totake responsibility for the improvement of their own life.There are four UNDP Projects in Ukraine that have applied the community baseddevelopment approach to local development: Crimea Development and IntegrationProgramme (CIDP); Chernobyl Recovery and Development Programme (CRDP);Municipal Governance and Sustainable Development Programme (MGSDP) andCommunity Based Approach to Local Development (CBA). All of them employ the socialmobilisation approach, but they each have specific objectives, target populations andinstitutional arrangements.Crimea Integration and Development Programme is a joint initiative of theinternational donor community: Canadian International Development Agency (CIDA),governments of Denmark, Greece, Netherlands, Norway, Swiss Agency for Developmentand Cooperation (SDC), Swedish International Development Cooperation Agency (SIDA)and Turkey. The Project goal is sustainable social and economic development of Crimeataking into account its national and cultural diversity.To reach this goal it was essential: (1) to facilitate the development of democraticgovernance and to invite members of multiethnic communities to take an active part insolving current issues in partnership with the local authorities. To accomplish this CIDPencouraged the villagers to unite into population bodies of self-organisation of population(BSP) – a specific legal form of community organisation, which define problems, developand implement projects aimed at overcoming these problems, while the emphasis is onmaking the residents themselves the initiators of the change. In the course of theprogramme realisation such direction as (2) economic development in the rural areas isimplemented. Thus, CIDP stimulates the establishment of the agricultural cooperatives,while this form of self-organisation provides to the farmers joint problem solving. Another
  13. 13. 13important programme direction is (3) encouraging tolerance and social cohesion within theCrimean society through education and culture. Realisation of the set goal also envisages(4) speeding up responsiveness to potential conflict zones through the human securitymonitoring system.Since 1995 and within the budget of USD 4.4 million over 629 communityorganisations with 400 villages (approximately 200 thousand CO members), where 419community projects for 143,000 beneficiaries which were implemented with a total cost of11.9 million dollars, were supported in Crimea. For CIDP each party’s contribution incommunity project was specified depending on the project. The community organisationand community project activity of CIDP was closed by 2008.More information about the CIDP project is available at Recovery and Development Programme was initiated by the UN and isimplemented with the aid of the donors, such as: UN Trust Fund for Human Security andthe Government of Japan, the Canadian International Development Agency (CIDA), SwissAgency for Development and Cooperation (SDC). The project goal is to support theGovernment of Ukraine’s efforts in mitigating long-term social, economic andenvironmental consequences of the Chernobyl disaster, creating favourable conditions oflife and providing sustainable human development in regions affected by Chernobyl.Achievement of this goal includes work in the following areas. Firstly, this involvespromoting improvement of the state policy. Secondly, it requires assisting citizens in self-organisation and self-governance, increasing their potential for defining, developing andimplementing priority programmes of social, economic, and ecological recovery anddevelopment. Thirdly, strengthening the capacity of organisations and institutions thatshould promote socio-economic development and ecological recovery of Chernobylaffected areas.The programme has been operating since 2002 in selected regions of Kyiv,Zhytomyr, Chernihiv and Rivne regions. During this time, based on the USD 6.6 millionbudget 279 community organisations in 192 villages (more than 20,000 members), where190 community projects were implemented with a total cost of UAH 18.5 million for some200,000 beneficiaries, were supported in these four regions. For CRDP the co-financingscheme is the following: local authorities’ part – by 45%, CRDP – by 40%, local business –by 10-15%, community – approximately 5%.More information about the CRDP project is available at Governance and Sustainable Development Programme is implementedby UNDP with the support from donors, such as: Canadian International DevelopmentAgency (CIDA), Swiss Agency for Development and Cooperation (SDC) and RoyalNorwegian Embassy. The programmes goal is to build capacity of local communities andmunicipalities to participate in joint decision-making and use this ability for multilateralcooperation and activities aimed at strengthening of the local socio-economic andenvironmental governance for sustainable development.Implementation of this goal includes the following tasks. First, capacity building ofthe central government concerning fiscal and administrative decentralization has to benefitthe local communities. Second, capacity building of the local authorities in defining,developing and implementing strategies for sustainable local development needs to occur.Thirdly, the capacity of communities in improvement of local social, economic andenvironmental conditions through self-organisation should be increased. The mainapproach is social mobilisation, which includes promoting the establishment of localcommunity organisations (Associations of Co-Owners of Multi-Apartment Houses and theirFederations, Civil-Society Organisations of schools and kindergartens and their Networks,Service Cooperatives). These institutions are founded on the principles of self-help andgood governance. Through the Programme interventions, the institutional capacity of these
  14. 14. 14organisations builds up so that they were able to plan, mobilize resources and identifypriorities to address their social, economic and environmental problems in a sustainableway. This is supported by the respective city councils and other national / internationaldevelopment agencies.The programme has been operating from 2004, currently it operates in 24 townsand 5 villages of 12 regions of Ukraine. During this time, based on the USD 6.6 millionbudget 550 community organisations in 29 cities (more than 56,000 members ofcommunity organisations) were created in these cities, and 272 community projects forabout 167,000 beneficiaries were implemented with a total cost of UAH 30.8 million. ForMGSDP the co-financing scheme is the following: local authorities’ part – by 45%, UNDP –by 45%, community part – not less than 10% (each year municipal authorities’ partincreases by 5% and MGSDP part – decreases by 10%, if the same communityparticipates in the Programme for the second time its share increases by 10%).More information about the MGSDP project is available at Community Based Approach to Local Development Project is a nationwideproject implemented in Ukraine funded by the European Union and UNDP with the goal tocreate favourable environment for sustainable social and economic development at thelocal level through self-organisation and social mobilisation of communities, developmentand implementation of the small-scale community initiatives in all regions of Ukraine andCrimea.The Project aims at restoration and efficient operation of basic local infrastructurefacilities (especially in such priority areas such as the health care, energy, environment,public water supply, and public transportation). In addition, project objectives includeimproving professional skills and knowledge of community organisations, strengtheningthe institutional capacity of self-governing community organisations and local authorities.The accent is made on determination of needs and priorities of community development aswell as management practices of local self-government through community organisations.The basic principle of this project dwells on direct participation of the community in solvingurgent problems. It relies on decentralized mechanisms for providing public services. It isassumed that communities themselves can best identify critical issues and priorities fordevelopment at the grassroots level. The key mechanism for the project implementation isto create a network of self-governing local community organisations capable of initiatingand implementing activities aimed at improving the living conditions of the communityresidents with local authorities, private business and other stakeholders’ participation. Forthis purpose in the course of the project specially created teams that helped communitiesto mobilize and organize themselves, conducted trainings for activists, and helped with thedevelopment and implementation of the community development plans. Implementation ofthe initiatives was held with the participation of the communities and other projectparticipants, including authorities. Initiatives were financed by the project funds, althoughcommunities themselves were making money contributions (not less than 5% of the valueof a particular initiative).The project has been operating from 2007 in 209 rayons of all 24 regions of Ukraineand Crimea. During this time over 1151 community organisations in 1125 villages (418 789members of community organisations), where 1310 projects were implemented on a totalbudget of UAH 193,6 million serving 1,209,069 beneficiaries, were supported in all regionsof Ukraine. For CBA co-financing scheme is the following: 50% contributes CBA, 45% –local authorities, 5% – community.More information about the CBA project is available at
  15. 15. 15Table 1.2.1.Comparative statistics on the four UNDP ProjectsUNDP Project /Key informationCIDP CRDP MGSDP CBAStart year 1995 2002 2004 2007Project budget USD 4.4 million USD 6.6 million USD 8.8 million EUR 13.3millionGeographicalscopeAR Crimea Kyiv, Zhytomyr,Chernihiv andRivne oblasts12 regions 24 oblasts andAR CrimeaCommunityorganisationscreated629 279 550 1151Settlementsinvolved400 192 29 1125Number ofcommunityorganisationsmembers200,000 20,000 56,000 418,789Percent offemale COmembers54% 55.5% 52.2% 57.8%Percent of maleCO members46% 44.5% 47.8% 42.2%Number ofprojectssupported419 190 272 1310Communitiesprojects’ budget(total amount ofinvestmentsfrom allstakeholders)USD 11.9millionUAH 18.5millionUAH 30.8millionUAH 193,6millionBeneficiaries 143,000 200,000 167,934 1.2 million1.3. Expected results of the approach impact and efficiencyAccording to the tasks specified in the UNDP documentation, which apply thecommunity based development approach to local development, the effectiveness of theapproach should be expressed in clearly defined indicators.In particular, community based development approach is considered “effective”,when the following conditions are fulfiled: the majority of the community members (at least80%) should participate in the activities of community organisations; significant part of thecommunity should be represented at the general community meetings; the vast majority ofthe population should be involved in the decision making process on any mattersconcerning their own communities. In addition, community organisations members shouldbe extensively informed about activities of the community organisations.According to the requirements of the approach, priorities of the communitydevelopment should be determined taking into account interests of the overwhelmingmajority of the community members and regardless of influence from representatives ofauthority representatives’.
  16. 16. 16In terms of contributions, all members of the community should pay membershipfees. Most stakeholders (community members, local authorities, local business andinternational donors) should acknowledge the existing co-financing scheme as convenientand effective.For efficient project operation and sustainable local development establishedsupport organisations and structures (CBO, CO, ACMHs, RC, MSU, LDF, MCSDF, OCC,and OIU) should be adequate and useful to carry out the outlined tasks for the needs oflocal development and for citizens/authority cooperation. All engaged parties should beready to support the established organisations in the future.One of the approach implementation directions was the development of humanresources. More specifically, there should be an increase in knowledge and utilization ofskills by community leaders and local officials.Activities of the community organisations should be transparent and accountable tothe community, and the outcomes should be available to all community members, who arethe potential beneficiaries.Due to implementation of this approach, the cost of created or rehabilitated servicesshould be effective in considering the cost of implementation of the community projects bythe community organisations. Similarly, the cost of providing these services should belower in communities where community projects were introduced in comparison tocommunities where no projects were introduced. Energy saving should also be taken intoconsideration.Citizens should be more satisfied with the quality of services, supported by thecommunity projects, and strive to maintain the established or rehabilitated infrastructurefacilities.As a result of the approach implementation the communities should become moreorganized, which should be manifested by a growing social cohesion within communities.1.4. Research Goal and ObjectivesTo perform a comprehensive impact assessment of the community baseddevelopment approach to local development UNDP/EU Community Based Approach toLocal Development Project commissioned to the Kiev International Institute of Sociologythe study "Evaluation of impact of community based development approach to localdevelopment introduced in the UNDP Projects in Ukraine”.The goal of the study was to evaluate the effectiveness and impact of thecommunity based development approach to local development, which was implementedby four UNDP Projects in Ukraine: Crimea Development and Integration Programme,Chernobyl Recovery and Development Programme, Municipal Governance andSustainable Development Programme and Community Based Approach to LocalDevelopment.Following objectives were identified to achieve this goal:– Examine the views of the target groups representatives about the effectiveness of themethodology of the community based approach;– Determine the views of the target groups representatives about the approach impact onthe local self-government of population;– Identify views of the target group representatives regarding the influence of theapproach on the provision of services in sectors supported by the projects;– Determine the views of the target group representatives about the approach impact onthe quality of life of the target groups.
  17. 17. 17– Identify factors of the approach implementation effectiveness and developrecommendations for dissemination of experience, improvement of approach andpolicy initiatives.1.5. Report structureThis report presents a number of structural parts that cover the methodology,results and conclusions of this evaluation research study.Section 2 of the research methodology explains in detail the areas of evaluation ofthe effectiveness and impact of the community based development approach to localdevelopment. In addition, it provides description of the methods, which were applied in thisevaluation. At the end of this part, the research hypotheses are defined, while clearlyspecifying what results obtained by which methods should indicate the accomplishment orfailure of the expected outcomes of the approach implementation.The section 3 deals with the assessment of the approach methodologyeffectiveness aimed at evaluating the effectiveness of mechanisms of the approachimplementation by the stakeholders at the local level, particularly at the rayon level and atindividual communities.Section 4 highlights the assessment of the approach impact on local self-organisation of the population. It provides comprehensive examination of establishedsupport organisations within the structure of local authorities, their level of transparency,accountability and equality. The quality of strategic planning using the principles of bottom-up planning is verified. The most attention is paid to the citizen-authority relationships,including the accessibility and openness of the authority to dialogue, the attitudes ofcitizens and authorities towards each other, changes in the co-operation between citizensand government and citizen satisfaction with the work of local authorities. Additionally, therole of local businesses in local development processes is clarified.The next section reveals assessment of the approach impact on provision ofservices in the sectors supported by the UNDP Projects, particularly with regard to the costof creation/rehabilitation and provision of services, quality of these services provision andsustainability of the established community infrastructure. In addition, there is a part thatpresents assessment of approach impact on the quality of life of the target groups, i.e. thecommunity members. An integrated assessment of changes in citizens’ lives is carried out:changes in the material conditions of life, in economic conditions, in the state of health,psychological self-feeling and social cohesion.Expert evaluation is presented in a separate section which describes theassessment of experience dissemination, search for success factors of the approach andidentifying the opportunities for improvement of the public policy regarding localdevelopment.The last section presents conclusions regarding evaluation of effectiveness andimpact of the community based development approach to local development, offers somerecommendations on improvement of the approach and public policy regarding localdevelopment.
  18. 18. 18SECTION 2. RESEARCH METHODOLOGY2.1. Areas and components of evaluation of the approach effectiveness and impactAccording to the objectives of this study, the evaluation was carried out in the fourareas: efficiency of the community based approach; the approach’s impact on local self-government; the approach’s impact on provision of services in sectors supported by theseprojects; the approach’s impact on the quality of life of the target groups.In each of the above areas components were defined, and each component wasfurther evaluated by one or more criteria.Efficiency of the community based development approach was evaluated by:1) Involvement of the population, defined by the following parameters:- Patterns of the population participation in the CO;- Level of citizens’ participation in general meetings;- Level of the population’s involvement in the decision-making process (on any issues,including planning);- Level of participation of the CO members in the joint co-financing of the communityprojects);2) Patterns of priorities setting (the degree of CO independence in setting prioritiesfor conducting community projects was verified);And 3) the effectiveness of the co-financing schemes (the effectiveness of thecurrent co-financing scheme was examined).The assessment of the approach’s impact on local self-government included sixcomponents:1) Appropriateness of the support structures established at various levels of localself-government was evaluated by the following criteria:- Relevance, effectiveness and usefulness of the established structures (LDF, CO, OCC,OCRC) for the local development processes and for the cooperation between citizensand local authorities;- The extent to which created structures are used outside UNDP Projects;- Level of the potential viability of the established structures;- The quality of human resources development and mutual learning of the communityleaders and local authorities representatives;- Degree of CO involvement in activities outside the UNDP Projects);2) Level of transparency, accountability and equality is evaluated by the followingindicators:- Transparency and accountability of the CO activities: in decision making, settingpriorities, regular reporting on the use of funds and financial protocols, carrying outtenders, and public audit;- Equality in the access to the benefits among the project target population (communitymembers);3) Quality of strategic planning and bottom-up planning (the quality of strategicplanning, bottom-up planning and application of these strategic plans was examined);4) Access to information (level of accessibility of information about activities of thelocal authorities was measured);
  19. 19. 195) The role of local businesses (motivation of local business to contribute to localdevelopment initiatives through the sponsorship of the CO projects was clarified);6) The citizens/authorities relationships were defined by the following criteria:- changes in the level of accessibility to the government officials and openness todialogue;- changes in the attitudes of citizens towards the authority representatives (includingchanges in the level of trust);- changes in cooperation between communities and authority representatives (qualitativeand quantitative);- Changes in citizens’ satisfaction with the government officials’ performance).The approach’s impact on provision of services in the sectors supported by theUNDP Projects was carried out by assessing four components:1) The cost of creation or rehabilitation of services/or social/or communalinfrastructure (effectiveness was determined with regard to the cost of community projects’implementation by the community organisations);2) The cost of service delivery (assessment of relative changes in the cost ofservices provision before and after community projects implementation);3) The quality of service delivery (changes in the level of satisfaction with theservices provision was measured: energy efficiency, water supply, education (schoolbuses), health care, environmental protection, social sector, carrying out trainings andlevel of energy savings);4) Sustainability of the established social/community infrastructure (the degree ofvalue of the created or rehabilitated social/community infrastructure and the level of itspotential viability);An important area was evaluating the approach impact on the quality of life of thetarget groups by five components:1) Changes in the material conditions of life (relative changes in the conditions oflife, comfort and quality of life, compliance with the UNDP Projects priorities wereexamined);2) Changes in the economic conditions (determined by direct and indirect relativechanges in the economic conditions);3) Changes in the health status (respondents were asked about relative changes oftheir health);4) Changes in psychological self-feeling (in particular, relative changes in personalconfidence level were investigated);5) Changes in social cohesion (relative changes in social cohesion were examined).2.2. Focus group discussions with citizens and local authoritiesFor the successful implementation of such a large and complex evaluative researchstudy it is critically important to gather sociological data using qualitative researchmethods. Taking into account this fact the approach implemented by various stakeholders,to learn about opinions and views, that is most adequate is the method of the focus-groupdiscussion. This method makes it possible to identify a possible range of opinions aboutperception of local development processes, including local self-government, servicedelivery and changes in the quality of life of the citizens. A variety of responses werereceived regarding the details of the community projects implementation and the extent ofbenefits that the citizens obtain. Discussions between participants who express
  20. 20. 20disagreement and opposing views about the same local development processes are alsoof great importance.Conducting focus groups provided a good opportunity for local development forums,which were attended by the representatives of both the communities and the localauthorities. While these forums were held in various regions of Ukraine, the locations wereselected from a range of regions and rayons. Therefore, rayons for the survey wereselected mainly from among those at which local development forums were conductedduring the field stage of the research. Since the forums were held in different rayons indifferent months without some consistent pattern, it is logical to assume that the forumswere randomly happening at the time of the research study and the generated sample isnot biased.In total, 11 focus groups were carried out - 8 for the CBA project and 3 for theMGSDP project. Since the diversity of ideas and fruitful discussions are amongadvantages of this method, the optimal number of participants is in the range of 8-12, theaverage was 10 participants.For the CBA project 1 focus group was held in each of the 8 macro-regions ofUkraine, specific to this project. Regions that, according to the expert opinion of two CBAnational project managers, are similar in nature in project implementation were combinedinto clusters – macro-regions. Individual areas within each macro-region were selected, soas to cover with the research rayons with varying degrees of effectiveness2. Moderatorsselected focus groups participants from the local development forums participants or (ifforums were not held at this time) from the various communities of different rayons, so thatthey represented the greatest number of stakeholders. Representatives of differentcommunities, heads of village councils, representatives of Rayon State Administrations,rayon councils, contact persons from Rayon State Administrations and coordinators fromOblast State Administrations participated in the focus groups. Representatives of localbusinesses had to provide a response to one extended question; therefore they wereinterviewed individually instead of participating in the two-hour focus groups.For the MGSDP project there was developed qualitative contrast sample of cities forconducting focus groups, thereby views of the stakeholders’ representatives from 3 citieswith various levels of success3were received. For partaking in the focus groups municipalcoordinators invited participants by quotas, so that there were both representatives of thevarious communities and representatives of the local authorities (city councils employees,municipal coordinators and employees of the municipal support units).2.3. In-depth interviews with regional expertsTwo other UNDP Projects - CIDP and CRDP – at the time of the research studywere almost completed; therefore, it was difficult to gather key participants to participate inthe focus groups. Thus, the most convenient method for obtaining quality informationabout activities of these projects was to conduct in-depth interviews with the regionalexperts. Stories of the representatives of the local population and local authorities on therealities and specifics of projects implementation are among advantages of this method.2The degree of effectiveness of the project implementation by rayons has been identified by the 7 criteriaapplying the method of expert questionnaire of the oblast project coordinators. For each rayon the resultingindicator was the sum of ratings by the 7 criteria with a maximum of 100 points. The rayon was classified ashaving high effectiveness if its indicator exceeded the median rayon estimate plus the standard deviation.The rayon was classified as having low effectiveness if its indicator was lower than the median rayonestimate minus the standard deviation.3Initially the expert questionnaire survey of project staff was conducted. On the ground of this survey results28 cities were ranked in terms of effectiveness of the project implementation (the 29thcity – Vynnytsia wasnot considered due to its relatively recent participation in the Project in April of 2010). For the focus groups, 1was selected as the most effective city, 1 as the least effective city and 1 city was selected with averageeffectiveness.
  21. 21. 21While potential respondents are geographically dispersed, telephone interview is the mostresource- efficient research method. In total, 27 expert structured in-depth interviews wereconducted over the telephone.For the evaluation of the CIDP project, in total, 12 respondents were interviewed,i.e. by 3 representatives of each of 4 target groups of respondents4. To learn the opinionsand views of different stakeholders interviews were held with community representatives,heads of village councils, representatives of rayon authorities and local UNDP project staff.To evaluate the CRDP project, in total, 12 respondents, out of the three typicalrayons covered by the project,5were interviewed, i.e. by 3 representatives of each of 4target groups of respondents. Interviews with community representatives, heads of villagecouncils, representatives of the rayon authorities and local UNDP project staff wereconducted for this project as well.An evaluation survey of the MGSDP project was conducted in the same 3 cities,which were selected for the focus groups. Expert interviews with the mayors wereconducted in each of the selected cities.2.4. In-depth interviews with the national expertsAt the end of the research study a generalized assessment of the effectiveness andimpact of the UNDP Projects applying the community based development approach tolocal development should be obtained. From this perspective, within the scale of theUNDP Projects, the most knowledgeable and competent subjects generally are nationalexperts: those who have worked or are currently working with these projects either assenior managers or at the national level. Their general conclusions on the effectiveness ofthe use and dissemination of the approach methodology, local self-government, publicpolicy and strategic planning are based on their own long-term management experience.Such opinions are crucial to understanding the systemic patterns of implementation of thecommunity based development approach to local development.To obtain both professional and weighted estimates, 4 internal experts (by 1representative of the top-management from each of the UNDP Projects under evaluation)and 4 external experts (those having experience of working in the project or cooperationwith the project and representing the organisation external to the respective UNDPproject)6were interviewed. In addition, it was decided to interview an internal expertamong the UNDP management, who was the supervisor of all four projects. This helped toidentify general patterns of the approach’s implementation in Ukraine (without the specificsof individual projects). Thus, in total, 9 expert thematic in-depth interviews with nationalexperts were carried out.2.5. Survey of regional expertsSome aspects of the approach can be professionally evaluated precisely by localofficials or employees of UNDP Projects, who have worked at the rayon or regional level.Moreover, they are most knowledgeable about the situation in the respective rayons andregions, and thus, can provide accurate assessment of the relevance of the established4For the CIDP project 4 in-depth interviews in 3 randomly selected rayons of AR Crimea were planned. Fromthe list of rayons provided by the CIDP representatives 3 rayons were selected randomly. To increase theprobability of the quotas realisation, interviews with the representatives of rayon councils were conducted ona first priority basis. In the process of the sample realisation it turned out to be impossible to interviewrepresentatives of some quotas in selected rayons, therefore, by 1 community representative and 1 projectemployee from the 4thrandomly selected rayon were interviewed.5CRDP expert and representative selected 3 rayons together, typical in terms of community projectsimplementation.6Internal experts were selected on the criteria of the maximum experience of participation in the respectiveproject (number of years in project) and the largest scale of conclusions (the highest level of management).External experts were selected by recommendation of internal experts out of the project partners among thenon-governmental organisations or governmental agencies.
  22. 22. 22support structures and citizens/authorities relations. Considering this, municipal, rayon andregional coordinators from the local authorities and CBA project employees wereinterviewed.Standardized telephone survey via a computer is a quite suitable and cost effectivemethod for these interviews. For the assessment of the CBA project 150 respondents7were interviewed, for the MGSDP - 56 respondents8were interviewed using quota sample.The quantitative responses obtained represent the rayon, municipal and regionalcoordinators of the CBA and MGSDP projects, i.e. they provide the grounds to drawconclusions about the views of not only those surveyed, but also all coordinators at 0.05level of significance with an error not exceeding 4%9.2.6. Survey of community membersSince the residents of the respective communities obtain direct benefit from thecommunity projects, the most accurate information on the realities of implementation andassessment of approach impact on the quality of life can be obtained from surveys ofcommunity members. Therefore, to obtain quantitative data that would enable us to drawconclusions regarding the approach’s impact on the target groups, a survey, by personalinterview, was conducted of the community members.For the survey we stratified the communities and from each set of communities weselected the most typical community. We believe that this approach makes it possible toextrapolate respondents’ answers onto all communities of the respective UNDP project10.To ensure a sufficient number of responses 855 respondents were interviewed. Theresponse rate comprised 72%. 111, 112 and 106 respondents were interviewed toevaluate the CIDP, CRDP and MGSDP Projects respectively. Whereas for the CBA, thelargest project by regional coverage and community involvement, 213 respondents wereinterviewed. In total, 542 respondents were interviewed as a main group.In order to examine the projects’ impact, a survey of control groups was planned11:208 respondents for the CBA, 105 respondents for the MGSDP in total, and 3137For survey of experts of the CBA project all 25 oblasts of Ukraine were designated. It was planned tointerview 1 project employee in each oblast (by 1 out of 2 possible), 1 oblast coordinator of OSA or OC (by 1single possible, while in one oblast he refused, it was decided to interview one more rayon coordinator) and4 rayon coordinators (4 out of 8 possible). If there were more potential respondents than it was required bythe sample task, the respondents were selected from the list using the step.8For the survey of experts from the MGSDP project the objective was to interview representatives of all 28cities. For each city the sample task specified 2 representatives of the municipal support unit. Since therespondents also were selected according to the defined step, municipal coordinators were either sampledor skipped in the selection process. If it was impossible to interview the last respondent in any city (due tothe absence at the work place or refuse to participate), the city mayors were interviewed instead.9All statements in this report pertinent to the frequency distributions are based on statistical laws accordingto which, given the implementation of simple random sample, the distribution of variables values andrelationships between variables inherent to the respondents will be observed in the general population (inthis research study - among community members) with certain (here - with 95%) probability and with aspecified error.10The most theoretically feasible scheme of the assessment of the projects’ impact on communities would begenerating two representative samples for each project- the main (for the population that the projects’ impactis targeted towards) and control (for the population not covered by the project) and making twomeasurements - at the beginning and at the end of each project. However, the complete implementation ofthis approach would be too resource-consuming and the ratio of resources for the project itself and for itseffectiveness assessment would be inadequate. Therefore, we used some more economic modification ofthe approach to the projects impact assessment by using control groups, which, nevertheless, allowsevaluation of the most significant project results.11The experts suggested control communities for the survey. Those communities were to a maximumdegree similar to the main ones, in particular, by such parameters as the number of citizens and economicdevelopment, but they should not have participated in the UNDP Projects.
  23. 23. 23respondents in the control group. For the CIDP and CRDP projects surveys of controlgroups were not carried out, however preliminary research studies were conducted thatmay be used as a basis for comparison. Therefore, assuming that initially all variables ofthese communities were similar, all changes in the population’s self-organisation, quality ofservices and quality of life should have been caused by participation in the UNDP project.In the sub-sample for the CBA project the survey was conducted in all 8 macro-regions, i.e. in each macro-region a typical region was selected, in each typical region - atypical rayon, and in every typical rayon - a typical village (in total, 16 villages12). In thesub-sample for the MGSDP 8 typical cities were selected and in each typical city - a typicalcommunity (in total, 8 communities)13.In the sub-samples for the CIDP and CRDP 8 typical rayons were selected and 1typical village was selected in every typical rayon (a total of 8 villages) 14.2.7. Methodology of data analysisTo analyse the data of the focus groups and interviews (qualitative methods)complete transcripts were prepared based on video and audio records. While reading thetranscripts data, participants responses were classified by themes based on theconceptual scheme of the study. Then, for each theme typical responses were singled out;they are presented in this report. Individual citations inserted in this report were selectedfor the illustrative purposes, based on how precisely they illustrate the researchconclusions.12In the sub-sample for the CBA project on the first stage of sampling there 8 macro-regions were identifiedthat are typical for this project implementation, a process similar to that of the focus groups. In each macro-region the national managers selected 2 typical oblasts. In each of the selected oblasts according to therecommendation of experts, i.e. project staff in relevant oblasts, 2 typical rayons were selected, and in eachselected rayon, 1 village was selected for the survey (usually with a typical project for this rayon). For themain sub-sample group of this project 16 villages have been identified. In each village of the main groupaccording to the list of the community organisation members, potential respondents were selected byapplying the step with the randomly generated base number. The step was equal to the number of citizenson the list divided by the number of interviews to be conducted in the given community. The base numberhelped to identify the first respondent on the list. As a result of this procedure, the sample of respondents ineach community was random. In the villages from the control sub-sample group, selection of respondentswas carried out by the route method with randomly assigned base number for the selection of householdsand respondents in the households.13In the MGSDP project the cities involved differ in patterns of project implementation, the 2 most successfuland 2 lest successful were eliminated from the population as extreme cases. Among the rest 24 cities 8typical cities for the survey were randomly assigned – by generating random numbers in the SPSS program.In each of the selected cities experts, i.e. municipal coordinators, determined by 1 community for the survey(usually, those were ACMHs.) For the main sub-sample group of this project, 8 cities/settlements of urbantype have been identified. In every community from the main sub-sample group by the list of communityorganisation members potential respondents were selected randomly according to the abovementionedprocedure. For the control sub-sample group the same experts have suggested to interview 8 communities(usually ACMHs), similar to those involved in the project, but which have not participated in the UNDPProjects. We assume they were not funded according to the model replicated from UNDP Projects. In thecommunities from the control sub-sample group, selection of respondents was carried out by the routemethod with randomly assigned step for the selection of households and respondents in the households.14CIDP project was implemented only in Crimea, therefore 8 typical rayons and typical villages (not the bestand not the worst, rather medium in effectiveness) in the selected rayons were defined by experts –programme managers. As we applied typical sample, for the survey of community members the procedure ofselection of rayons and settlements for CIDP and CRDP was essentially analogues to the procedure for theCBA. – The final decision of selection a rayon and a settlement was made by an internal expert and a KIISrepresentative. In each village, by the list of the community organisation members, potential respondentswere selected randomly according to the abovementioned procedure. For the CRDP project adequateclusters were presented by 4 oblasts where the project was implemented. The CRDP expert together withthe KIIS expert selected 2 typical rayons per each oblast and at each of these rayons 1 village for the survey.In each village sample of respondents in every community was generated randomly according to theabovementioned procedure.
  24. 24. 24The analysis of respondents’ answers in the surveys of regional experts andcommunity members was carried out in a certain sequence. Initially, researchersexamined the frequency distributions of answers to specific questions in the questionnaire;for accuracy of comparisons all distributions of responses were calculated as a percentageof those respondents who answered the relevant question. After this, the statisticallysignificant differences between sub-groups were identified (for example, between differentsocio-demographic categories) 15. All statistically insignificant results were not included inthe argument because they lie within sample error and might be accidental (that is inanother sample they may be different). Therefore, in this report all statements regardingthe revealed differences imply that the differences are not random, but statisticallysignificant. In addition, analysis of statistical relationships between variables was carriedout (for example, between age and improvement of knowledge and skills)16. Therefore,within this report all mentioned relationships between variables are statistically significant.The optimal sequence of analysis is as listed in the following: identifying typicalpatterns of the approach implementation using qualitative methods (the range of possibleviews and assessments) and verifying quantitative distributions of the correspondingopinions of the community members and regional experts (the prevalence of these viewsand assessments among target groups) based on data collected by quantitative methods.In addition, estimates obtained by different methods were compared, for example,responses in surveys of regional experts and community members.2.8. Research operational hypothesesAccording to the objectives identified in the statutory UNDP Projects, which applythe community based development approach to local development, the effectiveness ofthe approach should be demonstrated by the following results17:Efficiency of the community based approach:• Participation of community members in community organisations (the actualparticipation of community members in the activity of communityorganisations18);• Majority of community organisation members at least once participate in generalmeetings (> 80%);• Involvement of the majority of community organisation members (> 80%) in thedecision-making process in community organisations on the basis of consensus,vote or questionnaire;• Extensively informing members of community organisations (> 80%) about theactivities of community organisations;• Solid (100%) payment of membership fees by the members of communityorganisations;15All statements about identified differences suggest that within the 0.95 confidence interval the statisticalhypothesis about percentage or arithmetic means difference can not be rejected. Hypotheses aboutsignificance of the percentage differences were tested by Х2-criterion, and hypotheses about the significanceof mean differences were tested by the Student t-test.16Hypotheses regarding the presence of relationships were checked by the Kendalls tau-b rank correlationcoefficient.17For detailed structure of evaluation see Appendix A.18In the CBA Project the level of participation of community members in community organisations is calculated interms of households against target households in the community. However, this criterion was not anticipated andconsequently not utilised in the survey. According to our data, the share of unique households comprises from 94.6% to100% for samples in different UNDP Projects, 97.7% on average. But as we do not know for sure the exact percent inresponses to different questions, the distributions are calculated for community members, not for households.
  25. 25. 25• Participation of non-members of community organisations in the implementationof community projects through voluntary contributions in the form of money orwork (presence of cases of such participation);• Additional contributions of the members of community organisations on avoluntary basis (presence of cases of additional contributions);• Identifying priorities for community development taking into account interests ofthe majority of community organisation members independently from theauthorities influence (percentage of those who think that the opinion of thecommunity members was more important is larger than the percentage of thosewho believe that the opinion of local authorities was more important);• Majority of stakeholders recognise the co-financing scheme as effective(percentage of those who define it as effective is larger than percentage of thosewho define it as ineffective);• Desire of local stakeholders (not UNDP Projects) to make larger contributionsthan the specified proportion (such a desire is articulated);The approach’s impact on the local self-government:• Established support structures (CO, ACMH, MSU, LDF, MCSD, MCSDF, OCC,OIU, RC) accomplish their objectives for the local development and for thecooperation between citizens and authorities (percentage of those who definethem as effective is greater than percentage of those who characterize them asineffective);• Utilization of the established support structures (CO, ACMH, MSU, LDF, MCSD,MCSDF, OCC, OIU, RC) for dissemination of information materials, approachmethodology and acquired experience to other communities (presence of casesof such dissemination);• Willingness of stakeholders to support established support institutions in thefuture (confidence in the functionality of institutions after completion of the UNDPProjects prevails over the opinion that they will not function);• Accumulation of knowledge and application of skills by the community leadersand local officials (presence of positive dynamics in the competence ofcommunity leaders and local authorities representatives);• Implementation of initiatives by community organisations outside of the UNDPProjects at their own expense (presence of cases of such initiatives);• Participation of community organisations in other competitions outside of theUNDP Projects (presence of cases of such participation);• Significant transparency of activities of community organisations (> 80% of themembers of community organisations are informed about activities);• Target population has equal access to the benefits created by communityprojects (> 80% of the members of the community organisations are confidentthat access is possible);• The interests of rural communities are accounted in the strategic plans of rayondevelopment and interests of urban communities - in the strategic plans of urbandevelopment (presence of cases of development of such plans with theparticipation of communities);• Increase in transparency of the local government activities (presence of apositive dynamics of transparency in the local government activities);
  26. 26. 26• Willingness of local businesses to sponsor local development by makingcontributions to the projects of community organisations (such a desire isarticulated);• Increase in the level of accessibility of local officials and openness to dialogue(presence of the positive dynamics of transparency of local governmentactivities);• Improvement of citizens’ attitudes towards local officials (presence of positivedynamics of citizens’ trust towards local officials);• Increase in cooperation between communities and local authorities (presence ofpositive dynamics of cooperation);• Increase in citizens’ satisfaction with the work of local officials (presence ofpositive dynamics of such satisfaction);The approach’s impact on the service delivery in the sectors supported by theseprojects:• Cost-effective establishment or rehabilitation of services relative to the cost ofthe community projects implementation by community organisations (highereffectiveness in comparison with the projects without community involvement);• Reducing the relative cost of service delivery (the cost of services is smallercompared to the communities where there were no community projects with thisapproach);• High satisfaction with the quality of received services, supported by thecommunity projects (percentage of satisfied respondents is larger than thepercentage of those unsatisfied);• Energy saving (greater savings in comparison with communities where therewere no community projects with this approach);• Members of the community organisations want to support established orrehabilitated infrastructure facilities;The approach’s impact on the quality of life of the target groups:• Relative changes in the quality of life of the community members (presence ofpositive dynamics in the quality of life - material conditions, economic conditions,health status);• Relative changes in the psychological self-feeling of the community members(presence of positive dynamics of self-confidence);• Relative changes in the social cohesion of the communities (presence of positivedynamics in social cohesion).
  27. 27. 27SECTION 3. EVALUATION OF THE APPROACH METHODOLOGYEFFECTIVENESS3.1. Involvement of the populationAnalysis of focus-group discussions demonstrates that the members ofcommunities which joined the project believe in themselves and try to change the life oftheir community.We can distinguish the factors inducing citizens to join the work of communityorganisations including community projects:• Persuasion by informal leaders of public opinion,• Support of local authority representatives,• Awareness of other communities’ positive experience.In-depth interviews and surveys of regional experts allow us to clarify somepeculiarities this process has. According to some representatives of communities and localauthorities, initially, when creating community organisations and launching communityprojects, difficulties occurred with involving local citizens as some of them were scepticaland others could not or did not want to invest their own money into the project. UNDPproject staff and local authorities have made a wide range of efforts to persuade citizens.They organised meetings to explain how community self-organisation should work,showed educational films and short clips on the local television. Resource centres ofdifferent kinds were extremely helpful. Consequently, as citizens saw that communityorganisations really worked and it was possible to implement community projects, theyparticipated in projects more actively.According to the data received by surveying community members, the total numberof citizens holding managerial positions in the board of a community organisation (forinstance, heads, secretaries, treasurers, members of functional groups or monitoringgroups) is 18.7% (14.4 to 20.3% in different UNDP Projects), which is a rather high figureand evidence of democratic management of community organisations.A prerequisite for real involvement of local residents in self-governance is theirparticipation in forms of direct democracy, in particular, in general community meetings.Community member survey findings demonstrate high level of involvement as forUkrainian realities (compared to 50-70% suggested in the initial hypothesis) – overall,general community meetings involve from 88.5% to 96.6% of community members indifferent UNDP Projects, the average proportion being 93.9% among all the respondents.The model approach in fact promotes a participatory form of governance.It is also worth noting that an equally high involvement level is seen both amongmen and women, citizens with different levels of education and those who live in unequallywealthy households. By virtue of their requirements, the projects focus on women’sparticipation and thus they have succeeded in the fulfilment of this requirement.Naturally, a significant proportion of the community organisation took part indecision making concerning community development issues, ranging from 71.2% to 95.5%in different UNDP Projects, the average being 86.8% among all the respondents, whichmakes it an undoubtedly overwhelming majority. This involvement level is nearly equal indifferent social-demographic categories of respondents.The decisions taken are announced to the community, on average 88.5% (78.9 to92% in different UNDP Projects) of community organisation members are informed ofthem. There are no statistically significant differences in awareness levels betweendifferent social-demographic population groups.
  28. 28. 28Graph 3.1.1.Levels of population involvement in decision making(Percentage of the main group, subsample sizes n of responses on each question areindicated in the graph below)For successful microproject realisation, community organisation members have topay membership fees. The survey demonstrates that 95.1% to 100% of the respondentspaid fees with exceptions ranging from 0% to 4.9%. Thus the poll of community membersin various UNDP Projects revealed that 0% to 4.9% of the respondents claim that they donot belong to any community organisations and do not pay membership fees. As thesurvey only listed community organisation members, some explanations are possible.These respondents may have misunderstood the question or may have already left thecommunity organisation. It is also probable that they created informal organisations whichdeveloped into a formal organisation at the settlement level, so the members of theinformal organisations became associated members and it is natural that some of themmight not be paying fees. The rest of respondents invest money regardless of their form ofmembership in a community organisation.Findings of in-depth interviews with regional experts lead to the conclusion that theysupported projects in various ways:• With the help of membership fees,• With additional money investment,• With free voluntary work,• By managing the work of the community organisations.Surveying community members has fully confirmed researchers’ preliminaryhypothesis about the population actually investing amounts of money exceeding thenecessary minimum (all respondents who gave an affirmative answer personallycontributed more than is required from an individual as an additional voluntarycontribution) – from 18% to 36% in different UNDP Projects, the average was around29.9% for all of the respondents. As it could be expected, the number of benefactorsincreases with growing wealth of the household that the respondent lives in. Thus, while23.7% of respondents living in poorer households sometimes invest larger sums thannecessary, the proportion of such respondents from wealthier households is 45.3%.86.893.988.50.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0% respondents, whoparticipated in decisionmaking (n=538)% respondents, whowere informedabout decisions taken(n=539)% respondents, whoparticipated in generalmeetings of communityat least once (n=539)
  29. 29. 29Graph 3.1.2.Occurrence of larger investments than necessary for project realisation(Percentage of the main group, subsample sizes n of responses on each question areindicated in the graph below)As it was repeatedly noted in focus-groups and in-depth interviews, it is morepossible for citizens to express their readiness to provide additional help in the communityproject realisation by doing certain work on community projects than by extra investments.This statement has been fully confirmed by the survey – over a half (from 54.8% to 84.8%in different UNDP Projects, the average being 76.6%) of respondents helped by doingwork on community projects.It would be appropriate to note that there are more male citizens who have donesome labour contribution than female ones (84.8% men compared to 73.5% women). Thenumber of those who provided some help in doing certain work is also somewhat largeramong respondents from ‘wealthy’ households (86.0% compared to 72.9-75.6% ofrespondents from ‘poor’ and ‘medium’ households).3.2. Pattern of priorities settingAccording to focus-group participants (both those representing communities andthose who represented local authorities), setting community priorities could be influencedboth by various interest groups and by heads of village councils, while representatives ofhigher authorities did not have the tools to directly influence community priorities.Regardless of the initial idea source and activity, community decision making wasdemocratic: everyone could voice their suggestions and take part in general meetings –and an overwhelming majority of the community expressed their suggestions andparticipated in general meetings (or at least in the community polls). The democraticprocedure of priorities setting guaranteed that they did reflect urgent needs of the majorityin a community.Similarly, the interviewed regional experts note that community priorities were set bylocal residents themselves by discussing and voting either in general meetings or througha poll. Both community representatives and local authority representatives say thatcommunities set their priorities themselves.According to the concept of local development involving the local community,community organisations are to be independent of local authorities in setting priorities forcommunity project realisation. However, answers to a neutral question presenting variousscenarios of priorities setting that was asked while surveying community membersrevealed the fact that local authority representatives exert indirect influence on community29.976.60.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0% respondents, whosometimescontributed morethan demanded(n=538)% respondents, whocontributedby performing worksfor communityprojects (n=539)
  30. 30. 30priorities setting by announcing their own opinions: from 0.9% to 9.6% of respondentsinvolved in different UNDP Projects (an average of 4.1%) note that authorities’ opinionswere mostly or solely determinative in setting community priorities.This can be partially caused by ignorance and existing stereotypes, as the more therespondents are aware of CO activity, the more they tend to consider that the communityopinion is deciding. It is probable that views on the issue are really connected with citizens’awareness of their community organisation activity – as shown by correlation analysis, themore aware the respondent was, the more likely he was to speak of the deciding role oforganisation members in setting community priorities19.Though the maximum proportion of those who believe that only authority opinionsmattered is 3.8% in one of the UNDP Projects, it remains a point at issue and correspondsto the reality where communities are partly dependent on funding from local budgets. Inspite of this fact, from 58.7% to 71.2% of respondents involved in different UNDP Projects(on average 63.2%) are convinced that deciding priorities setting were mainly or onlybased on community opinions.Graph 3.2.1.Respondents’ opinions of whose view is the most important in setting priorities forcommunity development(Percentage of the main group, subsample size n=538)3.3. Effectiveness of co-financing schemeConsidering statements of focus-group participants, all the interested parties regardthe co-financing scheme as generally good and efficient20. They sometimes suggestreducing bureaucracy, accelerating the process of document registration and funding.Another request is to make the UNDP financial contribution share part bigger as bothcommunities and local authorities are objectively unable to considerably augment their partof the funding.19Kendall tau-b correlation coefficient is 0,264 (p<0,001).20For CBA, the co-financing scheme is as follows – 50% contributes UNDP/EU, 45% – local authorities, 5%– community. For MGSDP: 45%+45%+10% (each year municipal authorities’ part increases by 5%,community part – by 10%). For CRDP the municipal authorities’ part was by 45%, UNDP – by 40%, localbusiness – by 10-15%, community – approximately 5%. For CIDP each party’s contribution was different ineach case.