IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Paper session 4 Giel Ton

609 views

Published on

Published in: Business, Economy & Finance
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
609
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Paper session 4 Giel Ton

  1. 1. CAPTURING CHANGES INORGANIZATIONAL SOCIAL CAPITALAN IMPACT EVALUATION OF A SMALL-GRANT INNOVATION FUNDIDS EVENT “Impact, Learning and Innovation: Towards a Research andPractice Agenda for the Future” – 26-27 March 2013 – IDS SussexGiel Ton
  2. 2. WHAT DO WE WANT TO KNOW
  3. 3. Intervention logic of innovation grantAPPROVEDBUSINESS PLANS(PROCESSINGACTITIVIES)INVESTMENTS INBUSINESSOPPORTUNITYORGANISATIONALMANAGEMENT OFBUSINESSOPPORTUNITYBUSINESS PLAN INOPERATIONFINANCIALSUSTAINABILITY OFTHE ORGANISATIONIMPROVEMENT INORGANISATIONALCAPACITIESIMPROVED ACCESSTO FINANCIALSERVICESSELFMANAGEMENTOF THEORGANISATIONMOREINCOME TOMEMBERSBETTERNUTRITIONALSTATUSBETTER SERVICESTO MEMBERSIMPROVED ACCESSTO NATIONAL ANDREGIONALMARKETSTRAINING ONBUSINESSOPPORTUNITYMORE SENSE OFBELONGINGNESSWITH THEORGANISATIONINITIALORGANIZATIONALCAPACITYGOOD QUALITYBUSINESS PLANSEFFICIENTQUALITYCONTROL ANDGRANTALLOCATIONSYSTEM
  4. 4. APPROVEDBUSINESS PLANS(PROCESSINGACTITIVIES)INVESTMENTS INBUSINESSOPPORTUNITYORGANISATIONALMANAGEMENT OFBUSINESSOPPORTUNITYBUSINESS PLAN INOPERATIONFINANCIALSUSTAINABILITY OFTHE ORGANISATIONIMPROVEMENT INORGANISATIONALCAPACITIESIMPROVED ACCESSTO FINANCIALSERVICESSELFMANAGEMENTOF THEORGANISATIONMOREINCOME TOMEMBERSBETTERNUTRITIONALSTATUSBETTER SERVICESTO MEMBERSIMPROVED ACCESSTO NATIONAL ANDREGIONALMARKETSTRAINING ONBUSINESSOPPORTUNITYMORE SENSE OFBELONGINGNESSWITH THEORGANISATIONINITIALORGANIZATIONALCAPACITYGOOD QUALITYBUSINESS PLANSEFFICIENTQUALITYCONTROL ANDGRANTALLOCATIONSYSTEMGENERAL EVALUATION QUESTIONSEXTERNAL EVALUATION OFBUSINESS PLAN IMPACTSCHECK KEYASSUMPTIONCHECKIMPACTCHECKIMPACT?TYPOLOGIESINTERNAL MONITORING OFGRANT IMPLEMENTATION
  5. 5. MIXED-METHODS RESEARCH DESIGN
  6. 6. Key impact pathway B:Business plans improve organisational resilience (org. social capital)Key impact pathway D:Grant investments release constraints in access to financial servicesKey impact pathway C:Business plans lead to increased economic performanceINTERVENTIONKey impact pathway A:Business plans improve capacity to pay organisational costs
  7. 7. INTERVENTION
  8. 8. Key impact pathway B:Business plans improve organisational resilience (org. social capital)Key impact pathway D:Grant investments release constraints in access to financial servicesKey impact pathway C:Business plans lead to increased economic performanceINTERVENTIONKey impact pathway A:Business plans improve capacity to pay for collective actionATTRIBUTION PLAUSIBLE =NET-EFFECT MEASUREMENT FEASIBLE =QUASI-EXPERIMENTAL DESIGNAPPROPIATECONTRIBUTION STORY = NEED FORMONITOIRNG INFORMATION
  9. 9. Data collection for building the impactcontribution story InterventionUse operational data forimmediate outcomes Ultimate outcomesReflect on (existing)secondary data on poverty Intermediate outcomesAppropriate proxies forkey outcome areas ConfigurationsProcess tracingand monitoring ofinterdependencies
  10. 10. Impacts ofinnovation grantson collectivemarketing groups
  11. 11. 1. Core design elements Time-series on key indicators:● Sales: total sales & sales of innovative product● Patrimony● Self-funding of recurrent cost● Access to funds/credit Treatment and comparison group● 30 farmer organizations that are funded● 20 that are not-funded (though eligible)● Stratified random selection Statistical tests and econometrics outcome variables:● Growth rates● Regression with year after implementation as explanatory variable
  12. 12. 2. Contribution story and threats to validity The grant is assumed to be a vital parts of a ‘package’ of causal factorsthat together is sufficient to produce the ultimate outcomes.• E.g. banks show increasing willingness to negotiate with producerorganisations that have some patrimony• E.g. through the investment the access to governmentprocurement (nutritional programmes) is improved but still dependon parti-political dynamics However, it is possible that the grants are not necessary: do you still seetraces of processes (partly) triggered by the intervention• Most likely, the grant is necessary in some configurations withsome type of organizations/markets, but not in all (‘what works forwhom under what conditions?’)• Tackling the threats to validity to the claim of contribution: counterfactualthinking• Can we explain the realization of the outcomes without makingreference to the grant (‘alternative explanations’)?
  13. 13. TOOL FOR MEASURING SOCIALCAPTAL
  14. 14. Qualitative tool: 38 reports with qualitativedescriptions (approx 15 pp/each)
  15. 15. Feeding learning processes
  16. 16. Escuela de líderes – Leadership course
  17. 17. Question 1 Question 2INHERENT TENSIONS IN COLLECTIVEMARKETING THAT NEED ORGANISATIONALMECHANISMS TO RESOLVE THEMThis tensionis present inthe activitieswe realizeHardlypresentNeverpresentWe have managed toresolve it withorganisationalagreements / internalregulationsWe arelooking forways toresolve itWe don’tneed toresolve it1- “Regulating member supply”Members sometimes protest that theorganisation does not buy all their produce?2- “Quality assurance systems”Are there members that try to deliver lowerquality products than is required?3- “Reduce the need for workingcapital”Members demand cash payment instead ofwaiting until the organisations has sold theproduct?4- “Prevention of disloyal behaviour”Are there members that sell part of theirproduce to other buyers though theypromised to sell to the organisation?5- “Ways to distribute profits”Do members accept that the organisationsdoes not distribute all its profits?6- “Differ benefits and services tomembers and non-members”Is there preferential treatment (e.g. price)when buying from members compared withnon-members?7- “Decide on investments andactivities that do not benefit all”Did the organisation projects or investmentsthat are only to the benefit of a sub-group ofmembers of the group?8- “Delegating and supervisingmarketing tasks”Do members accept that others in theorganisation take decisions on prices ofproducts sold without prior consult to theassembly?9- “Legal responsibility in contractsand loans”Do members take responsibility for eventualfines and sanctions related with salecontracts or loans that the board negotiates?10- “Manage political aspirations ofboard and staff”Do members accept that board members orteam staff take party politicalresponsibilities?Quantitativecomparative toolPregunta 1 Pregunta 2TENSIONESINHERENTES A LACOMERCIALIZACIÓN COLECTIVAEstatensión sepresentaen lasactividadesquerealizamoscomoorganizaciónSepresenta muypocono sepresenta nuncahemos logradoresolverlo conacuerdos/arreglos organizativosestamosbuscando laforma deresolverlononecesitamos resolverlo“Regular lacantidad a acopiarde los miembros”“Sistemas degarantía decalidad”“Reducir lanecesidad decapital de trabajo”“Prevenirdeslealtad en lasventas”“Maneras dedistribuir losexcedentes”“Diferenciar losbeneficios yservicios amiembros y nomiembros”“Decidir sobre
  18. 18. Table 3. Weighing factors used for calculating each tension-containment variable (Tx)ScoreQ1xThis tension is presentin their activitiesHardly present NeverpresentThey have managed to resolve 9 6 3ScoreQ2x They are looking for ways toresolve it6 4 2They don’t need to resolve it 3 2 0Tx=ScoreQ1 x *ScoreQ2x
  19. 19. CASE-BASED COMPARATIVEANALYSIS
  20. 20. Conventional statistical tricksTable.. Rotated component matrix on organizational performance indicatorsFactor 1:Economic performanceFactor 2:Organizational sizeFactor 3:Capital intensityPatrimony per member (Ln)-.025 -.006 .979Active members (Ln).189 .874 -.294Organizational Age (Ln).059 .897 .224Sales (Ln).886 .438 -.126Sales per member (Ln).991 -.043 .041Extraction Method: Principal Component Analysis.Rotation Method: Varimax with Kaiser Normalization.
  21. 21. Set-theoretic tricks: fQCA The visual inspection of scatterplots helped to detect anddescribe the diversity in the sector of economic farmerorganizations. To further deepen the analysis, other case-based comparativetechniques are needed to find groupings of organizations withdistinct scores on each of the tensions that compose ourconstruct, as certain organizations may perform functionswhere not all tensions are relevant and for which the absenceis not indicative of weaker capabilities. We expect to explain success with a configurational analysis:like Qualitative Comparative Analysis (Rihoux and Ragin,2008; Schneider and Wagemann, 2012), when we have the‘success-variable’ late 2013.
  22. 22. CHALLENGES AHEAD
  23. 23.  Practical issues:● Time-series: confidentiality and ‘contentiousproperty’● Quasi-experimental design: attrition of thecomparison group Fundamental problems● ‘Fishing’ versus ‘Iterative process to have a closerlook at cases’● ‘Success’ is multifacetal: what is the dependantvariable in fQCA?
  24. 24. Thanks!Sietze VellemaSietze.vellema@wur.nlGiel TonGiel.ton@wur.nl

×