Results measurement

277 views

Published on

Published in: Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
277
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Results measurement

  1. 1. RESULTS  MEASUREMENT  
  2. 2. Proving and Improving Results Proving  Results   Improving  Results   M&E  system   capable  of  credibly   demonstra5ng   programme   results   M&E  system   supports  improved   design  and   implementa5on  of   interven5ons   System  Requirements   ü Well  defined  indicators   ü Robust  measurement  tools   ü Robust  monitoring  plan   ü Baselines   ü Credible  a>ribu5on  strategy   ü Credible  aggrega5on  strategy   ü M&E  system  generates  5mely  and   relevant  informa5on  for  management   ü Knowledge  management  
  3. 3. M&E in a Market Systems Context Market  development  programmes  explicitly  acknowledge   that  systems  are  complex   •  M&E  needs  to  integrate  complexity  into  the  way  it   funcEons,  including  accommodaEng  for:   •  •  •  •  FormulaEng  and  tesEng  assumpEons,  piloEng  intervenEons,   proceeding  by  trial  and  error   Scaling-­‐up  intervenEons  that  work,  modifying  or  dropping   intervenEons  that  do  not  work   Providing  credible  evidence  on  sustainability,  systemic  change  and   poverty  reducEon  
  4. 4. Implications •  TradiEonal  approaches  to  M&E  are  insufficient  in  market   systems  context:   •  Measure  results  only  at  the  middle  and  end  of  the   programme   •  M&E  divorced  from  management  funcEon   •  M&E  system  uses  LogFrame  only   •  “EvaluaEon”  seen  as  an  external  acEvity  only   •  Use  of  Randomised  Control  Trials  
  5. 5. Impact Profiles High-­‐level  Impact   SCALE   SUSTAINABILITY   • Market  development   programmes  will  have   different  impact  profiles   • Can  take  1-­‐3  years  to   deliver  high-­‐level  impact   • Progress  can  be  bumpy!   Time   Project  LifeEme   Market  Development   Direct  Delivery  
  6. 6. RESULTS MANAGEMENT FRAMEWORK       • Theory  of  Change   • Results  Chains   • Monitoring  Plans  
  7. 7. WHAT IS A THEORY OF CHANGE?   Theories  of  Change  posit  the  underlying  causes   that  are  driving  poor  performance  and  resul5ng   limited  benefits  accruing  to  the  poor.       Theories  of  Change  then  provide  an  overview  of   the  systemic  change  needed  to  improve   performance,  with  a  specific  focus  on  how  the   poor  will  benefit  from  improved  performance.  
  8. 8. Monitoring & Results Measurement Cycle Theory  of   Change   1.  Draw  Interven5on   Results  Chain   7.  Report   Results   8.  Knowledge   Management   6.  Data   Analysis   2.  Define   Indicators   3.  Establish   Baseline   5.  Data   Collec5on   4.  Predict   Results  
  9. 9. Results Chains Results  Chains  map  the  systemic  change  process  of  an   intervenEon     •  RCs  are  more  granular  and  flexible  than  LogFrames   •  RCs  are  a  key  design  and  monitoring  tool   •    Best  PracEce:   •  RCs  developed  jointly  by  the  M&E  team  and   intervenEon  team   •  Every  RC  includes  boxes  to  capture  systemic   change:  sustainability,  copying  /  crowding-­‐in  
  10. 10. Results Chains: Basic Structure Impact  on  poor   men  and  women   • Increased  incomes;  more  secure  livelihoods   • Job  crea5on   • Economic  empowerment   Enterprise   performance   • Improved  produc5vity/compe55veness   • E.g.  improved  yields  for  small-­‐holder  farmers   System-­‐level   change   Ac5vi5es   • Improved  linkages  between  system  actors   • Improved  market  access   • Changes  in  formal/informal  rules   • Introduc5on  of  new/improved  interconnected   systems  (e.g.  policy,  inputs,  finance,  ICT)   • Programme  facilita5on  ac5vi5es  
  11. 11. Small-­‐holders  apply  new   prac5ces   Small-­‐holders  access  new   services   Copying  by  non-­‐project   supported  Millers   Millers  increase   incomes   Miller  delivers  embedded   services  to  small-­‐holders   Miller  adopts  new  supply  chain   prac5ces  (incl  performance   incen5ves  and  transparent  pricing)   Project  facilitates  strategic   planning  process  on  new   supply  chain  prac5ces   Project  iden5fies  Millers   willing  to  adopt  new  supply   chain  strategy   IMPACT   ACTIVITIES   Targeted  small-­‐holders   achieve  higher  yields  &  sales   Copying  by  non-­‐targeted   Small-­‐holders   ENTERPRISE   PERFORMANCE   Targeted  small-­‐holders   increase  incomes   SYSTEM  CHANGE   SYSTEMIC  CHANGE  
  12. 12. Establishing Baseline To  assess  change,  you  need  to  establish  a  baseline  for   each  indicator   •  Common  mispercepEon  is  to  conduct  one  big  baseline   study  of  the  target  populaEon  at  the  beginning  of  the   programme   •  “Treatment”  group  is  hard  to  define  at  start     AUribuEon  consideraEons  can  be  difficult  to  understand  at   start   •  SomeEmes  necessary  to  conduct  retrospecEve  baselines   •  •  •  Best  pracEce:  targeted  baselines  conducted  for  each   intervenEon,  based  on  clear  understanding  of  change   process  
  13. 13. Predicting Results •  Predict  results  for  each  box  in  the  Results  Chain:   What  change  do  you  expect?   •  By  when?   •  •  PredicEng  results  helps  with:   Planning  data  collecEon   •  TesEng  the  underlying  logic  and  making     •    assumpEons  explicit   •  Monitoring  progress     •  •  Best  pracEce:  predicted  impact  is  used  to  decide   whether  to  proceed  with  an  intervenEon  or  not  –   intervenEons  that  offer  low  predicted  Value  for   Money  (VFM)  are  dropped  or  modified  
  14. 14. Predicting Results: Rice Example Average  income  increase  by  20%  for  50  Smallholders  during   first  season   Average  yields  increase  by  20%  for  50  Smallholders  during   first  season   50  Smallholders  accept  service  condi5ons     Miller  offers  services  to  200  Smallholders  during  first  season   1  Miller  adopts  new  model   Project  supports  2  Millers  
  15. 15. Applying Degrees of Evidence MSE/Household  level:  May  be  able  to  aUribute   changes  to  project  intervenEon  through   staEsEcal  and  qualitaEve  research   Market  system  level:  Use  qualitaEve  techniques   to  determine  preponderance  of  evidence  on   effecEveness  at  market  level   Triangulate  evidence  from  all  sources    
  16. 16. Summary Check-list •  •  •  •  •  •  •  •  •  M&E  system  is  capable  of  “proving”  and  “improving”  results   Every  intervenEon  has  a  clear  results  chain   Specific  and  measureable  indicators  are  defined  for  each  box   in  the  results  chain   Baselines  are  established  for  each  indicator   Predicted  results  produced  for  each  intervenEon   Every  intervenEon  has  a  measurement  plan,  with   appropriate  aUribuEon  strategy   Data  analysis  is  Emely  and  accessible   Management  decisions  are  made  on  the  basis  of  evidence/ results   Results  are  aggregated  (controlling  for  overlap)  and  reported   in  an  accessible  way  

×