Independent Advisory Committee on Development Impact (IACDI) THE STORY SO FAR

Role of IACDI To assure the independence and quality of evaluations at DFID, and to agree a programme of independent evaluations To try to ensure that recommendations and lessons of evaluations are followed up A challenge for a part time committee meeting 3 times a year. But we have a highly expert and talented group, and will see what we can do.

To assure the independence and quality of evaluations at DFID, and to agree a programme of independent evaluations

To try to ensure that recommendations and lessons of evaluations are followed up

A challenge for a part time committee meeting 3 times a year. But we have a highly expert and talented group, and will see what we can do.

IACDI in Practice Discussion and conclusions to date. Scope of IACDI interest – to cover both “self evaluation”, and central evaluation department. Assuring independence, quality and clout. Assuring follow up and lesson learning. Future workplan. DFID evaluation policy. EVD 3 year workplan. Transparency and ways to have influence Publish minutes Annual letter to Secretary of State

Discussion and conclusions to date.

Scope of IACDI interest – to cover both “self evaluation”, and central evaluation department.

Assuring independence, quality and clout.

Assuring follow up and lesson learning.

Future workplan.

DFID evaluation policy.

EVD 3 year workplan.

Transparency and ways to have influence

Publish minutes

Annual letter to Secretary of State

Independent evaluation and Self-evaluation Parallel roles of “independent” evaluation and mainstreamed monitoring and “self-evaluation”. IACDI wishes to be involved with both. Question about how best to set standards and check quality of “self-evaluations” whether of projects or country programmes. QA role for evaluation department?

Parallel roles of “independent” evaluation and mainstreamed monitoring and “self-evaluation”. IACDI wishes to be involved with both.

Question about how best to set standards and check quality of “self-evaluations” whether of projects or country programmes. QA role for evaluation department?

Independence without Isolation: or Influence without Capture Have spent much time at last two IACDI meetings discussing this. Aim is to put structures and systems in place that will help assure independence, impact and effectiveness of evaluation in DFID. Have used international best practice yardsticks to guide us on what is needed. Views on whether we have it right and/or what is missing?

Have spent much time at last two IACDI meetings discussing this.

Aim is to put structures and systems in place that will help assure independence, impact and effectiveness of evaluation in DFID.

Have used international best practice yardsticks to guide us on what is needed. Views on whether we have it right and/or what is missing?

Independence, impact and effectiveness: IACDI Conclusions The creation of IACDI was itself intended to increase the independence and impact of DFID’s evaluation department, but more is needed if IACDI is to be in a position to give the assurances about independence and impact envisaged in our terms of reference.

The creation of IACDI was itself intended to increase the independence and impact of DFID’s evaluation department, but more is needed if IACDI is to be in a position to give the assurances about independence and impact envisaged in our terms of reference.

Short term proposals An agreed Departmental policy on evaluation that meets internationally recognized criteria, and is endorsed by IACDI Evaluation in DFID undertaken outside EVD to be greatly strengthened – with a role for EVD in QA. A written job description and arrangements for performance review – with a formal role for IACDI or its chair – to buttress the independence of the head of EVD Written protocols for unimpeded access by EVD to information in DFID; for rules of engagement with DFID staff in discussing draft reports; for rules for avoiding EVD staff conflicts of interest; and a written policy on disclosure of reports. Head of EVD to make an annual report drawing lessons and common themes from reports completed during the year. DFID Management Board to hold at least one annual discussion of evaluation work

An agreed Departmental policy on evaluation that meets internationally recognized criteria, and is endorsed by IACDI

Evaluation in DFID undertaken outside EVD to be greatly strengthened – with a role for EVD in QA.

A written job description and arrangements for performance review – with a formal role for IACDI or its chair – to buttress the independence of the head of EVD

Written protocols for unimpeded access by EVD to information in DFID; for rules of engagement with DFID staff in discussing draft reports; for rules for avoiding EVD staff conflicts of interest; and a written policy on disclosure of reports.

Head of EVD to make an annual report drawing lessons and common themes from reports completed during the year.

DFID Management Board to hold at least one annual discussion of evaluation work

Proposals to be Implemented over 2 - 3 Years Head of EVD should be accountable to the Permanent Secretary of DFID, and be able to make reports without clearance from the management line The post should have a more senior grade and status to give greater visibility and clout. Need to explore ways to protect the budget for evaluation. IACDI should have a central role in deciding EVD’s budget in future. Changes in modalities (and control of the head of EVD over) staffing for EVD, probably with more staff recruited from outside DFID - over time EVD may need to change the balance towards an increased role for EVD staff and a lesser role for external consultants. Future heads of evaluation should probably be appointed on an understanding precluding employment elsewhere in DFID, and the contract should be for a fixed term, possibly renewable on the advice of IACDI.

Head of EVD should be accountable to the Permanent Secretary of DFID, and be able to make reports without clearance from the management line

The post should have a more senior grade and status to give greater visibility and clout.

Need to explore ways to protect the budget for evaluation. IACDI should have a central role in deciding EVD’s budget in future.

Changes in modalities (and control of the head of EVD over) staffing for EVD, probably with more staff recruited from outside DFID - over time EVD may need to change the balance towards an increased role for EVD staff and a lesser role for external consultants.

Future heads of evaluation should probably be appointed on an understanding precluding employment elsewhere in DFID, and the contract should be for a fixed term, possibly renewable on the advice of IACDI.

Evaluation follow up A number ideas about how to ensure better follow up to evaluations, including…. Making evaluation recommendations more relevant, specific, focused; and ensuring a DFID response to all of them. More outreach about the conclusions of evaluations, bringing outside influence to bear Having a regular report from DFID on follow up to past evaluations. DFID have agreed to this, and also to nominate Directors responsible for follow up to each evaluation, who could meet with IACDI if necessary. Other suggestions welcome…

A number ideas about how to ensure better follow up to evaluations, including….

Making evaluation recommendations more relevant, specific, focused; and ensuring a DFID response to all of them.

More outreach about the conclusions of evaluations, bringing outside influence to bear

Having a regular report from DFID on follow up to past evaluations. DFID have agreed to this, and also to nominate Directors responsible for follow up to each evaluation, who could meet with IACDI if necessary.

Other suggestions welcome…

EvD’s future work programme Have discussed selection criteria Key criteria include Size and/or speed of growth of DFID spend or policy interest Opportunities for joint evaluations Fit with a balanced EvD programme of work, with a mix of in depth and light touch studies.

Have discussed selection criteria

Key criteria include

Size and/or speed of growth of DFID spend or policy interest

Opportunities for joint evaluations

Fit with a balanced EvD programme of work, with a mix of in depth and light touch studies.

Future work programme Will discuss at our October meeting a list of options for inclusion in a rolling 3 year programme of EVD evaluations. Suggestions welcome. Will then consult on this, together with a draft DFID Evaluation Policy, over the winter. Already clear that DFID contributions to multilaterals is one important topic. Another will be DFID’s use of budget support as an aid modality. Need to consider balance between mainstreamed “self evaluations” and EVD “independent” evaluations. Need to leave space for increasing number of evaluations carried out jointly with other agencies

Will discuss at our October meeting a list of options for inclusion in a rolling 3 year programme of EVD evaluations. Suggestions welcome.

Will then consult on this, together with a draft DFID Evaluation Policy, over the winter.

Already clear that DFID contributions to multilaterals is one important topic.

Another will be DFID’s use of budget support as an aid modality.

Need to consider balance between mainstreamed “self evaluations” and EVD “independent” evaluations.

Need to leave space for increasing number of evaluations carried out jointly with other agencies