1
Benchmarking Australian Aid
Results from the 2013 Australian Aid Stakeholder Survey

Stephen Howes and Jonathan Pryke
Deve...
Introduction

3
Why?
• Very hard to benchmark aid effectiveness.
• In many areas, informed judgement indispensable
– cf. aid review submis...
Other stakeholder/perception surveys
• Social accountability
– Citizens’ report cards
– Often aid funded

• In aid
– Multi...
What we asked about
•
•
•
•
•
•
•

Basic information about respondents
The effectiveness of Australian aid
The objectives ...
17 aid challenges
• 17 attributes which are important for aid
effectiveness and/or support.
• Drawn from the 2011 Independ...
Survey design
• Sampling frame for
– Australian NGOs (large and small)
• Target 104 respondents: response rate of 65%

– M...
Respondents
Summary
Phase I NGOs
Phase I Contractors
Phase II (Self-selected)
Grand total

Phase II (self-selected)
Academ...
About the respondents
• 48% female
• Average age: 45
• 79%: strong or very strong knowledge of the aid
program
• 76%: 5 or...
A typical question:
Is Australian aid effective?

100%

5

90%
80%
70%

4
3.7

3.6

60%

3.5

3.5

3.6

3.5

3.6
3.4

3.5
...
Results

12
1. Effectiveness is partly in the eye of
the beholder
100%

5

90%
80%

4

4.1

70%

3.5

60%

3

50%
40%

2

30%
20%

1

...
But views on most aid challenges are quite
similar across levels of perspective
Comparison of views on aid challenges at o...
2. Some disagreement, but more
agreement.
Use of Australian aid to fund advisers
100%
90%
80%

11%

12%

14%

48%

45%

40...
Internal divisions on others
The Australian aid program to sub-Saharan Africa is:
100%
90%
30%

80%
70%

41%

37%

57%

60...
There is more that unites than divides
various aid stakeholders
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1

best

worst

Al...
3. Overall, Australian aid is good and
improving
Responses to survey questions relating to overall aid effectiveness
100%
...
And there is quite a lot we like
Views on sectoral and geographic priorities
100%
90%

21%

14%
27%

24%

30%

80%

46%

7...
Positive comments
• “The increase in funding has impacted on effectiveness in that
there is now much more visibility and n...
4. But there is an unfinished aid reform agenda
Questions about the previous government’s strategy and its
implementation
...
And improvement is needed across the board…
Average responses for the 4 aid attribute categories
5

100%
90%

4

80%
70%

...
… and according to all stakeholder groups
Average score for the 17 aid attributes for different stakeholder groups
5

4

3...
(a) Limiting discretion
100%

5

90%

80%

4

70%
60%

3

3.3

50%

2.8

40%

2.6

2

30%
20%

1

10%
0%

0

Strategic cla...
(b) Enhancing the performance feedback loop
100%

5

90%
80%

4

3.4

70%
60%

3.2

50%

3.1

3
3.0

2.7

2.7

2.6

40%

2...
(c) Building public support
100%

5

90%
80%

4

70%
60%

3

50%

2.7

2.6

40%

2

30%
20%

1

10%
0%

0

Effective commu...
(d) Managing the knowledge burden
100%

5

90%
80%

4

70%
60%
50%

3
3.0
2.6

40%

2

2.3

30%

1.7

20%

1

10%
0%

0

E...
Two (relative) strengths and two
weaknesses stand out
100%

5

90%
80%

4

70%
60%

3.4

3.3

3.2

50%

3.1

3
3.0

3.0
2....
5. More on staff & delays
Staff turnover in AusAID
100%
90%

100%
14%

Very low

80%

90%

Low
45%

100%

6%
10%

12%
5+ y...
Staff continuity/turnover
• Key focus on qualitative comments
• “Turnover of staff in key positions - compromises strength...
Decision-making delays
100%

5

90%
80%

4

Comments
Great
strength
Moderate
strength

70%
60%

3

Neither
strength nor
we...
Transaction costs high and rising
100%

Low

90%
80%

Decreasing

3%

Constant
Medium

70%
60%

11%

39%
51%

50%

Increas...
6. The importance of strategic and
commercial aid objectives
Perceived weight of different aid objectives out of 100
100%
...
We’re realistic, but we’d still like poverty
reduction to be given more weight
Desired weight of different aid objectives ...
Conclusion

35
Summary of results: good but very
improvable
• The aid program is seen to be good and improving
• There is perceived to be...
Implications
• Labor/earlier Coalition Government put a good reform
agenda in place, but didn’t follow through.
• Current ...
Concluding remarks
• There is more that unites than divides the aid
community.
• The aid community needs to do more to mak...
What should and what we thought
would happen to aid volumes
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
What should happen...
Thank you!
www.devpolicy.org
https://devpolicy.crawford.anu.edu.au/australian-aid-stakeholder-survey

40
Upcoming SlideShare
Loading in...5
×

Benchmarking Australian aid: Results from the 2013 Australian aid stakeholder survey

42

Published on

In July and August of 2013 the Development Policy Centre surveyed 356 stakeholders in the Australian aid program. The survey asked them what they thought about the Australian aid program, what they liked, what they didn’t like, and what needed to be done to improve our aid. And now the results are in. This is a recording of the presentation Stephen Howes, lead author of the report and Director of the Development Policy Centre, delivered at the reports launch

Published in: Education, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
42
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Benchmarking Australian aid: Results from the 2013 Australian aid stakeholder survey

  1. 1. 1
  2. 2. Benchmarking Australian Aid Results from the 2013 Australian Aid Stakeholder Survey Stephen Howes and Jonathan Pryke Development Policy Centre Crawford School of Public Policy Australian National University 12 December 2013 2
  3. 3. Introduction 3
  4. 4. Why? • Very hard to benchmark aid effectiveness. • In many areas, informed judgement indispensable – cf. aid review submissions and hearings • Need to overcome the insider/outsider divide. • We should heed the views of those we ask to deliver the aid program. • Good timing! 4
  5. 5. Other stakeholder/perception surveys • Social accountability – Citizens’ report cards – Often aid funded • In aid – Multilateral surveys common – Bilateral surveys less so • This one unique in its focus on aid effectiveness. 5
  6. 6. What we asked about • • • • • • • Basic information about respondents The effectiveness of Australian aid The objectives of Australian aid Sectoral and geographic focus Modes of delivery Aid volumes Questions relating to individual engagement 6
  7. 7. 17 aid challenges • 17 attributes which are important for aid effectiveness and/or support. • Drawn from the 2011 Independent Aid Effectiveness Review. • Divided into four groups – – – – Enhancing the performance feedback loop Managing the knowledge burden Limiting discretion Building public support • Asked about in relation to the aid program or AusAID, and for some at the individual activity level 7
  8. 8. Survey design • Sampling frame for – Australian NGOs (large and small) • Target 104 respondents: response rate of 65% – Major development contractors • Target 44 respondents: response rate of 84% For both groups, we went after senior executives. This was Phase I, from mid-June to August. • Other groups self-selected – This was Phase II, from mid-July to August • Pre-selected more reliable than self-selected, but the degree of commonality across all groups gives credence to the self-selected results. 8
  9. 9. Respondents Summary Phase I NGOs Phase I Contractors Phase II (Self-selected) Grand total Phase II (self-selected) Academia NGO Australian government Developing country government Multilateral or regional organization Development Contractor Consultant Other Total Response rate Responses 68 37 251 356 65% 84% Responses 38 70 55 9 15 25 26 13 251 Proportion 19% 10% 71% 100% Proportion 15% 28% 22% 4% 6% 10% 10% 5% 100% 9
  10. 10. About the respondents • 48% female • Average age: 45 • 79%: strong or very strong knowledge of the aid program • 76%: 5 or more years experience in international development • 80%: directly engaged with the aid program • 77% living in Australia • Self-selected group: younger, more female, more junior, more likely to be overseas, less likely to be directly engaged. 10
  11. 11. A typical question: Is Australian aid effective? 100% 5 90% 80% 70% 4 3.7 3.6 60% 3.5 3.5 3.6 3.5 3.6 3.4 3.5 3 50% 40% 2 30% 20% 1 10% 0% 0 NGO executives Contractor Self-selected executives Very ineffective Effective Academics NGOs (Phase Australian Multilateral Contractors II) government and (Phase II) and developing consultants country govt Ineffective Very effective All Neither effective nor ineffective Overall score 11 Error bars: 95% confidence intervals
  12. 12. Results 12
  13. 13. 1. Effectiveness is partly in the eye of the beholder 100% 5 90% 80% 4 4.1 70% 3.5 60% 3 50% 40% 2 30% 20% 1 10% 0% 0 Aid program Respondent's activities Very ineffective Ineffective Neither effective nor ineffective Effective Very effective Overall score (rhs) 13 Error bars: 95% confidence intervals
  14. 14. But views on most aid challenges are quite similar across levels of perspective Comparison of views on aid challenges at own activity level and program level 5 4 3 2 3.4 3.4 3.4 3.2 2.8 3.0 3.0 2.8 2.7 2.3 1 2.5 2.0 0 Transparency Adequate and Strong monitoring Rigorous evaluation Avoidance of Quick decision predictable funding micromanagement making by AusAID by AusAID Activity Aid program/AusAID 14 Error bars: Range of stakeholder group responses
  15. 15. 2. Some disagreement, but more agreement. Use of Australian aid to fund advisers 100% 90% 80% 11% 12% 14% 48% 45% 40% 41% Self-selected All 27% 26% 70% 60% 50% 61% 40% 30% 63% 20% 10% 12% 0% NGOs Contractors Excessive At about the right level Insufficient 15
  16. 16. Internal divisions on others The Australian aid program to sub-Saharan Africa is: 100% 90% 30% 80% 70% 41% 37% 57% 60% 29% 50% 29% 40% 30% 28% 25% 41% 20% 10% 29% 35% 18% 0% NGOs Contractors Too big The right size Self-selected All Too small 16
  17. 17. There is more that unites than divides various aid stakeholders 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 best worst All NGOs (Ph I) Contractors (Ph I) Self-selected Academics NGOs (Ph II) Australian government Multilateral and developing country govt Contractors (Ph II) & consultants 17
  18. 18. 3. Overall, Australian aid is good and improving Responses to survey questions relating to overall aid effectiveness 100% 5 90% 80% 70% 4 4.1 3.7 3.5 60% 3.3 3.5 3 50% 40% 2 30% 20% 1 10% 0% 0 Own activity Very negative Aid program Negative Relative to ave. OECD donor Neutral Positive Improving? Impact of scale-up Very positive Overal score (rhs) 18 Error bars: range of stakeholder groups
  19. 19. And there is quite a lot we like Views on sectoral and geographic priorities 100% 90% 21% 14% 27% 24% 30% 80% 46% 70% 60% 50% 40% 75% 73% 48% 60% 68% 44% 30% 20% 22% 10% 0% 6% 11% Health Humanitarian and disaster response Too much weight 10% 12% Sustainable economic development Asia Pacific 5% Education Effective governance The right weight Too little weight 19
  20. 20. Positive comments • “The increase in funding has impacted on effectiveness in that there is now much more visibility and need to be accountable to the Australian public.” • “The intent of effectiveness has greatly increased over the past few years, the implementation is still lagging, but it is getting better.” • “Overall I think our aid program has improved over the past few years in reach and effectiveness.” • “Aid effectiveness is improving year by year. There is still a way to go though.” 20
  21. 21. 4. But there is an unfinished aid reform agenda Questions about the previous government’s strategy and its implementation 100% 5 90% 80% 4 70% 3.7 60% 3 3.2 50% 40% 2 30% 20% 1 10% 0% 0 Appropriateness of strategy Very negative Negative Neutral Implementation of strategy Positive Very positive Overall score 21 Error bars: 95% confidence intervals
  22. 22. And improvement is needed across the board… Average responses for the 4 aid attribute categories 5 100% 90% 4 80% 70% 3 60% 50% 2.9 2.8 2.7 2.6 2.4 40% 2 30% 1 20% 10% 0 0% Limiting discretion Feedback loop Great weakness Moderate strength Public support Moderate weakness Great strength Knowledge burden All Neither strength nor weakness Overall score (rhs) 22 Error bars: Range of responses for individual attributes
  23. 23. … and according to all stakeholder groups Average score for the 17 aid attributes for different stakeholder groups 5 4 3 2.9 2 3.1 2.7 2.7 2.5 2.7 2.8 2.3 1 0 NGO execs Contractor Self-selected Academics NGOs (Ph II) Australian Multilateral Contractors execs government and (Ph II) & developing consultants country govt 23 Error bars: Range of responses for individual attributes within that stakeholder group
  24. 24. (a) Limiting discretion 100% 5 90% 80% 4 70% 60% 3 3.3 50% 2.8 40% 2.6 2 30% 20% 1 10% 0% 0 Strategic clarity Predictability of funding Selectivity Great weakness Moderate weakness Neither strength nor weakness Moderate strength Great strength Overall score 24 Error bars: 95% confidence intervals
  25. 25. (b) Enhancing the performance feedback loop 100% 5 90% 80% 4 3.4 70% 60% 3.2 50% 3.1 3 3.0 2.7 2.7 2.6 40% 2.0 2 30% 20% 1 10% 0% 0 Transparency Focus on results Aid Strong performance monitoring management Rigorous evaluation Realism of Appropriate expectations attitude to risk Quick decision making Great weakness Moderate weakness Neither strength nor weakness Moderate strength Great strength Overall score 25 Error bars: 95% confidence intervals
  26. 26. (c) Building public support 100% 5 90% 80% 4 70% 60% 3 50% 2.7 2.6 40% 2 30% 20% 1 10% 0% 0 Effective communication Great weakness Moderate strength Moderate weakness Great strength Political leadership Neither strength nor weakness Overall score 26 Error bars: 95% confidence intervals
  27. 27. (d) Managing the knowledge burden 100% 5 90% 80% 4 70% 60% 50% 3 3.0 2.6 40% 2 2.3 30% 1.7 20% 1 10% 0% 0 Effective use of partnerships Staff expertise Avoidance of micromanagement Staff continuity Great weakness Moderate weakness Neither strength nor weakness Moderate strength Great strength Overall score 27 Error bars: 95% confidence intervals
  28. 28. Two (relative) strengths and two weaknesses stand out 100% 5 90% 80% 4 70% 60% 3.4 3.3 3.2 50% 3.1 3 3.0 3.0 2.8 2.7 2.7 2.7 2.6 2.6 2.6 2.6 40% 2 2.3 2.0 30% 1.7 20% 1 10% 0% 0 Great weakness Moderate weakness Neither strength nor weakness Moderate strength Great strength Overall score 28 Error bars: Range of average responses across stakeholder groups
  29. 29. 5. More on staff & delays Staff turnover in AusAID 100% 90% 100% 14% Very low 80% 90% Low 45% 100% 6% 10% 12% 5+ years 40% 70% 1-2 years 10% Very high 6 months to a year 0% 27% Agree Neither agree nor disagree Disagree 30% Less than 6 months 25% Strongly disagree 10% 10% 0% 0% All 32% 50% 20% 20% 10% 70% 40% 28% 30% 38% 2-5 years 60% High 30% Strongly agree 22% 29% 50% 40% 90% 80% 60% Neither high nor low 50% 20% AusAID manager in place long enough to be effective 80% 70% 60% Time in place for AusAID manager All All 29
  30. 30. Staff continuity/turnover • Key focus on qualitative comments • “Turnover of staff in key positions - compromises strength of relationships and creates negative impact on organisational knowledge.” • “Staff turnover resulting in loss of corporate memory.” • “Staff turnover is one of its biggest weaknesses, as this leads to inefficiencies and confusion.” • “Transaction costs may reduce but are systematically high due to AusAID staff moving positions - previous discussions etc are then lost.” • A long-standing problem • Simons Review (1997): “The [Review] Committee is also concerned about the extent of staff mobility in AusAID. This was raised in many of the submissions received, and during overseas visits. It is far from being a new issue. It was raised in a review of ADAB, a predecessor of AusAID, as far back as 1986 (Fuchs 1986)…” • Hollway Review (2011): “The most consistent feedback the Review Panel received was that AusAID’s effectiveness was undermined by the rapid turnover of staff.” 30
  31. 31. Decision-making delays 100% 5 90% 80% 4 Comments Great strength Moderate strength 70% 60% 3 Neither strength nor weakness 50% 2.5 40% 2 2 30% 20% 1 Moderate weakness Great weakness 10% “Too much turnover in the field and at times a reluctance there to make quick decisions.” “Timeliness and lack of wanting to make a decision.” “…The transaction costs are more timerelated than financial.” “A lot of talk, a lot of documentation, a lot and excessive managerialism that has led to paralysis in decision making” Overall score 0% 0 AusAID (333) Own activity (255) 31
  32. 32. Transaction costs high and rising 100% Low 90% 80% Decreasing 3% Constant Medium 70% 60% 11% 39% 51% 50% Increasing 40% 30% High 58% 20% 10% 38% 0% Level Change 32
  33. 33. 6. The importance of strategic and commercial aid objectives Perceived weight of different aid objectives out of 100 100% 90% 21 16 19 24 21 14 20 17 19 39 42 40 41 41 41 80% 70% 60% 34 39 39 41 42 40 50% 40% 30% 20% 45 44 40 35 39 47 10% 0% NGO executives Contractor Self-selected Academics executives Poverty reduction NGOs Strategic interests Australian Multilateral Contractors government and and developing consultants country govt All Commercial interests 33
  34. 34. We’re realistic, but we’d still like poverty reduction to be given more weight Desired weight of different aid objectives out of 100 100% 8 80% 9 7 9 23 90% 14 22 17 8 12 20 25 28 27 70% 13 25 10 23 60% 50% 40% 75 30% 59 68 74 69 66 59 63 68 20% 10% 0% NGO executives Contractor Self-selected Academics executives Poverty reduction NGOs Australian Multilateral Contractors government and and developing consultants country govt Strategic interests Commercial interests All
  35. 35. Conclusion 35
  36. 36. Summary of results: good but very improvable • The aid program is seen to be good and improving • There is perceived to be an unfinished aid reform agenda. • There are weaknesses apparent across all four sets of aid challenges covered by the survey. • Only 2 of the 17 challenges are seen as strengths by half or more of stakeholders. • 7 are seen as weaknesses by half or more of stakeholders. • The most serious weakness identified is high staff turnover, and the second most is slow decision making. • Advancing the national interest is already seen to be given significant weight as an aid objective; it is perceived to have more weight than poverty reduction as an aid objective, and more weight than it deserves. 36
  37. 37. Implications • Labor/earlier Coalition Government put a good reform agenda in place, but didn’t follow through. • Current time is one of risk for the aid program. • But also opportunity. • Most important message from the survey is the need to redouble efforts on comprehensive aid reform. – This is a bigger challenge than realignment with the national interest. – And bigger than any geographical or sectoral reorientation. – Corporate reform is crucial, but not sufficient. – Broad-based reform is needed. 37
  38. 38. Concluding remarks • There is more that unites than divides the aid community. • The aid community needs to do more to make its voice heard. • This survey provides a great source of benchmarks. • Doing it again in two years time is one way to track progress. • Welcome your comments. 38
  39. 39. What should and what we thought would happen to aid volumes 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% What should happen What would happen under Labor Increase by inflation or less What would happen under Coalition Increase by more than inflation 39
  40. 40. Thank you! www.devpolicy.org https://devpolicy.crawford.anu.edu.au/australian-aid-stakeholder-survey 40
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×