The document discusses how evaluation systems, like the Michelin Guide, can help development institutions improve results.
[1] The Michelin Guide is a trusted evaluation system that motivates restaurants to constantly monitor and improve quality. [2] Independent and self-evaluation systems play a similar role for development organizations in monitoring progress, identifying issues, and adapting over time. [3] A culture of self-evaluation from project start to finish is essential for success, just as internal monitoring incentivizes quality control for Michelin-rated restaurants.
Evaluation for Results-Oriented Management (or, What Can Development Stakeholders Learn from the Michelin Guide?)
1. evaluations that matter
Evaluation for
Results-oriented Management
Caroline Heider, DIRECTOR GENERAL, INDEPENDENT EVALUATION GROUP
02.20.2017
2.
3. Evaluation for Results
3
What a nerdy subject, what a contentious field… For
some, what we do as evaluators seems peripheral, if
not irrelevant. But in reality, we are all evaluators, and
we evaluate all the time.
To explain how and why, let’s turn to something that
we all love. Food!
4.
5. Guide Michelin
5
We all recognize the Guide Michelin as
a mark of excellence when it comes to
the world of food.
As a consumer, a patron, just reflect: when and how
do you use the Guide Michelin?
• You rely on it for the credibility of its stars.
• You trust it when you want a special meal for a
wonderful occasion.
• You know you will have a fantastic evening at any
Michelin star restaurant…
6. Guide Michelin
6
Now think of it from the perspective of
the chef, the staff, the restaurant:
• The stress of the assessment
• The thrill of competition
• The motivation and care to pay attention to every
detail.
7. At its heart, the Guide Michelin is an
evaluation system…
7
• Customers value the system: they know what it means and they rely on its
credibility. And: they know what they want, and it is more than a plate
with food in front of their nose!
• The restaurant, chefs and staff – know what it means. They fear it but also
like the challenge of achieving a Michelin star, and it motivates them to
pay attention – to monitor and self-evaluate – everything from the quality
of the food that’s bought, to the way it is prepared and served, to the
attention that guests receive throughout the experience, and the décor of
the restaurant. It is not just a plate of food that is served, it is much more!
8. So what can development institutions
like the World Bank Group
or other organizations
take from the Guide Michelin?
10. So what can development institutions
like the World Bank Group
or the Central Bank of France
take from the Guide Michelin?
This is what
independent
evaluation is
about.
11. Getting and sustaining results requires that we have
internal systems to monitor, measure, assess what
we do, what the results are, where the bottlenecks
lie and constantly learn and adapt. And that is what
monitoring, self-evaluation, and independent
evaluation is about.
12. Things we need to deliver as evaluators
Close an information gap
Provide a credible source of
information
Deliver timely insights
There are more…
13. Our clients have role to play as well…
Openness and receptivity among
colleagues
Active engagement to discuss
A willingness to take a step back to
gain new insights
14. Our shared goal
To serve our clients better
Evaluators
The organizations
and projects we
evaluate
16. What can we in development learn from
Guide Michelin?
It is important that the system works, because it incentivizes
internal monitoring and evaluation systems. The maître d’, who
monitors that the waiters have laid the tables, the sous-chef
who makes sure the kitchen help has done whatever
preparations are needed for the day. And, the chef who samples
the food, evaluates whether it meets expectations and serves
the reputation of his/her establishment. All contribute to
making the system function and deliver the right results.
17. The same is true for the development
community
It is essential that there is a culture and a system to pay
attention, observe, track and record progress, fix problems as
they arise, and step back at the end and reflect on what we
have created. Independent evaluators play a key role but they
cannot deliver on all of that. On the contrary: this kind of
reflection and care for ongoing operations comes out of a
commitment to quality and delivery of results. Our evidence
shows that a functioning self-evaluation – from start to finish –
is an essential factor of success.
What a nerdy subject, what a contentious field… For some, what we do as evaluators seems peripheral, if not irrelevant. But in reality, we all/all of you here are evaluators, and we evaluate all the time.
Before I explain how and why, allow me talk about something else. Food!
Who in the room is NOT familiar with the Guide Michelin?
We all recognize the Guide Michelin as a mark of excellence when it comes to the world of food.
As a consumer, a patron, just reflect: when and how do you use the Guide Michelin?
You rely on it for the credibility of its stars.
You trust it when you want a special meal for a wonderful occasion.
You know you will have a fantastic evening at any Michelin star restaurant…
Now think of it from the perspective of the chef, the staff, the restaurant:
The stress of the assessment
The thrill of competition
The motivation and care to pay attention to every detail.
At its heart, the Guide Michelin represents an evaluation system par excellence:
The users value the system: they know what it means and they rely on its credibility. And: they know what they want, and it is more than a plate with food in front of their nose!
The restaurant – chef and all staff – know what it means. They fear it but also like the challenge of achieving a Michelin star, and it motivates them to pay attention – to monitor and self-evaluate – everything from the quality of the food that’s bought, to the way it is prepared and served, to the attention that guests receive throughout the experience, and the décor of the restaurant. It is not just a plate of food that is served, it is much more!
Above all we should think of client satisfaction. We are facing circumstances of ever increasing challenges. Multilateralism, together with democracy, is being put to extraordinary stress tests, from leaders and populations. This means, we will need to demonstrate not just how many loans we approved, or how much infrastructure was built, but who benefitted, or how these investments lead to broad-based growth.
What do we draw from these examples?
Both these evaluations closed an information gap, provided a credible source of information, and were timely. These are things that we as evaluators need to deliver. And there are more.
But, equally important, there was openness and receptivity among colleagues at the World Bank Group to accept that things were not “just perfect” but could improve, and active engagement to discuss and take the new agenda forward. Evaluation can step back, take a different perspective to help you gain insights you would not have in your day-to-day pressures of putting together projects.
But, equally important, there was openness and receptivity among colleagues at the World Bank Group to accept that things were not “just perfect” but could improve, and active engagement to discuss and take the new agenda forward. Evaluation can step back, take a different perspective to help you gain insights you would not have in your day-to-day pressures of putting together projects.