Information about the MSC technique for monitoring and evaluation of programs.

1. THE
MOST SIGNIFICANT CHANGE
(MSC) TECHNIQUE
RARE, 2013 - Assessment Task 2
A dialogical approach to information
gathering for monitoring and evaluation.

2. Group Presentation
Member‟s Profiles
Dorothy - located in Dili, East Timor.
Working in international aid within the security
sector.
Karen - located in Jakarta, Indonesia. Currently
working in international aid within the education
sector on programs funded through Australian Aid.
Carol - located in rural Victoria, Australia. Working
for an NGO within the welfare sector.

3. Presentation Content
1. Description of MSC.
1. Purpose of MSC.
1. Appropriate contexts
and uses of MSC, and
inappropriate contexts
and uses of MSC.
1. Strengths of MSC.
2. Weaknesses of MSC.
3. Example of the MSC
approach.
4. MSC activity for the class.

4. Introduction
• The Most Significant Change technique (MSC), is
located in the monitoring form of evaluative inquiry.
• MSC can be used in a program cycle for monitoring
and for program evaluation.
• MSC collects qualitative data rather than the
traditional quantitative reporting in program
monitoring.

5. 1. What is MSC?
Planning
Implementation
Monitoring
Outputs
Quantitative
Reporting
The Most Significant
Change technique is
a participatory form
of program
monitoring and
evaluation.
Typical
Program
Cycle

6. …What is MSC?
• MSC contributes to program monitoring and
evaluation processes through capturing
participants stories, in their own words.
• The story recording and subsequent hierarchical
analysis assist workers and other stakeholders to
understand the impacts of the program.
• Program improvement and worker learning are
thus enhanced through the MSC technique.

7. …The MSC Story
MSC Time Line
1994: Davies
began to develop
the method
In Bangladesh
1996: Called the
„Evolutionary Approach
to Organizational
Learning‟ (Davies)
1998: Termed
„monitoring without
indicators‟
(Guijt, Arevalo, &
Salsdores)
1999: Also known as
„The Story Approach‟
(Dart)
1999: First used in
Australia, called the
„Most Significant
Change‟ Technique
(MSC) Dart et. al.
2009: Further use in
Australia – The
Ambassador
Project, Whitehorse,
Victoria
2000: MSC
approach internet
discussion group
established
2005: Published
in the American
Journal of
Evaluation (Dart
& Davies)

8. 2. What does the Most Significant
Change (MSC) technique do?
• Uses a concept of „dynamic values inquiry‟.
• Searches for, and deliberates over, the value of
significant program outcomes.
• Directs work toward explicitly valued directions
and away from less valued directions.

9. …Collecting SC Stories
Formulating an open ended question to
collect Significant Change (SC) stories:
1. “Looking back over
the last month…
2. …what do you think
was…
3. ..the most significant…
4. …change…
5. …in the quality of
people’s lives…
6. …in this community?”

10. …The MSC Process
Participant stories
collected
Stories discussed & placed
in domains
SC stories chosen &
reasons stated for choices
Qualitative program
outcomes and impacts
Verification, quantification
& report
MSC
Program
Cycle
Incorporates
feedback loops
within the
project, and/or
organisational
hierarchy.

11. …MSC Valued Directions
Voices of
participant's
Workers hear effects
of work and reflect on
practice
Stages of
participatory
discussion, review
and SC selection
Selection group
reflects on practice
Funders &
managers reflect
on desirable
outcomes
A picture of what
participant's
value, stakeholder‟
s value, and
funder‟s value.
Cycle of
improvement
& learning

12. 3. Where does MSC fit?
Introduction
• The evaluator must always choose approaches
and methods which are the most appropriate for
the situation.
• MSC implementation can be adapted to suit
some situation specific characteristics.
• However, MSC is more useful and better aligned
to certain contexts than others.

13. …Where does MSC fit?
Everywhere? Anywhere?
• MSC has been implemented in developing and
developed countries (e.g. Australia and Solomon
Islands).
• Across the range of organisations and institutions
e.g. government, community groups and businesses.
Therefore, is MSC appropriate for any
country, program, project or institution?

14. …Where does MSC fit?
Contexts
Two elements to consider for appropriateness of
context for MSC for the program, project or
institution are:
• Level of complexity.
• Internal characteristics.

15. …Where does MSC fit?
Context - Level of Complexity
Appropriate Inappropriate
Highly Complex Simple
Multilevel Single level
Multiple Stakeholders Few Stakeholders
Divergent Values Focussed Values
Conflicting Opinions Agreement
Ambiguity Clarity

16. …Where does MSC fit?
Context - Internal Characteristics
Characteristic Appropriate Inappropriate
Culture of
Program,
Project or
Institution
Learning Culture Accountability Culture
Improvement Oriented Success Oriented
Open to New
Approaches
Closed to New Approaches
Understanding
of MSC
Accurate Inaccurate
Realistic Expectations Misguided Expectations
Initial Pilot Attempt Full Implementation

17. …Where does MSC fit?
Context - Internal Characteristics
Characteristic Appropriate Inappropriate
People Leadership Stable Changing Leadership
Senior Personnel
Support
Commitment and
Involvement
No Senior Personnel
Support, Commitment or
Involvement
Resourcing Adequate finances Considers MSC „cheap‟ with
inadequate provision
Knowledgeable
Personnel
Considers MSC „easy‟ -
lacking knowledge
Realistic allocation of
time for all MSC phases
Considers MSC „quick‟ with
unrealistic time allocations

18. …Where does MSC fit?
For anything?
• It is possible to use MSC for planning, reporting,
public relations, communication, monitoring and
evaluation.
But what are the more appropriate uses for MSC?

19. …Where does MSC fit?
Uses
Appropriate Inappropriate
Complementary technique within a
suite of M & E methods
Only technique used for M & E
Focus on extreme experience Focus on average experience
Program Improvements Retrospective evaluation for a
finalised program or project
Reflective Practice
Program Outcomes and Impacts
Fluid & Emerging Outcomes Known Outcomes
Changing Focus Static Focus

20. …Where does MSC fit?
Uses
Appropriate Inappropriate
Capturing Unexpected Change Changes Expected
Attaining Shared Vision Vision Established
Participatory Approach Confidentiality Issues
Evaluation Capacity Development Evaluation Knowledge with Experts
Capturing „voice‟ of beneficiaries Ignoring beneficiaries
Qualitative – Hearts & Minds Quantitative – Statistics & Numbers

21. 4. Strengths of MSC
• Inclusive and participatory at all levels.
• Transparent.
• Reflects a strong story telling tradition:
Common in many non-Western cultures.

22. • Contributes to organisational learning:
Provides opportunities to discuss what the
project was trying to achieve and how it can be
improved.
Can help improve program management.
• Non-threatening:
No right or wrong.
Genuine input from participant‟s perspectives.
…Strengths of MSC

23. • Highlights unexpected changes.
• Provides data on impact and outcomes.
• Highlights social benefits.
…Strengths of MSC

24. 5. Weaknesses of MSC
• Costly:
Time.
Human resources.
• Captures extreme cases – not the average.
• Validity and reliability of stories.

25. • Language – can be a barrier/impediment to the
process.
• Difficulties with domains – “fuzzy”.
…Weaknesses of MSC

26. 6. MSC In Action
Click on the link to see a great example of
MSC in action:
http://www.youtube.com/watch?v=NkuJ69zKScU

27. To engage you further in the MSC technique we
have put together some activities for you to try.
Please refer to the activity section within the
written component of our MSC presentation.
7. MSC Class Activity