• Save
Complexity-Aware Monitoring: Briefing on USAID’s New Discussion Note
Upcoming SlideShare
Loading in...5
×
 

Complexity-Aware Monitoring: Briefing on USAID’s New Discussion Note

on

  • 410 views

Washington Evaluators Brown Bag ...

Washington Evaluators Brown Bag
by Heather Britt and Melissa Patsalides
February 25, 2014

How can we monitor effectively in a dynamically changing and unpredictable situation? Many monitoring approaches measure the predicted – desired results, planned implementation strategies and forecasted pathways of change – using indicators expected to provide useful information over the life of the project. Complexity-aware monitoring is intended to compliment performance monitoring by tracking the unpredictable. Three principles and five recommended approaches monitor the unforeseen and unforeseeable so that projects can remain relevant and responsive. Recommended approaches include sentinel indicators, stakeholder feedback, process monitoring of impacts, most significant change, and outcome harvesting. Complexity is commonly misconstrued as synonymous with conflict. However, most development contexts contain a mix of simple, complicated and complex aspects. Complex aspects are characterized by interrelationships, non-linear causality, and emergence. Complexity-aware monitoring approaches are useful in a wide variety of programming contexts. USAID’s Complexity-Aware Monitoring Discussion Note is intended to raise questions, stimulate dialogue, and -- most of all -- inspire experimentation. Heather Britt and Melissa Patsalides will discuss complexity-aware monitoring and USAID’s efforts to support experimentation with these approaches in the Agency. Stay tuned for more news on Learning Lab!

The Note can be found on USAID's Learning Lab website: http://usaidlearninglab.org/library/discussion-note-complexity-aware-monitoring.

Heather Britt is a Monitoring & Evaluation Specialist with USAID/PPL’s Program Cycle Service Center. Heather is a skilled M&E capacity builder with experience working with a diversity of development actors including INGOS, UN agencies, foundations, and local organizations. She has been based in the Middle East for the past twelve years and speaks Arabic. Her evaluation work has included programming related to gender and to fragile states. She has a special interest in systems thinking applications, complexity and emerging methods in evaluation.

Melissa Patsalides is the Monitoring and Evaluation Team Leader in USAID’s Office of Learning, Evaluation and Research (LER). This team is responsible for improving the practices of performance monitoring and evaluation within USAID and for encouraging the use of evidence gained from these practices, by providing guidance and tools, building capacity, and supporting an organizational culture that values M&E. Melissa worked with the Peace Corps and a number of NGOs before joining USAID more than 10 years ago. She draws on considerable experience in difficult programming environments as she promotes cutting-edge M&E solutions in the Agency.

Statistics

Views

Total Views
410
Views on SlideShare
345
Embed Views
65

Actions

Likes
0
Downloads
0
Comments
1

4 Embeds 65

http://washingtonevaluators.roundtablelive.org 42
http://www.washingtonevaluators.org 12
http://washingtonevaluators.org 9
http://www.washingtonevaluators.roundtablelive.org 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Complexity-Aware Monitoring: Briefing on USAID’s New Discussion Note Complexity-Aware Monitoring: Briefing on USAID’s New Discussion Note Presentation Transcript

    • Complexity-Aware Monitoring Briefing on USAID’s Discussion Note Heather Britt & Melissa Patsalides Washington Evaluators, 25 February 2014
    • What is Complexity? Certainty / Agreement Matrix Cynefin Framework
    • Complicated versus Complex Complicated • Many interacting elements; central organizing element • Repeatable patterns of cause and effect • Multiple causal pathways Complex • Interrelations: web of diverse elements; no central organizing element. • Unpredictable cause & effect: Patterns visible retrospectively. Can be non-linear. • Emergence: Dynamic and unpredictable. Unknown elements. Patterns discernible retrospectively.
    • Complexity Myth Busters Myth #1 • Complexity is synonymous with conflict. (Closer to the) Truth • Most development contexts include complex aspects. • Terms like “complex environments” can be confusing.
    • Complexity Myth Busters Myth #2 • Everything is complex. (Closer to the) Truth • Projects and their environments most often include a mix of simple, complicated aspects and complex aspects. • Avoid blanket descriptors like “complex projects” and “complex environments”
    • Complexity Myth Busters Myth #3 • Operating effectively in complexity requires expensive technical tools. (Closer to the) Truth • Simple rules and principles steer effective practice in complexity.
    • Operating in Complexity
    • Operating in Complexity Frisbee Flight Simulation • Models based on initial speed, shooter angle, and shooter height • Holding all other factors constant • Can’t account for changes in wind speed and direction, and other factors
    • Operating in Complexity
    • Performance Monitoring & Complexity-Aware Monitoring Performance Monitoring measures the predicted. Results intended by us Pathways of change planned by us Implementation strategies designed by us Indicators Targets
    • Performance Monitoring & Complexity-Aware Monitoring Complexity-Aware Monitoring tracks the unpredictable. Results (beyond those originally intended by us) Factors & actors outside the project Multiple pathways of change & feedback loops Systems qualities Triggers
    • Performance Monitoring & Complexity-Aware Monitoring Performance monitoring & complexity-aware monitoring complement one another. Performance monitoring for simple & complicated aspects of projects and strategies. Complexity-aware monitoring for complex aspects of projects and strategies.
    • Practicing Complexity-Aware Monitoring 3 Principles 5 Approaches
    • Principles of Complexity-Aware Monitoring 1. Synchronize monitoring with the pace of change 2. Attend to performance monitoring’s 3 blind spots a. broader range of outcomes b. alternative causes c. full range of non-linear pathways of contribution 3. Consider relationships, perspectives, and boundaries (3 key systems concepts) a. the structures, processes, and exchanges linking actors and factors within a system b. different perspectives within a system c. what is in and what is outside the system
    • 5 Recommended Approaches 1. Sentinel Indicators 2. Stakeholder Feedback 3. Process Monitoring of Impacts 4. Most Significant Change 5. Outcome Harvesting
    • 5 Recommended Approaches A Sentinel Indicator: • Is a proxy for complex processes that are difficult to study in their entirety – A sentinel species is a proxy for an eco-system – Stock-outs are a proxy of market efficiency • Is easily communicated • Signals the need for further analysis and investigation Why important? • A simple way to monitor complexity
    • 5 Recommended Approaches Stakeholder Feedback • Family of approaches • Privileging perspectives of partners, beneficiaries or those excluded from a project • Seek diversity rather than consensus Why important? • Complexity is diverse • Knowledge of the system is partial and predictability is low.
    • 5 Recommended Approaches Process Monitoring of Impacts • How a result at one level is used to achieve results at the next level (outputs to 1st level results) • Predicted and emergent processes • Between results boxes in a LogFrame Why important? • Bounds areas of complexity most critical to success • 2 blind spots: factors outside the project, feedback loops
    • 5 Recommended Approaches Most Significant Change • Collection and analysis of stories describing the most important project outcomes • Perspectives of different stakeholder groups are represented in the criteria for determining a significant change Why important? • Captures broad range results (intended/unintended, positive/negative). • Makes diverse perspectives on results explicit.
    • 5 Recommended Approaches Outcome Harvesting • Discover results without reference to predetermined objectives, and work backwards to determine the contribution. • Emphasis on verification and describing contribution Why important? • Captures broad range results (intended/unintended, positive/negative).
    • Complexity Champions to Test New Methods LER support
    • Next Steps Work collaboratively to -• Identify complex aspects of activity/project/strategy – Distinguish complex from complicated and simple • Select approach – Suitable to monitoring purpose – Suitable to activity/project/strategy – Suitable for a trial (timing, resources, learning culture) • Build learning circle for trial
    • What LER is Offering LER will provide support for complexity-aware M&E • Are We Working in Complexity? Questionnaire & workshop (February – March) • Decision support – Which approach is best? • Webinars, case examples & resources for approaches • TA for trials of new methods space for innovation. • Support intentional learning at all stages of trials • Complexity-Aware Evaluation Discussion Note (forthcoming)