Monitoring and Evaluation of Knowledge Management Simon Hearn, ODI, [email_address] Ewen LeBorgne, IRC International Wa...
Overview
<ul><li>“ It is, in fact, nothing short of a miracle that the modern methods of instruction have not entirely strangled th...
Definitions “ When I use a word, it means just what I choose it to mean — neither more nor less”
Monitoring and evaluation <ul><li>OECD definitions: </li></ul><ul><ul><li>Evaluation : The systematic and objective asses...
Monitoring and evaluation <ul><li>OK, but... Any definition must recognise: </li></ul><ul><ul><li>M&E as universal functio...
Knowledge <ul><li>Objective and subjective </li></ul><ul><li>Individual and society </li></ul><ul><li>Facts and values </l...
Development <ul><li>Often conceptualised as a service industry </li></ul><ul><li>Delivery of even basic services (roads, s...
Challenges
Challenges in M&E of KM4D <ul><li>KM4D does not as yet have a well grounded theory </li></ul><ul><li>Knowledge for develop...
Signposts
1. KM ripple model Hulsebosch et al (2009) Performance improvement Changed practices Knowledge capital Knowledge process-e...
2. The KM Framework Talisayon (2009)
Need a better understanding of what intangibles are Based on Talisayon (2009) Value creation through intangibles Human Ca...
Need a better understanding of knowledge transitions Nonaka and Takeuchi (1995) SECI
Need a better understanding of how knowledge is put to use Graham et al (2001) Knowledge to action cycle
Need a better understanding of organisational factors affecting knowledge use Ramalingam (2005) The RAPID Framework for Kn...
We need to understand the level of complexity Snowden (2002) Cynefin framework
Summary: a range of perspectives <ul><li>Ontological: What world-views are reflected in the initiative and how do we reco...
Your reflections? <ul><li>Do you identify with these signposts? </li></ul><ul><li>What signposts do you use? </li></ul><ul...
Multiple knowledges: M&E as multiple partners
Whole-of-community M&E <ul><ul><li>HOLISTIC SOLUTIONS </li></ul></ul><ul><ul><li>SPECIALISED ADVICE </li></ul></ul><ul><...
Multiple knowledges (Brown 2008) <ul><ul><li>HOLISTIC KNOWLEDGE </li></ul></ul>Focus, vision . <ul><ul><li>SPECIALISED K...
Rejected knowledges <ul><ul><li>HOLISTIC KNOWLEDGE </li></ul></ul>Airy-fairy . <ul><ul><li>SPECIALISED KNOWLEDGE </li></u...
Local knowledge Holistic knowledge Individual knowledge Collective knowledge as a nested set A collaborative system Spec...
Port Pirie: small town with the biggest lead smelter in the world COMMUNITY SPECIALIST ORGANISATION HOLISTIC FOCUS Peopl...
New alliances in Port Pirie COMMUNITY SPECIALISTS ORGANISATIONAL HOLISTIC Outrage, political action Technical skills, adv...
<ul><li>M&E as collective learning </li></ul><ul><li>multiple interests </li></ul><ul><li>multiple knowledges </li></ul><u...
Next steps: - The IKM-E approach - Emergent questions on the horizon
Our approach: Multi-evidence based? <ul><li>Each knowledge community uses different M&E criteria, evidence bases, database...
Our approach: Purposes of conducting M&E <ul><li>Financial accountability </li></ul><ul><li>Operational improvement </li><...
Our approach: KM as collective learning Describe Design Do Develop Initiative <ul><li>Key to nested knowledge cultures: ...
Our approach: critical questioning <ul><li>A series of questions at each step of the way </li></ul><ul><ul><li>Overall, a...
Emergent questions on the horizon <ul><li>How would our approach work in practice? </li></ul><ul><li>Specific methods and ...
What now?
IKM-E + KMIC = IKMEKMIC? <ul><li>Avoiding overlaps... </li></ul><ul><ul><li>Connecting KMIC and IKM (blogs...) </li></ul><...
Additional resources <ul><li>IKM-Emergent website: http://wiki.ikmemergent.net </li></ul><ul><li>The giraffe , Working ...
Thank you!
Upcoming SlideShare
Loading in …5
×

Monitoring And Evaluation Of Knowledge Management Elb

2,535 views

Published on

Presentation from the IKM-Emergent group presenting work on M&E of knowledge management. Presentation given during the KMIC webinar organised by USAID.

0 Comments
3 Likes
Statistics
Notes
no profile picture user

  • Be the first to comment

No Downloads
Views
Total views
2,535
On SlideShare
0
From Embeds
0
Number of Embeds
1,321
Actions
Shares
0
Downloads
26
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • IKM-Emergent Working Group 3: Management of knowledge Our Journey
  • The journey
  • We were encouraged to be as practical as possible in today’s webinar and leave you with some great tools like Outcome Mapping, Appreciative Enquiry, Contribution Analysis, etc etc. But instead what we think would be much more useful is to give you a few signposts that can help you decide how to use whatever tools you decide to use. Because it’s not the tool that is the most useful thing but how it is used and what it is used for. Some tools have these aspects built in to a certain degree, but that is all the more reason to need to make these questions explicit. So we’re going to give you a taste of a few different perspectives on knowledge management in the hope that some of them will resonate with your thinking and help you understand how you can assess you initiatives. We are dealing with diverse world views and different forms of knowledge, power and governance Effective methodologies are needed that hold this diversity We need to add value to understanding, justifying and improving knowledge for development processes While we have no clear answers, there are signposts to the way this could be done.
  • 1. An intervention aimed at facilitating knowledge flow 2. Knowledge capital that is created 3. This leads to changed practices in people, teams or organisations 4. This then leads ultimately to performance improvement
  • Simple framework that can be applied to any situation: intangibles of some kind are used for effective decision making, which leads to some kind of valuable result. This simple frameworks carries with it a strong message that when we M&amp;E KM, we must think through the full value chain, not just the km intervention.
  • Describes the processes that move knowledge between tacit and explicit states. this can help us understand the knowledge processes that are going on in any given intervention and thus enable us to see where and how knowledge is being used.
  • Set within an organisational context Looks at four overlapping dimensions that play a role in defining the success of a knowledge and learning strategy in an organisation Suggests that dealing with the four in an integrated and coherent manner may make strategies more effective – thus gives us a model to evaluate these strategies
  • Four domains of knowledge – all of which are valid within different contexts. Provide common process we can monitor Sense-categorise-respond – everything is given and defined Sense-analyse-respond – interpretation is needed but the skills are trainable probe-sense-respond - act-sense-respond
  • IKM has adopted the concept of multiple knoweldges which can help clarify these things and make them explicit.

    • Monitoring And Evaluation Of Knowledge Management Elb

    1. 1. Monitoring and Evaluation of Knowledge Management Simon Hearn, ODI, [email_address] Ewen LeBorgne, IRC International Water and Sanitation Centre, [email_address] Valerie Brown, Australia National University, [email_address]
    2. 2. Overview
    3. 3. <ul><li>“ It is, in fact, nothing short of a miracle that the modern methods of instruction have not entirely strangled the holy curiosity of inquiry.” </li></ul><ul><li>- Albert Einstein </li></ul>
    4. 4. Definitions “ When I use a word, it means just what I choose it to mean — neither more nor less”
    5. 5. Monitoring and evaluation <ul><li>OECD definitions: </li></ul><ul><ul><li>Evaluation : The systematic and objective assessment of an on-going or completed project, programme or policy, its design, implementation and results. </li></ul></ul><ul><ul><li>Monitoring : A continuing function that uses systematic collection of data on specified indicators to provide indications of the extent of progress and achievement of objectives (abridged) </li></ul></ul>
    6. 6. Monitoring and evaluation <ul><li>OK, but... Any definition must recognise: </li></ul><ul><ul><li>M&E as universal functions, not specialised roles </li></ul></ul><ul><ul><li>Presence of different worldviews </li></ul></ul><ul><ul><li>Validity of evidence from different knowledge domains* </li></ul></ul><ul><ul><li>The ethical basis for the desired social change </li></ul></ul><ul><ul><li>The importance of the unexpected and the intangible </li></ul></ul>
    7. 7. Knowledge <ul><li>Objective and subjective </li></ul><ul><li>Individual and society </li></ul><ul><li>Facts and values </li></ul><ul><li>Tacit and implcit </li></ul><ul><li>E.g. Western scientific conception of knowledge as ‘justified true belief’ vs African concept of Ubuntu </li></ul>
    8. 8. Development <ul><li>Often conceptualised as a service industry </li></ul><ul><li>Delivery of even basic services (roads, sanitation..) requires an understanding of the social, political and economic contexts </li></ul><ul><li>Thus, development is more like a knowledge industry (Powell 2006) </li></ul><ul><li>But development is more than donor aid and we must recognise civic-driven change also </li></ul>
    9. 9. Challenges
    10. 10. Challenges in M&E of KM4D <ul><li>KM4D does not as yet have a well grounded theory </li></ul><ul><li>Knowledge for development practice is still young </li></ul><ul><li>KM4D goes beyond what is labeled ‘KM’ </li></ul><ul><li>Competing ontological and epistemological perspectives (and related knowledge systems) </li></ul><ul><li>Existing reporting frameworks are designed for a service industry rather than a knowledge industry </li></ul><ul><li>There can be no simple cause-effect relationship </li></ul><ul><li>KM initiatives often lack explicit linkages to individual, specialist, organisational or social results </li></ul><ul><li>Knowledge is not static </li></ul><ul><li>Lack of methods for interpreting intangibles </li></ul>
    11. 11. Signposts
    12. 12. 1. KM ripple model Hulsebosch et al (2009) Performance improvement Changed practices Knowledge capital Knowledge process-enhancing activities
    13. 13. 2. The KM Framework Talisayon (2009)
    14. 14. Need a better understanding of what intangibles are Based on Talisayon (2009) Value creation through intangibles Human Capital Structural Capital Relationship Capital Tangible Assets Motivational Factors Cognitive Factors
    15. 15. Need a better understanding of knowledge transitions Nonaka and Takeuchi (1995) SECI
    16. 16. Need a better understanding of how knowledge is put to use Graham et al (2001) Knowledge to action cycle
    17. 17. Need a better understanding of organisational factors affecting knowledge use Ramalingam (2005) The RAPID Framework for Knowledge Strategies
    18. 18. We need to understand the level of complexity Snowden (2002) Cynefin framework
    19. 19. Summary: a range of perspectives <ul><li>Ontological: What world-views are reflected in the initiative and how do we recognise them? </li></ul><ul><li>Epistemological: What are the knowledge domains contributing to M&E and how do they relate? </li></ul><ul><li>Socio-political: Who has a stake in the monitoring process and who has power? How can we monitor these interdependent relationships? </li></ul><ul><li>Methodological: How to choose tools and approaches relevant to the parties and processes involved? </li></ul><ul><li>Operational: How do we organise M&E activities according to each of the knowledge domains? </li></ul>
    20. 20. Your reflections? <ul><li>Do you identify with these signposts? </li></ul><ul><li>What signposts do you use? </li></ul><ul><li>How do you see these models supporting your work? </li></ul>
    21. 21. Multiple knowledges: M&E as multiple partners
    22. 22. Whole-of-community M&E <ul><ul><li>HOLISTIC SOLUTIONS </li></ul></ul><ul><ul><li>SPECIALISED ADVICE </li></ul></ul><ul><ul><li>ORGANISATIONAL SUPPORT </li></ul></ul><ul><ul><li>COMMUNITY INTERESTS </li></ul></ul><ul><ul><li>INDIVIDUAL COMMITMENT </li></ul></ul>
    23. 23. Multiple knowledges (Brown 2008) <ul><ul><li>HOLISTIC KNOWLEDGE </li></ul></ul>Focus, vision . <ul><ul><li>SPECIALISED KNOWLEDGE </li></ul></ul>Environment, Health, Finance… , <ul><ul><li>STRATEGIC KNOWLEDGE </li></ul></ul>Organisational structure, aims <ul><ul><li>LOCAL KNOWLEDGE </li></ul></ul>Shared community event <ul><ul><li>INDIVIDUAL KNOWLEDGE </li></ul></ul>Personal lived experience
    24. 24. Rejected knowledges <ul><ul><li>HOLISTIC KNOWLEDGE </li></ul></ul>Airy-fairy . <ul><ul><li>SPECIALISED KNOWLEDGE </li></ul></ul>Jargon <ul><ul><li>STRATEGIC KNOWLEDGE </li></ul></ul>Self-serving <ul><ul><li>LOCAL KNOWLEDGE </li></ul></ul>Anecdote <ul><ul><li>INDIVIDUAL KNOWLEDGE </li></ul></ul>Biased
    25. 25. Local knowledge Holistic knowledge Individual knowledge Collective knowledge as a nested set A collaborative system Specialist knowledge Organisational knowledge
    26. 26. Port Pirie: small town with the biggest lead smelter in the world COMMUNITY SPECIALIST ORGANISATION HOLISTIC FOCUS People long resigned to risk Health Centre stays aloof Mine muzzles council INDIVIDUAL Children diagnosed with lead KNOWLEDGES STRUCTURE CONDITIONS Fear for future livelihood
    27. 27. New alliances in Port Pirie COMMUNITY SPECIALISTS ORGANISATIONAL HOLISTIC Outrage, political action Technical skills, advocacy Public/private good Children’s well-being I NDIVIDUAL Parent, grandparent
    28. 28. <ul><li>M&E as collective learning </li></ul><ul><li>multiple interests </li></ul><ul><li>multiple knowledges </li></ul><ul><li>collaborative action </li></ul>
    29. 29. Next steps: - The IKM-E approach - Emergent questions on the horizon
    30. 30. Our approach: Multi-evidence based? <ul><li>Each knowledge community uses different M&E criteria, evidence bases, databases for judgments... </li></ul><ul><li>Individuals (experiences) </li></ul><ul><li>Communities (observations) </li></ul><ul><li>Experts (practitioner stories) </li></ul><ul><li>Organisations (monitoring reports as stated) </li></ul><ul><li>Holistic thinkers (ideas, forecasts) </li></ul>
    31. 31. Our approach: Purposes of conducting M&E <ul><li>Financial accountability </li></ul><ul><li>Operational improvement </li></ul><ul><li>Strategic readjustment </li></ul><ul><li>Capacity strengthening </li></ul><ul><li>Contextual understanding </li></ul><ul><li>Deepening understanding (research) </li></ul><ul><li>Self-auditing </li></ul><ul><li>Advocacy </li></ul><ul><li>Sensitisation </li></ul><ul><li>(From I. Guijt’s PhD thesis ‘seeking surprise’) </li></ul>
    32. 32. Our approach: KM as collective learning Describe Design Do Develop Initiative <ul><li>Key to nested knowledge cultures: </li></ul><ul><li>Individual </li></ul><ul><li>(Local) Community </li></ul><ul><li>Specialised </li></ul><ul><li>Organisational (strategic) </li></ul><ul><li>Holistic </li></ul>
    33. 33. Our approach: critical questioning <ul><li>A series of questions at each step of the way </li></ul><ul><ul><li>Overall, a sound questioning practice </li></ul></ul><ul><ul><li>And specifically, a guideline to tailor one’s approach: </li></ul></ul><ul><ul><ul><li>What questions to address? </li></ul></ul></ul><ul><ul><ul><li>Who to involve, in what function? </li></ul></ul></ul><ul><ul><ul><li>What tools and methods to choose? </li></ul></ul></ul><ul><ul><ul><li>What lessons to draw from the approach? </li></ul></ul></ul>
    34. 34. Emergent questions on the horizon <ul><li>How would our approach work in practice? </li></ul><ul><li>Specific methods and metrics to go ‘light’ </li></ul><ul><li>Particularly complexity-focused approaches </li></ul><ul><li>Power vs. collective? </li></ul>
    35. 35. What now?
    36. 36. IKM-E + KMIC = IKMEKMIC? <ul><li>Avoiding overlaps... </li></ul><ul><ul><li>Connecting KMIC and IKM (blogs...) </li></ul></ul><ul><ul><li>Organising another webinar? </li></ul></ul><ul><ul><li>Identifying different models / approaches? </li></ul></ul><ul><li>Having creative leaps... </li></ul><ul><ul><li>Reviewing the IKM papers? </li></ul></ul><ul><ul><li>Expanding parts of this paper? </li></ul></ul><ul><ul><li>Testing the IKM-E framework (later)? </li></ul></ul>
    37. 37. Additional resources <ul><li>IKM-Emergent website: http://wiki.ikmemergent.net </li></ul><ul><li>The giraffe , Working group 3 blog </li></ul><ul><li>Working paper 3: ‘Monitoring and Evaluation in Knowledge Management for Development‘ http://su.pr/5rqp8c </li></ul><ul><li>Background paper: ‘Monitoring and evaluating knowledge management strategies’ http://su.pr/28Q9Yu </li></ul>
    38. 38. Thank you!

    ×