Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this document? Why not share!

Stories that count. Narrative to monitor and learn

on

  • 2,380 views

Use of narrative (storytelling) in organizations (private and public) to drive learning and monitoring

Use of narrative (storytelling) in organizations (private and public) to drive learning and monitoring

Statistics

Views

Total Views
2,380
Views on SlideShare
2,379
Embed Views
1

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 1

http://www.globalemotion.de 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Stories that count. Narrative to monitor and learn Stories that count. Narrative to monitor and learn Document Transcript

  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez, MBA Graduate. HHL –Leipzig Graduate School of Management
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Foreword This paper started fifteen years ago in the white dot on the image. The thin line that timidly cuts the picture in two is the Pan-American road: at the bottom, eight hours and one hundred and fifty kilometres away, Panama City; at the top, the end of the road. The spot marks a place in Panama and in my own geography. Page 2
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 For two years I was an activist, one half of a two-person NGO. During 1991 and 1992 I devoted most of my time to raise funds for the Franciscan mission of Ipetí, in the Darién jungle. Gadi, my girlfriend during my first University years, had spent one month there in the summer of 1990 and was deeply impressed. She came back with the resolution to return the following summer –and, definitively, not empty-handed, but with the money to build a water tank, a water purifier and the complete piping system for Tortí, a local community. She would need all the help she could get. And there I was. In the long run, we also involved our closest friends and relatives and, ultimately, everyone we ever knew or met. At the beginning, we approached several NGOs. Discontent and frustration was the result. Expensive bureaucracy and suspicious looks was the most we got from mainstream NGOs. So what? We could do it ourselves! (You can do everything when you are twenty). We managed to raise the money for the water system that year –and for an ambulance, the following year. We gave speeches at our Universities, looked for support among the professors, engaged (and fought) our friends, convinced business people, tortured our families, organized rallies and worked in a fast-food restaurant to pay our tickets to the jungle (boy, you should try to deliver a pizza flying in a scooter over the Saturday night Madrid traffic!). We did all sort of things because we believed in what we were doing. We had never believed so strongly. It was a good cause, a pristine message. I am a son of my time, I belong to a generation with neither big ideals nor unfeasible dreams. The first generation that escaped not only the horror but also the memory of the war –that did not have to fight for its liberties and rights. Our parents had recently left the barricades and elected their peers to guide the world to a better place. But discontent and apathy soon arose: that was not what they had fought for. They were the last idealists. A declining illusion was in the air as we grew up cuddled by a sense of comfort and indifference towards a system that had been so costly to build, in lives and hopes. We awakened only to express our disillusionment. We paradoxically distrusted a political class whose message we could not identify with and rejected a market that had cushioned our freedom. Most of us decided to recline; some of us, to try to make a difference. The disenchantment with both the government and the market, on the one hand, and the absence of own goals to fight for, on the other, led us almost inevitably to social activism –if politicians were only capable of dialectic gymnastics we, the privileged youth, should take the relay and speak to the forgotten ones. NGOs offered a space for participation and engagement that political parties were no longer able to deliver. Their ‘outsiders’ attitude spelled an attractive yet exclusive message –those who didn’t embody their closed culture and discourse Page 3
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 were rejected. As a result, my contacts with NGOs were tumultuous, admiring their uncompromised and valuable work and refusing their radicalism. NGOs have dramatically changed in the last decade–and so has my opinion and attitude towards the civil society sector as a whole. My links to NGOs have been erratic (and often distant) during these years. However, although I might not praise some of their methods, I admire their courage, identify with many of their causes and recognize their value: they are a fundamental piece to understand our time. I have recently finished my business management studies. In the classroom I often thought of the benefits for NGOs of the tools I was taught. NGOs have already started to borrow some of the corporate world practices as a result of a changing look towards businesses. But they have to do more. Correspondingly, corporations might be also beginning to discover that they have something to learn from the nonprofit sector. A fruitful dialogue could be just flourishing. Guillermo Idáñez Leipzig, November 2005 Page 4
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Index Fore wor d__2 1. Civil Socie ty__7 1.1. The state of affairs__8 1.2. The big boom__11 1.3. The redefinition of civil society?__12 2. NGOs__15 2.1. We’re not from the Government, but we’re here to represent you__17 2.2. From outsiders to insiders__20 2.3. What they have to offer__22 2.4. Two major challenges: accountability and performance __23 3. Learning__26 3.1. Why it is important for NGOs to learn __26 3.2. By the way, what’s learning?__27 3.3. How do we learn?__29 3.4. Organizational learning__31 3.5. Learning organizations__32 3.6. Opening spaces for learning. Some tools and methods __37 4. Introduction to the ‘Most Significant Change’ technique __43 4.1. MSCT in six words__43 4.2. Monitoring for learning__44 4.3. Participatory monitoring__47 4.4. Narrative with a purpose__52 5. M SCT: origin, hist ory and proces s__54 5.1. Bangladesh, 1994__54 5.2. Indicators-based monitoring is not always enough__55 5.3. Travelling around the world with MSCT__56 5.4. Overview of MSCT process__58 6. M oving forward__64 7. Refere nces __66 Page 5
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 M SCT. User’s manual__70 1. What is the ‘Most Significant Change’ technique? __71 1.1. Definition__71 1.2. The heart__72 2. Implementation steps__73 2.1. Getting started__74 2.1.1. The Champions, ownership and management of MSCT_74 2.1.2. Clarification of purposes__74 2.1.3. Comments on building staff capacity__75 2.2. Defining the domains of change__77 2.2.1. Types of Domains__77 2.2.2. Selection of domains: who and how__78 2.3. Setting the reporting period__79 2.4. Collecting ‘significant-change’ stories__79 2.4.1. Whose stories to collect__80 2.4.2. What information to collect__80 2.4.3. Reporting formats and methods__81 2.4.4. Ethics__82 2.5. Selecting the ‘most significant change’ story__82 2.5.1. Selection of the ‘most significant change story’: who and how__84 2.6. Feeding back the results of the selection process__85 2.6.1. Content and form 2.7. Verifying the most significant stories of change__86 2.8. Conducting meta-monitoring__87 Appendix__89 Story collection formats__90 1. Learning to learn. Australia, 2003__90 2. CARE GoG MSC FORMAT. Ghana, 2004__91 3. Landcare (The Mallee Landcare Support Strategy) __92 4. ADRA FORMAT. Laos, 2004__93 ‘Significant-change’ stories__94 1. CARE GoG. Ghana, 2004__94 2. ADRA. Laos, 2004__96 3. Bougainville: Osi Tanata (NGO) __97 Case study__99 Facilitation Guide for story collection workshop__103 Page 6
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 “The citizen [of the United States] is taught from infancy to rely upon his own exertions to resist the evils and the difficulties of life; he looks upon the social authority with an eye of mistrust and anxiety, and he claims assistance only when he is unable to do without it” Alexis de Tocqueville 1. Civil Society What do registered charities, universities, non-governmental organisations, environmental groups, human rights organizations, community groups, hospitals, women's organisations, family counselling agencies, faith-based organisations, social clubs, professional associations, grassroots development groups, trades unions, self-help groups, social movements, business associations, coalitions and advocacy groups, homeless shelters and sports clubs have in common? They all belong to that crossroads called civil society where the relationship between culture, the market and the state is constantly discussed and redefined. Several actors with multiple purposes meet in an open field. Like many other ‘en vogue’ terms (think of networking or globalization), civil society sports a disputed meaning and lacks a clear-cut definition. Institutions of excellence such as the Centre for the study of Global Governance of the London School of Economics1 or the Centre for Civil Society Studies of the Johns Hopkins Institute for Policy Studies2 periodically proclaim the inadequacy and limitations of existing definitions and strive to construct their own terminology. Academics and practitioners spend hundreds of pages attempting to offer a more accurate and inclusive (or restrictive) definition of civil society. This is not a sterile debate; despite the growing magnitude of the ‘civil society phenomenon’, very few comprehensive studies3 have been conducted so far and none offer conclusive results. An unambiguous definition would delineate the limits and facilitate a precise analysis. But probably it is too artificial, pretentious and self-defeating to try to control quicksand. Civil society is such a blurry term, in part because of the innumerable actors that play in it (and the variety of their activities), but also because the differentiation among the nonprofit sector, the business world 1 “Civil society refers to the set of institutions, organisations and behaviours situated between the state, the business world, and the family”. 2 Civil society is integrated by “private, self-governing, non-profit-distributing voluntary organizations”-what Salamon understands as an ‘operative definition’. 3 “Civil society organizations have long been the lost continent on the social landscape of this world”, denounces Salamon in the 2003 Johns Hopkins University report ‘Global Civil Society. An overview’, which is an example of comprehensiveness. Page 7
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 and the government becomes more difficult every day as a consequence of their complex, interlaced and multilayered relations. These pages don’t intend to produce a new definition of civil society but to humbly build on the current ones and use them to frame the specific topic of this paper. Civil society, as Rooy4 explains, can be many things at the same time. However, for the purpose of this paper and, more specifically, for the brief assessment of the present situation of civil society (mostly in the developed world), I will use a ‘working and open definition’ that focuses on the commonalities of the actors and their intentions: Civil society encompasses private organizations that, although they are not part of the government, are expected to deliver public services and not only to look for profits5. 1.1. The state of affairs In the year 2003, Lester M. Salamon, of Johns Hopkins University’s Institute for Policy Studies, published a study6 on civil society. The review captures in facts and figures a multifaceted and diverse environment. The report depicts, albeit with significant differences among countries, revealing aspects of a powerful sector: 4 - Values and norms : ‘Civil’ is the determinant word. Civil pulls a normative understanding of the concept as the ideal to pursue; a society that is both tolerant and cooperative. - A collective noun : The most common definition: Civil society comprises the voluntary sector, advocacy groups, NGOs, social agents of all sorts, human rights organizations and other non- governmental or non-profit players. - A space for action : In this sense, civil society is not the actors, but the space in which they act. Or, as the UNDP (United Nations Development Program) defines it: “Civil society is, together with the state and the market, one of the three spheres that interface in the making of democratic societies”. - A historical moment : Several authors see civil society as the culmination of a process once a set of prerequisites have been fulfilled: primacy of the individual, shared public space and firm consolidation of human rights, among others. - Anti-hegemony : Civil society emerges as a reaction to a well-established single discourse represented by the multinational corporations. - Antidote to the state : Finally, civil society is seen very often as the antithesis of the state. Because of the lack of confidence in the state –the state should yield less power. And, because, as a consequence of globalization, the legitimacy of the sate becomes less clear. 5 I will also use the terms ‘civil society sector‘ and/or ‘civil society organizations’ (CSO) to refer to the organizations comprised in that definition. 6 “Global Civil Society. An overview”. The research project begun in 1991 initially covering 13 countries and progressively extended to the current 35 (developed world: Australia, Austria, Belgium, Finland, France, Germany, Ireland, Israel, Italy, Japan, Netherlands, Norway, Spain, Sweden, United States and United Kingdom. Developing world: Argentina, Brazil, Colombia, Egypt, Kenya, Mexico, Morocco, Pakistan, Peru, Philippines, South Africa, South Korea, Tanzania and Uganda. Transational: Hungary, Poland, Romania, Slovakia and Czech Republic). It is a comparative, systematic, collaborative, and empirical approach that has as main objectives to: a) document the scope, structure, financing and role of the civil society sector, b) explain why this sector varies from region to region, c) evaluate the impact that the organizations from the civil society sector are having, d) improve awareness, and e) to build capacity. Page 8
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 (1) The muscle of civil society is usually placed on its political and social influence. However, the civil society sector appears as a key economic power as well: It accounted for $1.3 trillion in expenditures during the second half of the nineties, was a major employer (a workforce of almost forty million people –fifty seven per cent of which were paid) and, if it were a country, it would be the seventh strongest economy in the world. The civil society sector as a major economic force The civil society “GDP” (2) The role of the civil society sector is no longer limited to the provision of services7. It is true that services (from education to health care) comprise the biggest piece of the nonprofit organizations activities–but not exclusively. In the last years, advocacy and community building have emerged as major roles as well. Salamon also draws the attention to what he calls ‘expressive functions’: artistic, religious, cultural, ethic, social and recreational functions that all together account for 32% of the total activities of the civil society sector. 7 This aspect will be commented in more detail in the sub-section 1.3. Page 9
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 How they spend their time (3) Finally, the report reveals that revenues no longer come primarily from private philanthropy. Instead, income stems mainly from governments, providing more than one third of the incomes and, over all, memberships’ fees that, alone, represent half of the sources of revenue of civil society organizations. Where the money comes from Page 10
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 1.2. The big boom Civil society organizations (CSOs) have indisputably earned a voice (and a loud one) in our time. Its main players influence public policies and private strategies and attract hundreds of thousands worldwide. By itself, this is remarkable, but when we consider that only a few decades ago the sector had only a sporadic presence, then the pace at which it has grown over the last 30 years and its current importance is simply astonishing. The rapid growth of the sector can be seen, in a way, as the public’s response to equally fast changes in our society –and the government and the market reaction to them. (1) The seventies and eighties witnessed a sharp financial growth in the civil society sector. Nonprofit organizations worldwide benefited from generous donors that saw CSOs as propellers of democracy: ‘The motive given by donors for supporting civil society is essentially that a strong civil society will demand a more democratically accountable and transparent state, and lead to sustainable good governance. In addition, citizen participation is central to the idea of civil society. Thus, civil society brings together both the “good governance” agenda and the concern with participatory approaches to development that became widely accepted in development policy (if not in practice) during the 1980s.’8 (2) In addition to massive donor funding9, CSOs receive the support (or at least the approval) of millions of citizens disappointed with traditional institutions. Disenchantment with businesses and governments adds to a mounting concern for the suffering of others and the indignation for permanent social injustice and environmental degradation. Businesses are condemned for being merciless and increasing social inequalities and governments are perceived as inefficient, non- responsive and unable to tackle the great issues of our times. On the contrary, CSOs appear responsive, effective and rooted in the values of the people. (3) Furthermore, the last decades have seen the decline of the state in many countries and its withdrawal from important spaces of the public sphere in many others. On the one hand, the structural adjustment imposed by the IMF and The World Bank to the developing world has been systematically translated into 8 Clayton, Andrew, Oakley, Peter & Jon Taylor. Page 3, Civil Society Organizations and Service Provision, October, 2000, UNRISD (United Nations Research Institute for Social Development). 9 This trend is certainly coming to an end. The days of generous and ‘ask-no-questions’ donors are over. Today less money is available than a decade ago. On the one hand, firms are spending more on their own CSR programmes and less on CSO. And on the other hand, donors are becoming more demanding, looking for CSO that are consistent with their own values and looking for some kind of return on their contributions. Page 11
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 dramatic reductions of the public expenditure in health care, agricultural and education programmes. On the other hand, and as a consequence of the wide acceptance of the ‘New Public Management’10 as the model for the reform of the public sector, the state is moving away from the provision of innumerable services and contracting them out to the private sector –or not covering those spaces anymore. A myriad of CSOs have been created over past decades to take over those areas abandoned by the state. (4) Finally, the internet and communications technologies such as mobile telephony have exponentially multiplied the networking and action opportunities for organizations and individuals. Activism is easier than ever. Organizations can mobilize their bases, expand the reach of their campaigns and build partnerships in a way impossible to imagine not so long ago. These advances also reach individuals, empowering them to take action and motivating a boom in social activism. 1.3. The redefinition of civil society? The avalanche of demands placed on CSOs has resulted in them assuming a broader scope of activities and stronger responsibilities in multiple faces of the public life. This has opened the door to new opportunities but also to serious challenges. (1) Service functions, especially education and social services, account for most CSOs’ work. It would fill dozens of pages to attempt to list the constellation of services that CSOs provide, ranging from fighting AIDS in Mozambique to training Roma in Bulgaria and empowering rural communities in Guatemala. Yet, despite the enormous differences among services, two common features stand out: first, the vast offering is the response to the identification of specific problems and pursues the amelioration of the suffering of the ill and the deprived. And second, and perhaps more importantly, CSOs have developed and implemented their own particular solutions. Moreover, in the last decade services offered by CSOs have not been limited to what the state no longer does but have scaled up to mainstream activities, in some cases CSOs becoming major partners for governments and private businesses. (2) Nevertheless, a vigorous civil society does not stop at the provision of public services; it goes further to ‘provide citizens with vehicles to exercise private 10 First introduced in the UK, USA and New Zealand, it has quickly expanded worldwide. The ‘NPM’ promotes a significant reduction of the role of the state. It aims at reducing the high levels of public expenditure, increasing efficiency and reducing bureaucracy by assigning a greater role in the provision of services to the private sector. Page 12
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 initiative’11. CSOs face important social expectations ‘to secure guarantees of formal legal, political and civil equality…and to secure the law and institutions that safeguard liberty …’12 Therefore, through the promotion of active participation, the education in citizenship and the supply of effective platforms for the expression and preservation of our most valuable rights, CSOs emerge as ‘value guardians’ to counteract the power of governments and corporations. (3) Besides their role as service providers and ‘value guardians’, CSOs have taken on another two important functions: ‘social capital’ building and advocacy. On the one hand, CSOs represent the adequate forum, some authors believe, to capture the spirit and willingness of the people to engage in social activities that, overtime, sets the basis for social infrastructure. Putnam13 who defines social capital as ‘the collective value of all 'social networks' and the inclinations that arise from these networks to do things for each other’ recognizes in it the essence for maintaining democracy: all sort of organizations generate this kind of capital yet CSOs’s appeal to social purposes and mutual support places them in a pivotal democratic role. On the other hand, policy advocacy has progressively come to the forefront activities of most CSOs: these organizations’ roles in unearthing social and environmental issues to the majority, identifying unaddressed needs14 and denouncing injustices by giving expression to the voiceless, have become a crucial barometer to asses the health of our society. Thus, CSOs play a key role in enhancing democracy. “In general, civil society helps attune governments to their populations and strengthen mechanisms of democracy and accountability15”. However, there is a very thin line between voicing citizenry demands and self-appointing oneself as representative of society. CSOs must evaluate very cautiously their role in democracy as, although they might claim a society-wide support, they lack the kind of legitimacy that only ballots provide. 11 Gary Johns. The NGO Challenge: Whose democracy is it anyway? Gary Johns is an active member of the Australian think tank ‘Institute of Public Affairs’. Before joining the IPA in 1997 he served in the Australian Parliament from 1987 until 1996 and held Parliamentary Secretary posts in Health and Treasury, and was Assistant Minister for Industrial Relations and Special Minister of State until the 1996 election. He is a strong supporter of CSO –though a critical one. http://www.ipa.org.au 12 Idem supra 14 13 Putnam, Robert. ‘Bowling alone: The collapse and revival of American Community’, Simon & Schuster, 2000. 14 Not necessarily and only social and environmental, but also referring to local communities, political issues, ethnic problems… 15 John D. Clark, NGO Unit, World Bank Page 13
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Many factors condition the attributes and the role of civil society, among them, the societal roles that we as citizens want to assign CSOs and, of the utmost importance, the latitude in the planet. CSOs are certainly important figures in the political life, present in almost every public policy decision in the developed world16. However, in many parts of the world, these organizations are the only element of cohesion in a torn-apart reality. Yet, regardless its country-specific weight, it is a fact that civil society worldwide has developed and grown at a break-neck speed in only a few decades, increasing its territory in the process. From an alternative ‘third way’, somewhere between the market and the state, the sector aims at becoming a true third option, at the same level of its counterparts. Civil society is thus expected to expand even further. At the same time, and as a consequence, the ground is moving under its feet. Many opportunities and challenges await CSOs in the coming years. But one thing is sure: They are here to stay and NGOs are leading the pack in that effort, or to say it in the words of M. Edwards, of the Ford Foundation, ‘If civil society were an iceberg, then NGOs would be among the most noticeable of the peaks above the waterline, leaving the great bulk of community groups, informal association, political parties and social networks sitting silently (but not passively) below17’ 16 Another matter is how truly effective their role is when it comes to transforming pressure into real practices and legislation. 17 Cited in SustainAbility’s The 21st Century NGO: In the Market for Change, 2nd edition, 2003 Page 14
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 “Organizations that are primarily defined by their labelling as non- (e.g. nongovernmental, nonprofit) or anti- (e.g. anti-globalization, anti-war) organizations, have a communication challenge to address. Some of them, at least, recognize the need to emphasize more positive, pro- messages.” SustainAbility, a consultancy and think-tank 2. NGOs18 And they are a big peak, indeed. In 1909, there were 176 NGOs operating internationally. Almost a century later, that number had rocketed to 40.000, 36.ooo of which appeared in the last 30 years19 (see chart next page20). The reasons pointed out in the previous chapter for the ‘civil society explosion’ (decades of massive donor funding, enormous public support, redefinition of the role of the state and the internet boom) and the roles that civil society organizations have assumed (service provision, ‘value guardians’, social capital building and advocacy policies) are also applicable to NGOs. Therefore, this chapter focuses neither on the reasons for the rapid growth of NGOs nor on a sound study of their role as service providers. Service provision continues to be fundamental –for the beneficiaries but also for NGOs, as the bedrock for the rest of their activities. Yet this chapter does not revolve exclusively around NGOs and services. Instead, the emphasis is put first on NGOs’ increasing political mode; second, on the fast transformation of the sector; third on the specific strengths of NGOs and, finally, on the challenges that lie ahead. 18 “The term non-governmental organization or NGO was not in general currency before the UN was formed. The first draft of the UN Charter did not make any mention of maintaining co-operation with private bodies. A variety of groups, mainly but not solely from the USA, lobbied to rectify this at the San Francisco conference, which established the UN in 1945…they greatly enhanced the UN's role in economic and social issues and upgraded the status of the Economic and Social Council (ECOSOC) to a "principal organ" of the UN. To clarify matters, new terminology was introduced to cover ECOSOC's relationship with two types of international organizations. Under Article 70, "specialized agencies, established by intergovernmental agreement" could "participate without a vote in its deliberations", while under Article 71 "non-governmental organizations" could have "suitable arrangements for consultation". Thus, "specialized agencies" and "NGOs" became technical UN jargon. Unlike much UN jargon, the term, NGO, passed into popular usage, particularly from the early 1970s onwards.” Willetts, Peter. What is a Non-Governmental Organization? City University, London. 19 Anheier, H., Glasius, M. & Kaldor, M. Global Society 2001, Oxford University Press, Oxford. 20 The first two charts of this section are taken from SustainAbility’s The 21st Century NGO: In the Market for Change, 2nd edition, 2003 –Pages 12 and 37 respectively. Page 15
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 NGOs are here to stay Page 16
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 2.1. We’re not from the Government, but we’re here to represent you “Mike Moore, former Prime Minister of New Zealand and Director- General of the World Trade Organisation from 1999 to 2002 tells the story, ‘All the UN agency heads meet once a year under the chairmanship of Kofi Annan ... at one meeting, an agency head shocked me by stating: “We are in a post-parliamentary, post-democratic age; nation states can’t function any more, politicians are despised and people can’t even be bothered to vote anymore.” He went on to assert that the future of governance was with international organizations in partnership with NGOs representing civil society, bypassing politicians. And of course many NGOs subscribe to and push this theory, it gives them power, status and resources.21’” Reflecting on NGOs’ political activities is important for a number of reasons. First, despite being a fairly new phenomenon, it has a tremendous push as the quotation above illustrates. Second, it represents a qualitative leap forward for NGOs, in many ways signalling their transition from ‘outsiders’ to ‘insiders’. Third, NGOs usually base their legitimacy on their members and supporters and on the kind of activities that they develop. Fourth, the principle that underlies the work of NGOs is not under attack, but there are growing calls for greater transparency, accountability and performance. And, finally, if NGOs are not capable of meeting these demands, they could end up damaging their credibility. Of course, NGOs do provide services: Emergency response, short-term relief and long term rehabilitation; service delivery, such as education, clean water and health care; fund raising; involvement in long-term development projects like the UN ‘Millennium Goals’; southern NGOs’ capacity building22 and many more are among the countless and tireless tasks in which modern NGOs engage. However, being that important (and, more than often, vital for local communities) it is their shift towards consolidating civil society, policy influencing and advocacy roles what places them at the centre of the political debate. Surveys about public trust (see chart next page) consistently show similar results: governments and corporations score very poorly while NGOs enjoy enormous levels of trust from citizens, especially when it comes to sensitive issues such as human rights or the environment. The trust received from the public and 21 Johns, Gary. The NGO Challenge: Whose democracy is it anyway? 22 Bashyam, Leo. The role of Northern Development NGOs (Christian Aid). Bashyam highlights some main areas of activity in which Northern NGOs engage for the development of the South (developing world). However, it can be argued that these areas of activity are not exclusive of Northern NGOs involved in the development of the South but common, at least many of them, to most NGOs. Page 17
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 the absolute commitment of their members and supporters represents NGOs’ solid foundations. The public at all levels trust NGOs, outweighing all other institutions – media, businesses and governments. This broad support emerges from the public’s identification with NGOs’ values and the kind of activities in which NGOs engage. Furthermore, members are usually driven by a set of values that steer their work and provide NGOs with a great dose of ‘raw energy’. On the top of that, people that only a few decades ago would have joined political parties to promote and drive social and political changes, today would rather join NGOs –international NGOs’ membership have grown dramatically in the last ten years. As a result of this internal and external support, NGOs claim legitimacy to represent society and demand participation in public policy. The public prefers NGOs to Governments and firms (1) Civil society as a whole and NGOS specifically, play a very active role in the EU, participating in countless initiatives and proposals. “Take the “dialogue with civil society” that took place during the Convention on the Future of Europe, which last year produced a new draft constitution for the European Union. Sensitive to the charge that it was dominated by Europe's political elite, the convention set up civil-society contact groups, allowing NGOs to air their views. Giuliano Amato, one of the convention's vice-presidents, stressed the importance of “the support of civil society in legitimising the final outcome of the convention's work”. Page 18
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Happily, civil society seemed supportive of the idea of expanding EU power. The working group on citizens and institutions called for “the government of the Union to be in the hands of the commission, which alone was capable of representing the common interests of its citizens.” Five NGOs on this working group were invited to deliver this message directly to the convention23.” These lines from ‘The Economist’ exemplify the position that NGOs have achieved at the European Union (EU) institutions. Amato places CSOs (and, among them NGOs) at the core of the political discussion; Wallstrom, Communications Commissioner, also concedes NGOs a preferred status in the decision-making process of European policies. The EU has incorporated CSOs into its daily agenda. (2) Likewise, NGOs with ‘consultative status’ at the UN (and they are more than three thousand) have gained great power since the reforms of 1996, even to the point that Riggs and Huberty affirm that “by opening-up UN policymaking processes to more and more NGOs Kofi Annan and other UN officials are dramatically reshaping the world organization. The UN is fast becoming an international organization composed of many different constituent bodies, or ‘stakeholders,’ not just member states. NGOs are a driving force behind UN efforts to develop the institutions and policies of what is called ‘global governance24”. Precisely, since 1996, NGOs with ‘consultative status’ attend UN and UN specialized agencies conferences and meetings, are allowed to speak at these meetings and circulate written statements and, more importantly, to participate in the planning sessions and suggest agendas. Yet the EU and UN are not the only ‘fori’ for NGOs political aspirations. In a way, what’s happening in the international scene is the amplification of what’s been going on for years at the national level, where some initially ‘pure’ advocacy organizations are rapidly becoming undoubtedly political -“…a part of civil society wants to change the state, in particular, by redefining democracy from hierarchical representation to community participation25” (see chart next page) 23 The Economist. A rigged dialogue with society, Oct 21st 2004. 24 Riggs, D. and Huberty, R. NGO Accountability: What the US Can Teach the UN. 25 Blood, Robert. Should NGOs be viewed as political corporations? Page 19
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 And what’s next? The EU, UN and many local and international institutions recognise the growing political influence of NGOs and their energetic and crucial role in strengthening democracy. NGOs have proven to be key drivers in democratic transitions and continuously augment civil society by increasing the voices that address the market and the state26. 2.2. From outsiders to insiders The confrontation between NGOs and businesses has been a constant for many years -newspapers are filled with hundreds of stories of NGOs’ attacks on corporations. Furthermore, dialogue between them seemed almost impossible as 26 But the cries from some of the NGO sector for further liberalising democracy and self-electing themselves as community representatives crash against the most basic rules of accountability and representation. Electoral democracy serves the function of providing effective instruments to asses the concerns of the majority –but, unfortunately, it does not reach every group’s demands or needs. NGOs can represent and voice those claims ‘left aside’ probably even in a more successful way than the state. Yet NGOs cannot be raised to the same level of governments. In a democratic society rules have to come from consensus, with explicit reference to the people; and NGOs are subjected neither to the periodic scrutiny of elections nor to deliver what has been compromised with the majority vote–simply put; citizens and businesses have not asked NGOs to regulate their lives. Page 20
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 both corporations and NGOs rejected the practices and methods of the other. However, a new kind of relationship is emerging as a consequence of a changing landscape. (1) Corporations are redefining their language and actions: For the first time, corporations are talking very much like NGOs, are involved in social and environmental actions and provide services that once were exclusive to NGOs – becoming thus real competitors for the nonprofit sector. As a result, NGOs have to pay special attention to issues like positioning, competitive advantage and branding, and, clarify their message to avoid a possible ‘ethical squeeze’. (2) Funds are scarcer today than ten years ago –and shrinking. Donors are more exigent, asking for some sort of ‘return on their investment’ and look for those NGOs that fit into their own objective-oriented values. Besides, Governments no longer give NGOs preferential treatment in outsourcing social services; instead, NGOs have to compete for contracts with corporations. This new situation forces NGOs to take a new approach to funding: On the one hand, they have to fight for appropriate funds and demonstrate that they make good use of them. On the other, NGOs have to look for alternative commercial sources of income such as membership fees, fees for services and products sales. (3) As NGOs become more established, and are under greater pressure, they converge to general principles and look for advantages in the exploitation of businesses’ strategies –they are beginning to ‘recognize that markets are central to their future27’ and to build solid partnerships with firms, once opponents. Thus, there’s a general tendency among mainstream NGOs to adopt corporate models of governance28 and to accept free-market laws as the governing rules. Consequently, a 27 SustainAbility. SustainAbility is a strategy consultancy and independent think tank focused on corporate responsibility and sustainable development. As a consultancy it places a special emphasis in engaging stakeholders, driving best practices and promoting non-financial reporting. SustainAbility nurtures close and constant relationships not only with the corporate world but also –and specially- with NGO’s and public institutions. SustainAbility periodically publishes a report on NGO’s, “The 21st century NGO”–the latest, on December 2003. What makes it different from the previous editions is the massive involvement and participation of different stakeholders –among them, UN and more than 200 people from NGOs interviewed. The report acknowledges the increasing role (and determinant in many cases) that NGO play in our societies. Yet, it also points out how the current landscape is very fast. “The 21st century NGO” main target are the huge branded multinational NGOs –however, everyone in the non-profit and NGO sector can learn important lessons from the recommendations drawn in this paper. Four are the key drivers that will change the role, responsibilities and relationships of NGOs in the coming years, according to SustainAbility: their definitive “integration” in the market economy, globalization, the civil society boom and their own governance. In essence, NGOs will have to accept markets as central to their future, define where they stand –and what they stand for- and revisit some aspects of their internal agenda. 28 Their internal organizations now resemble very much those of private businesses: Leaders are appointed, not elected; Executive power is concentrated in the hands of a professional leadership that runs the organization like a brand; and members and supporters have a very limited saying. One only has to take a quick look at the ‘The Economist’ classifieds to discover that WWF is looking for a Brand Page 21
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 new kind of executive, the social entrepreneur, is taking command of NGOs; he pays attention to market forces (without losing sight of the organizations’ underlying missions) and seeks to use the skills of the business world. “Today, nonprofit boards are expected to govern — to determine the direction of the organization, to make plans and policies, to employ, support, and evaluate the chief executive, to approve budgets and monitor expense, to raise funds and promote the organization’s cause.’29” NGOs are rethinking the way they work, slowly becoming more and more business-like –especially major NGOs, which have undergone an important internationalization process. Professional governance and solid performance are becoming a must. 2.3. What they have to offer Most NGOs are already aware of the need for change –which speaks favourably about their flexibility and capability to adapt- and are in the process of improving their chances of success in public policy and service provision. The assignment is vast; only those NGOs flexible enough will not end against the wall. Only well-resourced NGOs will succeed. (1) NGOs are certainly adaptable and flexible and, generally speaking, willing to take risks and future-oriented –that’s how they have built their strengths and why they are comfortable with change. (2) Expertise is among the core strengths of NGOs. They provide essential knowledge about local conditions and the poor, getting very close to the people and understanding and articulating their demands. This nearness to the field enables them to accurately fine-tune programmes through social assessments and participatory research. Moreover, their capacity-building work serves to empower thousands of communities to organize themselves. Development NGOs grow valuable networks of partnerships with grass-root communities that become priceless in assessing the real needs of the people and sustaining NGOs’ own businesses –R. Napier, CEO of WWF goes as far as to say that “WWF is only as strong as its network30” (3) NGOs very early acknowledged the power of the media and make superb use of it, showing magnificent communication skills –whether to drive campaigns against certain corporations or governments, or to raise the public’s attention over Manager and Greenpeace, for an Information Officer. ‘The times they’re a-changin’. 29 Idem supra 31 30 Cited in SustainAbility’s The 21st Century NGO: In the Market for Change, 2nd ed., 2003. Page 22
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 environmental or social issues. Correspondingly, the media has deposit a great confidence on NGOs and look for advice in their areas of expertise. John D. Clark31 perfectly summarizes NGOs’ contribution to society as a whole: “Governments…need NGOs to help ensure their programmes are effective, well-targeted, socially responsible and well understood. The media, likewise, need NGOs because they trust their local knowledge and alternative perspectives they offer. The public needs them because of their services and mobilising capacity – helping citizens express their voice… Parliamentarians need them for policy guidance, for feedback on what people want and as watchdogs in monitoring public programmes and enhancing the accountability of officials.” It is in these strengths and in the needs that they address, and neither in their presence at international institutions nor in their massive membership base, where NGOs should look for legitimacy. Modern representative democracies support NGOs not for altruistic or ethical reasons but because they are effective. Legitimacy equals usefulness. However, up to this date, it was enough for NGOs to say they were effective. Not any more. The shifting environment and NGOs’ increasing importance imply, among other issues, greater demands for accountability and proven performance. 2.4. Two major challenges: accountability and performance ‘SustainAbility’, a consultancy, offers NGOs’ boards a risk-mapping tool to guide them through expected trouble waters –and to try to transform areas of risk into spots of opportunity. The consultancy identifies two main areas of risk, governance and performance, and four broad issues that NGOs boards will have to address and resolve sooner or later: accountability and transparency (as part of good governance) and funding and standards (as within performance). Yet the consultancy’s is not a sole opinion; the literature (mostly coming from the civil society sector itself) is full with claims pinpointing the weaknesses of NGOs, all converging to the same issues: lack of accountability32, deficient assessment of impact, poor monitoring and evaluation analysis and doubtful effectiveness. 31 NGO Unit, World Bank. Clark, John D. NGOs and the state. Chapter 10.8, The Companion to development studies. 32 A situation that led Smilie, a classical in the development sector, to term accountability the “Achilles’ heel” of the NGO movement. Page 23
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Of course, these are not the only flaws that affect NGOs but they are the ones that, not properly resolved, can have a more devastating impact on NGOs most precious asset: trust, a perishable commodity. Good intentions and sound values are no longer sufficient. Trust is now based on much more mundane things: the responsibility for performing according to agreed-on terms, within a specified time period, and with a stipulated use of resources and performance standards33; and the proficiency in acquiring funds and using these resources efficiently and effectively in achieving specific outcomes. (1) Accountability and performance are strongly linked. The demands for greater accountability come from both the outside (if NGOs are constantly demanding Governments and businesses responsibility for their work, shouldn’t they also be accountable?) and the inside (as a trigger to increase organizational learning and performance). NGOs are gradually abandoning a narrow “principal- agent model34” approach to accountability and moving towards a more open-ended strand of thinking based on the view of organizations as socially constructed entities: accountability is not limited to the adequate use of funds but expands to “carrying out effective, appropriate work which stays true to the needs of the clients and the values of the organization itself35 ”. NGOs are accountable, therefore, to several groups of stakeholders: downward, to their beneficiaries; upward, to donors and governments; inward, to themselves; and horizontally, to their partners and peers. (2) Until fairly recently, there was the common assumption that NGOs were better than governments or businesses at delivering social services, tackling poverty reduction and implementing development projects. That is both a dangerous generalization and a false one. As NGOs begin to be judged by their success and not by their intentions, several concerns arise regarding their effectiveness. Although there are some extraordinary NGOs, many of them suffer severe weaknesses that hamper their performance: lack of professionalism (operating very often without any sense of strategic planning), close culture resulting in domineering leaderships, poor administrative systems, “weak contextual analysis and inadequate monitoring and evaluation systems36”. In brief, NGOs, generally speaking, are bad at performance measurement (at assessing progress toward the achievement of results) and management (at building on that process and learning from it). How 33 www.gse.harvard.edu/hfrp/projects/afterschool/resources/ost_terms.html 34 “Within the context of the non-profit sector, such an understanding leads NGOs to focus on their accountability relationships with donors (upward accountability), to the neglect of other stakeholders such as their beneficiaries (downward accountability)” Lloyd, Robert. The Role of NGO Self-Regulation in Increasing Stakeholder Accountability 35 Lewis, David. NGO: questions of performance and accountability 36 idem supra 39 Page 24
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 are NGOs going to explain their responsibility to all their stakeholders if they show serious difficulties in demonstrating persuasive conclusions about the results of their work and the usefulness of such results –and even the usefulness of the work itself? NGOs have enormous challenges and opportunities facing them. Yet probably the most pressing one is not to ‘integrate’ themselves into the market but to first learn from (and about) themselves. An introspective look at what they’re doing well and, more importantly, at what they’re doing badly and why, is a fine start towards the kind of competitive NGOs we all need. Articulating efficient mechanisms for learning –and becoming better with the experience- is a priority. “Over the long run, superior performance depends on superior learning37” 37 Senge, Peter M. MIT Sloan School of Management. Page 25
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 “The drive to learn is as strong as the sexual drive; it begins earlier and lasts longer” Edward T. Hall, anthropologist 3. Learning 3.1. Why it is important for NGOs to learn The limited communication between the profit and the nonprofit sector delayed until fairly recently the interest of the latter in organizational learning; Major international NGOs like CARE and Oxfam began to orient part of their efforts towards a more effective learning only ten years ago. NGOs’ greater relevance, a progressive shift from project implementation towards a wider scope of tasks, a changing landscape (increasing competition, shrinking budgets, business- like procedures) and greater demands for effectiveness is forcing most NGOs to find ways to learn from their experience and improve their activities –and optimistically, to become learning organizations. Britton38 synthesizes the most imperative reasons for NGOs to focus on learning: (1) the nature of their activities; (2) increasing organizational effectiveness; (3) developing organizational capacity; (4) making the best use of limited resources, (5) strengthening partnerships, and (6) closing the gap between monitoring and evaluation and planning. (1) In an environment extremely difficult to foresee, rapidly-changing and even messy at times, understanding what’s happening becomes indispensable to succeed –and, more than often, simply to survive. (2) It is important not only to understand what’s going on but, primarily, what works, what doesn’t and why. Improved effectiveness is the single most important reason for NGOs to invest on learning39. It is a priority for NGOs to be effective –and be able to demonstrate it. NGOs must have the skills to measure the impact of their actions, understand and reflect on past experiences and extract the 38 Britton, B. Organisational Learning in NGOs: Creating the Motive, Means and Opportunity, 2005. “Britton is involved in consultancy, organisational development, training, facilitation, evaluations and management development with a wide range of organisations in the UK, Europe and many less developed countries in Africa, South Asia and SE Asia. Britton has been involved in development work with voluntary organisations and NGOs in the UK and twenty-six other countries since 1980. Prior to joining Framework in 1999, he worked for fourteen years with Save the Children in the UK as a consultancy project manager and staff development officer, and then in South Asia as Regional Adviser on Human Resource Development.” http://www.framework.org.uk 39 Of course, this is an instrumental conception of learning, understanding it as the means to achieve a goal (increased efficiency). The ideal situation would be for NGOs to consider learning a goal in itself, therefore following a more speculative path that truly stimulates creativity and produces innovative solutions. However, most organizations first need to put systems in place that ensure that learning occurs. Page 26
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 learning from those experiences to design better future programmes. Furthermore, a decisive emphasis on learning produces numerous outcomes and possible solutions and thus reduces the chances of repeating old errors. (3) The development of skills for continuous enhancement processes is a pre- requisite for improving NGOs effectiveness. Upgrading of skills; reviewing and improving the methods used to promote organisational development; planning and organising evaluations; revising organisational goals and objectives, and organisational restructuring40 are essential for better effectiveness. Learning increases the capacity of the staff to contribute to the goals of the organization by constantly challenging and reshaping them. (4) Not only effectiveness is an obligation; efficiency is too. In rare occasions do NGOs (or anyone) have all the resources they need to run constraint-free projects. Learning can and should help to make better use of existing limited resources. (5) Learning within and outside the organization, with and from insiders and outsiders helps to develop common understanding, a sense of community and to share values and goals. (6) Monitoring and evaluation (M&E) are usually the weakest points of every programme and project development. It is a far too common mistake in both the profit and the nonprofit sector to monitor and evaluate exclusively for accountability reasons. The obvious result is that monitoring rarely affects ongoing projects and that the outcomes of evaluation are seldom applied to new projects. However, widening the purpose of M&E from solely seeking an external impact (accountability) to also impelling learning has the potential of producing a true internal impact: improved performance –by effectively applying what has been learnt through the M&E process. 3.2. By the way, what’s learning? “Learning covers all our efforts to absorb, understand, and respond to the world around us. Learning is social. Learning happens on the job every day. Learning is the essential process in expanding the capabilities of people and organisations. … Learning is not just about knowledge. It is also about skills, insights, beliefs, values, attitudes, habits, feelings, wisdom, shared understandings and self-awareness. Questioning, listening, challenging, enquiring and taking action are 40 www.polity.org.za/html/govdocs/white_papers/social97gloss.html Page 27
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 crucial to effective learning. There is no one right way to learn for everybody and for every situation.41” Learning has been a recurrent topic in organizational development, strategy, and management studies in general, especially since the beginning of the nineties. Scholars and practitioners pay a growing attention to learning and incessantly present new procedures, policies, structures and methods for effective learning. Yet surprisingly, very little effort has been put so far into answering the very first question: what is learning and how does it occur? Almost inevitably, we have to look into the educational field for an answer. Smith42 suggests distinguishing between learning as a product and as a process. As an outcome, learning is defined as change in behaviour resulting from a process that can be easily recognized. Although a limited one, that was the predominant view till the eighties and, indeed, provided a fundamental insight about learning: it all revolves around change. Today, learning is widely understood not as a result, but as an unpredictable and winding process: “a process by which behaviour changes as a result of experience43”. Thus, true learning entails progression and transformation. This conception of learning led to the development of different theories attempting to explain how and/or why change happens: (1) The behaviourist theory focuses on the relation between the change in behaviour and the environment; (2) The cognitivists, however, place the emphasis on the individual’s mental process, on the act of knowing; (3) The humanistic approach attempts to explain the learning process as personal fulfilment and, finally, (4) the social and situational orientation theory understands that learning is not exclusively in the acquisition of mental models and structures to understand the world. Learning is also, and mainly, in the structures themselves, in the relationships with others. Ultimately, learning is a linking process putting together past and future, reflection and action –giving purpose to our being. Learning implies the generation and proactive44 utilization of knowledge. 41 Chetley, A. & Vincent, R. Learning to share learning: an exploration of methods to improve and share learning, 2003. 42 Smith, M. Learning theory, Infed, 2005. Mark K. Smith specializes in the field of informal education and lifelong learning. He is the Rank Research Fellow and Tutor at YMCA George Williams College, London and Visiting Professor in Community Education, University of Strathclyde, Glasgow. He has written several books (Creators not Consumers (1982), Developing Youth Work (1988), Local Education (1994), Informal Education (1996, 1999, 2005 with Tony Jeffs) and Youth Work (2005 edited with Tony Jeffs)) and, since 1995, edits and regularly writes for Infed (the informal education hompage). Infed is part of the UK National Grid for Learning and a reference on its own for informal education and lifelong learning with more than four million annual visits. http://www.infed.org 43 Maples and Webster, 1980, cited in Smith’s Learning theory, 2005. 44 As opposed to the simple storage of knowledge; real learning is more to do with the constant ‘engagement’ with knowledge -fostering, using, modifying, revisiting it Page 28
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 3.3. How do we learn? Accepting that learning is a social process45 also means admitting two important assumptions: that learning is experiential46 and that it involves engagement with others –and, thus, that is not only important acquiring and using knowledge but also sharing it. (1) Experiential learning refers to the non-institutionalized learning that takes place in everyday life as a result of direct involvement in the events that constitute our experiences. Learning is a living practice: a continuous spiral in which having an experience is followed by taking the time to examine and reflect on it. Then, the conclusions that emerge from the reflection are distilled into abstract concepts that can be applied to other experiences. Experiential learning reflects the way in which most of us learn and emphasises the importance of the direct relationship with the phenomena that serve as basis for learning47. Learning from experience is an essential component of every learning construction. Kolb’s learning spiral 45 All this argument is directed towards explaining the nature of collective learning. Indeed, people learn in several ways: by acquiring knowledge and skills that add to pre-existing one (acquisitive theory) by nurturing one’s knowledge structure(constructivist theory) –or simply by memorising. However, most organizational learning practitioners take experiential learning as the starting point. 46 Kolb, David A. Experiential learning, 1984. Kolb is the author of the most established model of experiential learning. 47 Unluckily, too many practitioners have taken Kolb’s ‘primary-experience’ learning model as the sole one –specially in educational institutions- marginalizing the necessary secondary learning instances to support direct experimentation. Page 29
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 CARE’s adaptation of Kolb’s experiential learning model48 (2) For a powerful strand of thinking, the epicentre of learning has moved from individual to the group –and, more specifically, to the interactions amongst the individuals. Here, two main concepts dominate: social capital and communities of practice –or Putnam49’s and Lave & Wenger50’s ideas, the latter building upon the foundation laid by the former. a) “Social capital consists of the stock of active connections among people: the trust, mutual understanding, and shared values and behaviours that bind the members of human networks and communities and make cooperative action possible51”. Social capital is the glue that brings cohesion and meaning to both the group and the individual and facilitates learning, funnelling the flow of experiences and knowledge. Therefore, social capital is more than the grouping of persons –though it drives people into groups, into communities. b) Lave and Wenger’s idea of the communities of practice as ‘learning webs’ rest upon that glue: Learning is not reduced to the mere attainment of 48 Fletcher, Gillian, Magar, Veronica & Noij, Frank. Learning by Inquiry: Sexual & Reproductive Health. Field Experiences from CARE in Asia, 2005. CARE International. The chart shows how an international NGO like CARE uses the model for organizational learning purposes. CARE calls the process ‘reflective learning’ or ‘learning by inquiry’. 49 Putnam, R. Bowling alone: America’s declining social capital, 1995. 50 Lave, J. & Wenger, E. Situated learning. Legitimate peripheral participation, 1991. 51 Cohen & Prusack, In good company. How social capital makes organizations work, 2001. Cited in Smith’s Social capital, 2005. Page 30
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 knowledge; instead, a relational view of learning emerges and it is understood as a process of social participation and cooperation. We learn with and from others. 3.4. Organizational learning Organizational development scholars have picked up social capital and communities of practice theories as the starting point to explain how it is possible for organizations, and not only individuals, to learn. On the one hand, social capital, as a fundamental element of the body of an organization, provides the means for achieving relevant benefits52, among them, “better knowledge sharing, due to established trust relationships, common frames of reference and shared goals53”. And, on the other, the ‘community concept’ develops further the idea of organizational learning by transferring the value from the individual to the community, and, thus, the ‘learning products’ to the social level, making it easier for the organization to capture them. Organizations now acknowledge informal networks, groupings and, particularly, work teams as essential vehicles to tackle unstructured problems, share learning and develop and generate organizational knowledge. Organizations, definitively, recognise the connections that ‘groups’ generate, and the structures and practices that it entitles as fundamental to improving performance –and to achieving strategic goals. In the light of the social perspective, organizational learning occurs since people don’t learn in isolation, rather in company. Moreover, an individual’s learning environment –conformed by his/her mates, but also by the instruments available and the processes that they use- constitute both the place for learning and the source of learning itself. Thus, learning impregnates the whole organization. 52 (1)Better knowledge sharing, due to established trust relationships, common frames of reference, and shared goals, (2)Lower transaction costs, due to a high level of trust and a cooperative spirit (both within the organization and between the organization and its customers and partners), (3)Low turnover rates, reducing severance costs and hiring and training expenses, avoiding discontinuities associated with frequent personnel changes, and maintaining valuable organizational knowledge, and (4)Greater coherence of action due to organizational stability and shared understanding, Smith’s Social Capital, 2005. However, Smith also warns that very little research has been conducted so far in this field to provide incontestable results. Thus, some of these claims are based, at least partially, on analogies with other institutions. 53 Idem supra 53 Page 31
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Yet, as Nevis54 stresses, organizational learning is not something new or exclusive to certain firms. Organizational learning, understood as “the capacity or processes to maintain or improve performance based on experience55”, happens in every organization –a different matter is how effective organizations are in keeping, disseminating and making use of what they learn. Indeed, all organizations learn as they produce and evolve; learning is inherent to progression and to the process of identity construction –socialization, as the basis for organizational culture, generates learning. Moreover, Nevis goes as far as to say (and probably, that’s what differentiates organizations effective at learning) that “the value chain of any organization is a domain of integrated learning… [to accept this] is to think of the work in each major step, beginning with strategic decisions through customer service, as a susbsystem for learning experiments”. Certainly, an organization’s most precious knowledge lies hidden within its specific core competences; the difficult task, however, is to unearth and benefit from that richness. Whereas there’s no clear-cut agreement of what is a learning organization (see next section), which is usually discussed more in normative than in descriptive terms, there seems to be a common understanding about how the learning process develops in organizations. Such a process involves, at least, the following steps: knowledge acquisition, knowledge sharing, and knowledge utilization. Nevertheless, as it happens in the case of individuals, the course is non-linear and iterative as a consequence of the formal/informal, explicit/tacit ways in which learning takes place. 3.5. Learning organizations There are several definitions of what the learning organization ought to be: (1) Senge characterizes learning organizations as “organizations where people continually expand their capacity to create the results they truly desire, where new and expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people are continually learning to see the whole together.56” 54 Nevis, Edwin C., DiBella, Anthony J. & Gould, Janet M. Understanding organizations as learning systems. Organizational Learning Centre, MIT Sloan School of Management. The paper is the result of a recent research project. Edwin (director of special studies at the Organizational Learning Centre, MIT) and his colleagues studied how learning takes place in four different companies (Motorola, Fiat, Electricité de France and Mutual Investment corporation) 55 Indem supra 54 56 Senge, Peter. The fifth discipline. The art and practice of the learning organization, 1990. Page 32
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 (2) Marsick understands them as “organization(s) that (are) able to transform (themselves) by acquiring new knowledge, skills, or behaviours. In successful learning organizations, individual learning is continuous, knowledge is shared, and the culture supports learning. Employees are encouraged to think critically and take risks with new ideas.57” (3) Finally, Garvin defines the learning organization as that one “skilled at creating, acquiring and transferring knowledge, and at modifying its behaviour to reflect new knowledge and insights.58” The above definitions shape a continuum from abstraction to concretion, from a more idealistic to a more realistic perspective –and they resemble somehow the evolution of the understanding of the learning organization. Senge, without any doubt the most influential thinker in the field of organizational learning, was named ‘Strategist of the century’ in 1999 by the ‘Journal of Business Strategy’. The one million copies sold of his book ‘The fifth discipline: the art and practice of the learning organization’ (1990) attests to the tremendous influence of his ideas. Not only did the corporate world adopt his thoughts immediately, also the nonprofit sector enthusiastically embraced his concept of the firm59. Senge proposes a complex, demanding and attractive model (see chart next page) for the learning organization. This organization must excel in the mastery of five different dimensions: systems thinking, personal mastery, mental models, building share vision and team learning. Only an organization that commands all those five ‘component technologies’ is capable of “adaptive learning (which is about coping) and generative learning (which is about creating)60”. 57 Marsick, Victoria. The Learning Organization: An Integrative Vision for HRD, Http://www.astd.org/astd/Resources/performance_improvement_community/Glossary.htm 58 Garvin, David. Building a learning organization. Harvard Business Review, 1993. 59 One of the symptoms of the lack of effective communication between the two ‘worlds’, the corporate and the nonprofit, is the constant reluctance to accept the other’s ideas. Not in the case of Senge; his ideas have permeated both the for-profit and non-for-profit sectors. 60 Senge, Peter M. The Leader's New Work: Building Learning Organizations. Sloan Management Review, 1990 Page 33
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Characteristics of the learning organization, according to Senge61 Characteristic Definition Associated best practice Positive by-product Systems thinking The ability to see 1.Practicing self mastery (see Long-term improvement or (group) interrelationships rather than next row) change; decreased linear cause-effect; the ability to 2.Possessing consistent mental organizational conflict; think in context and appreciate models continuous learning among the consequences of actions on group members; Revolutionary other parts of the system. 3.Possessing a shared vision over evolutionary change. 4.Emphasis on team learning Personal mastery The ability to honestly and 1.Positive reinforcement from Greater commitment to the (individual) openly see reality as it exists; to role models/managers organization and to work; less clarify one's personal vision. 2.Sharing experiences rationalization of negative events; ability to face limitations 3.More interaction time between and areas for improvement; supervisory levels ability to deal with change. 4.Emphasis on feedback 5.Balance work/non-work life Mental models The ability to compare reality or 1.Time for learning Less use of defensive routines in (individual) personal vision with perceptions; 2.Reflective openness work; less reflexivity that leads to reconciling both into a coherent dysfunctional patterns of understanding. 3.Habit of inquiry behaviour; less avoidance of 4.Forgiveness of oneself difficult situations. 5.Flexibility/adaptability Shared vision The ability of a group of 1.Participative openness Commitment over compliance, (group) individuals to hold a shared 2.Trust faster change, greater within picture of a mutually desirable group trust; less time spent on future. 3.Empathy towards others aligning interests; more effective 4.Habit of dissemination communication flows. 5.Emphasis on cooperation 6.A common language Team learning The ability of a group of 1.Participative openness Group self-awareness; (group) individuals to suspend personal 2.Consensus building heightened collective learning; assumptions about each other 3. Bi-directional communication learning "up and down" the and engage in "dialogue" rather flows; hierarchy; greater cohesiveness; than "discussion". enhanced creativity. 4.Support over blame; 5.Creative thinking Senge’s contribution has been enormous62: it has opened the door for a renewed concept of the corporation and provided an ideal goal to strive. Yet his 61 http://www.albany.edu/sph/Hoff_learning/hpm_tim_learnorg.htm . The University of Albany, NY, in collaboration with the School of Public Health and the Albany Medical College, and under the programme ‘Creating a learning environment for residents. Around patient safety and mistakes’, developed this analysis of the characteristics of the learning organization. Taking Senge’s approach as the basis, it also includes elements of Schön’s and Argyris’ theories. Schön devoted much of his work to the study of the ‘learning society’ and Argyris is the ‘father’ of the single and double loop theory of learning: Single loop learning refers to problem solving without substantial alterations to the existing framework. Double loop learning consists in questioning the underlying assumptions and beliefs, and changing them if required. 62 And it continues to be. His latest book, “Schools that learn” is a provocation for educational institutions to re-thinking the way they teach –and learn. Page 34
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 construction is, in some aspects, excessively intentional and idealistic, and thus, too exigent; Senge, as well as many other theorists, write about what organizations should be, when most organizations must still figure out who they are and how they can improve. Therefore, I believe with Garvin63 that organizations need more practical and humble approaches to, step by step, become more effective at learning. Instead of talking about learning organizations, I prefer to speak about organizations that are capable of learning effectively –of acquiring, sharing and using knowledge in a way that add to improved performance. In that sense, Nevis’ (et al.) contribution is edifying. The authors suggest looking at organizations as learning systems, and propose a descriptive, non- judgemental model with two complementary sides: learning orientations (the values and practices that inform and determine learning) and facilitating factors (the structures and processes that influence learning and its effectiveness). This model aims at assessing the current ‘learning strengths’ and identifying those areas in which the organization can do better. (1) Nevis’ learning orientations reflect the real drivers for learning in an organization, and locate the organizational sites where learning can be found. The authors present the orientations in opposite terms (see next chart). Testing the learning status against the orientations’ chart can be useful in two ways: a) in helping the organization positioning in a realistic learning matrix and b) in assisting the organization in deciding next directions. Nevis learning orientations64 Learning orientations External (adaptation) Knowledge source Internal (innovation) Products & Services (less learning Focus Processes (more learning oriented) oriented) Personal (individual knowledge) Documentation mode Public (publicly available knowledge) Informal (casual daily interaction) Dissemination mode Formal (defined procedures of sharing) Incremental (adaptive) Learning Transformative (radical) Design (engineering & production) Value-chain Deliver (marketing & services) 63 Garvin, David. Building a learning organization. 64 A personal interpretation of Nevis, Edwin C., DiBella, Anthony J. & Gould, Janet M. Understanding organizations as learning systems. Organizational Learning Centre, MIT Sloan School of Management. Page 35
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Individual Skills development Group/ team (2) The combination of orientations that steers learning in the organization results dramatically influenced by a conjunction of facilitating factors65: a) Organizational culture is, above all, the most influential feature: the learning style is ultimately a product of the culture (that thing that employees do when bosses are not around) as learning is embedded in the essence of the organization. Indeed, there are cultures that facilitate and increase learning effectiveness: -An environment where learning is fostered, rewarded and valued; that implies, among other things, the provision of sufficient resources –money, but also time and space. - A true climate of openness where everyone is approachable and there are not carved-in-stone principles; an experimental mindset where boundaries can be pushed forward for the rich exchange and stimulation of ideas. Moreover, as learning is much to do with power and control, if an organization wants to cultivate the innovation and creativity that should arise from learning, tight and rigid structures must be kept at a minimum. - Finally, it is also important to nurture and promote participation across the organization to involve people from all levels in the decisions that affect them, share responsibilities and develop capacities and skills. b) Clearly, another determinant is a committed and involved management; authentic champions living the learning. Managers that advocate for learning represent an enormous influence -declaring learning an organizational goal on its own and, hence, indicating to the staff their true dedication. c) The ability to scan the environment in which it operates, to understand the relevance of what others are doing, and to comprehend the organization’s relative position is an important factor to expedite learning. d) Equally relevant is the awareness of the need for improvement, what Nevis calls the ‘performance gap’. This recognition is particularly vital 65 The factors here listed are not solely Nevis’. I have combined some of Nevis, also included some Smith’s and Britton’s suggestions and contributed with some ideas myself. Page 36
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 since it usually leads to the acceptance that the organization must learn in order to enhance its performance, to get where it wants to be. e) Finally, a pledge to continuous education (in the sense of the Japanese “Kaizen”), to cultivating the feeling that things could always have been done better, and not the mere offering of training programmes certainly provides a good incentive for learning. Nevis’ proposition first and foremost value relies on two aspects: (1) It provides a visible and pragmatic framework that encourages reflection about the learning capabilities (and the strategic consideration of learning) within a given organization; (2) It stresses the importance of conceiving learning, not as an add-on that can be plugged in to the organization, but as something that emerges from an organization’s core competencies and that is deeply integrated in its value chain. Accepting both considerations can be priceless in the attempt to pull the right levers for effective learning. 3.6. Opening spaces for learning. Some tools and methods Few would deny today the power of the previous arguments. Although most organizations agree to the relevance of learning and some of them even go as far as to design the proper procedures and standards, yet very few are actually able to translate these concepts into organizational reality. Organizations strive to find the most adequate practices to capture, share, create, and measure, learning. The business world, governments and the military66 have produced some very interesting and effective frameworks, tools and methods. Interestingly enough, however, some of the most innovative and even ‘revolutionary’ practices to ensure learning come from the nonprofit sector. The huge pressures to demonstrate results and the shrinking funds (as the most powerful reasons, but no the only ones) seem to have sharpened the talent of certain NGOs. Learning emerges from necessity. It is true, as it has been said before, that the sector, by and large, suffers a ‘learning deficit’: NGOs still have a long way to go (as do most corporations, as well) 66 Companies like IDEO, P&G, Toyota, Xerox and 3M just to mention a few are widely recognised as strongly innovative firms thanks to the procedures and systems in place that allows them to ‘learn’. Likewise, military debriefings like the LBDA (Learning Before, During and After) method and programmes like NASA’s ASK with its emphasis in storytelling pursue enhancing current learning mechanisms to increase efficiency and effectiveness. Page 37
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 to learn effectively. Nevertheless, many NGOs are investing in non-conventional approaches that are producing excellent results. So let’s take a hands-on approach67 and review some practical propositions in the literature: (1) Britton’s ‘spaces for learning’: (2) The Canada School for public service’s ‘inventory of learning tools’, and, finally, (3) the results of Chetley and Vicent’s research. (1) First, Britton offers a range of very specific actions (formal and informal, at the organizational and the individual level) for the facilitation of learning through the opening of mechanisms, spaces and time (see next chart). Creating the space for learning68 Individual Collective Formal Organisations can: Organisations can: • Legitimise learning by building it into job • Build learning objectives into project and programme descriptions. plans and organisational strategy. • Manage workload planning to avoid overload. • Develop team work as a required way of working. • Use HR mechanisms such as staff supervision and • Develop mechanisms for establishing collective appraisal to monitor and evaluate individuals’ responsibility for results. contributions to organisational learning. • Set up action learning sets, learning groups and • Ensure that each person has an individual plan for communities of practice. their own learning and development. • Organise training courses, workshops, conferences and • Develop ‘reflective practitioner’ competences. meetings. • Set up individual mentoring and coaching schemes. • Introduce ‘no-travel’ times, ‘homeweeks’ and ‘reflection • Encourage and enable attendance at training periods’. courses, workshops, conferences and meetings. • Commission learning reviews to examine themes of work. • Create opportunities for individuals to represent the • Create cross-functional teams to develop guidelines, organisation in networks. procedures or policies. • Encourage individuals to write articles for publication. • Include an explicit ‘lessons learnt’ section in all regular reporting formats. Informa Individuals can: Organisations can: l • Build in time for reflection at the end of each day and • Provide physical space that encourages informal at significant stages of pieces of work. networking. • Engage in informal networking. • Set up intranets, newsletters or other ways of keeping • Join and use on-line discussion forums. people informed about each other’s work. • Develop ‘habits’ that support reflective practice (e.g. keep a learning journal). These ‘spaces’ have to be filled with tools and methods. The right method for learning has to emerge from the non-linear process of change that is learning. 67 This sub-section comments some of the minimums of the learning organization to gradually direct the focus towards a specific method described in the next sub-section: monitoring. 68 Britton, B. Organisational Learning in NGOs: Creating the Motive, Means and Opportunity, 2005. Pages 31 and 32. Page 38
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 While planning and the commitment of time and space are primordial, a method distant from the organizational culture will not help. As a result, there is no step- by-step procedure to guarantee the transformation into a learning organization69. The moment we understand learning as a continuous process of change, the learning procedures, policies and structures evolve as the learning itself occurs. (2) The Canada School of public service70 (CSPS) presents an “inventory of learning tools”. It is not an inclusive package, and most of the recommended instruments lack a sense of comprehensiveness –they are too narrow and limited to specific situations. Nonetheless, the inventory has three important virtues. First, it transforms the guiding principles into concrete actions. Second, its classification of the tools into a) diagnostic, b) instructional, c) knowledge sharing, and d) knowledge generation tools, helps to clarify the purpose of the instruments within a learning framework. And, third, it is focused and hands-on. Although an exhaustive review of the tools71 is impossible here, it’s worthwhile highlighting a few. a) The diagnostic tools are designed to asses the organizational ‘status quo’. Among the ones selected by CSPS, the following ones are particularly helpful: i) Learning histories72, which borrows from the MIT’s Centre for Organizational Learning, and basically consists in the recording of ‘lessons learned’ (stories with attached explanations of their meaning) to explain when, how and why learning took place, and what influences it had in the organization. ii) Qualitative reporting, benchmarking and audits: - “Intellectual capital accounting techniques” as the CSPS calls it, started in Scandinavia (i.e. Skandia) a few years ago and is gradually gaining acceptance elsewhere –especially in the UK, but also in countries like Spain (Unión Fenosa’s Intellectual Capital Report73). 69 That is certainly not the opinion of the myriad of consultancies that, under the commercial significance of organizational learning, have developed and offer countless ‘learning templates’ 70 “The Canada School of Public Service is the common learning service provider for the Public Service of Canada. It brings a more unified approach to serving the common learning and development needs of public servants and helps ensure that all Public Service employees across Canada have the knowledge and skills they need to deliver results for Canadians”. http://www.myschool-monecole.gc.ca 71 Tools: a) diagnostic: learning histories, learning audits, individual learning plans, organizational learning plans and learning self-assessment guides; b) instructional: curriculum-based instruction, presentation-based instruction, learning exercises and, mentorships and apprenticeships; c) Knowledge sharing: expertise mapping, knowledge depositories, learning centres and portals, and network tools; d) knowledge generation tools: brainstorming sessions, brainsqueezing sessions, learning teams, skunkworks and intelligence gathering. 72 Learning histories Project: http://ccs.mit.edu/lh/ 73 http://www.unionfenosa.es Page 39
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 - Comparative analysis via best-practice case studies can be a helpful way to asses where one stands. - And, finally, self-assessment audits provide a quick snapshot of the situation of the organization –Marcia Conner, of ‘Fast Company’ magazine offers a neat example of an audit (see next chart). The [simple] learning audit74 Pro-learning culture (+) 1 – 5 Anti-learning culture (-) 1–5 - People at all levels ask questions and share stories about - Managers share information on a need-to-know basis. successes, failures, and what they have learned. People keep secrets and don’t describe how events really happened. - Everyone creates, keeps, and propagates stories of - Everyone believes they know what to do, and they individuals who have improved their own processes. proceed on that assumption. - People take at least some time to reflect on what has - Little time or attention is given to understanding happened and what may happen. lessons learned from projects. - People are treated as complex individuals. - People are treated like objects or resources without attention to their individuality. - Managers encourage continuous experimentation. - Employees proceed with work only when they feel certain of the outcome. - People are hired and promoted on the basis of their - People are hired and promoted on the basis of their capacity for learning and adapting to new situations. technical expertise as demonstrated by credentials. - Performance reviews include and pay attention to what - Performance reviews focus almost exclusively on what people have learned. people have done. - Senior managers participate in training programs designed - Senior managers appear only to “kick off” management for new or high-potential employees. training programs. - Senior managers are willing to explore their underlying - Senior managers are defensive and unwilling to values, assumptions, beliefs, and expectations. explore their underlying values, assumptions, beliefs, and expectations. - Customer feedback is solicited, actively examined, and - Customer feedback is not solicited and is often ignored included in the next operational or planning cycle. when it comes in over the transom. - Managers presume that energy comes in large part from - Managers presume that energy comes from “corporate learning and growing. success,” meaning profits and senior management bonuses. - Managers think about their learning quotient, that is, their - Managers think that they know all they need to know interest in and capacity for learning new things, and the and that their employees do not have the capacity to learning quotient of their employees. learn much. Total (for pro-learning culture) Total (for anti-learning culture) b) Knowledge sharing tools pursue the organization-wide dissemination of learning, making it available at all levels and promoting the creation of new knowledge –an attempt to systematically spot, collect and propagate it. Knowledge maps and knowledge depositories are two preferred ones: 74 Marcia L. Conner. Conner currently writes for the ‘Fast Company’ magazine and is the general manager of ‘Agelesslearner’. She has written numerous books on organizational learning and advised hundreds of companies. This assessment was first published in Creating a Learning Culture: Strategy, Technology, and Practice, 2004, http://www.agelesslearner.com Page 40
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 i) Knowledge maps75 are visual representations of the flow of knowledge within an organization, identifying not only the source and destination but also the path and the reasons. A comprehensive knowledge map includes what travels and how. Making explicit this information can help organizations drive their learning efforts. ii) Knowledge depositories such as electronic databases, e-learning centres in the form of extra/intranets, portals and blogs. (3) Finally, Chetley and Vincent, from ‘Exchange’, conducted a wide- reaching research76 for the UK Commission for Health Improvement in 2003. The aim of the report77 was to identify best learning practices as a basis to encourage learning across the British National Health System organizations, and to help them improve their practice. However, the authors did not limit the study to the British health sector –nor even to the public sector. Instead they looked both at the public sector and corporations, international institutions and NGOs all over the world, searching for effective learning practices. The result is an exciting collection of 15 learning methods and tools that work (see next chart). The approaches range from easy and concrete tools related to a particular aspect of a firm, to complex system-wide analysis referring to a whole society. They include cross-national workshops (like the one held between the Tanzanian Commission for Science and Technology and the Dutch-based International Institute for Communication and Development); organizational multi-sectoral teams (World Bank); using non-written methods –video, in this case- to document and share learning (ActionAid -NGO) and the use of storytelling in monitoring, like the ‘Most Significant Change’ technique (Voluntary Service Overseas -NGO). 75 Other forms of ‘mapping’ include ‘expertise mapping’ and ‘learning maps’. 76 Chetley, A. & Vincent, R. Learning to share learning: an exploration of methods to improve and share learning, 2003. 77 As the authors explain, the report “is an illustrated literature review drawing on studies in the fields of education, psychology, organisational learning, personal learning, and participatory approaches to explore understanding of good learning practice. It includes more than 15 case studies that illustrate methodologies and approaches used to share learning in the business, public, and voluntary sectors, paying particular attention to the types of processes that encourage engagement with diverse communities of interest or multiple stakeholders”. Page 41
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Chetley’ and Vincent’s summary of possible learning approaches according to the complexity of the approach and the organisational level it works on78 Org. level / Specific unit or Programme/ Organisational/ Societal/ complexity process project level strategic environmental Tool • E-mail exchange • After Action Review (BP) • Lessons database (Soul City/Puntos) • Electronic workspaces (Bellanet, DAC, • Outcome measures (Bellanet, World Bank) Exchange) (World Bank) • Collective reflection (New Zealand) Integrated • Communities of Practice • ALPS (ActionAid) • Appreciative Inquiry approach (Xerox) (Canada) • Multi-Sectoral Team Learning (World Bank) System-wide • Tanzania local • Story-telling/Most • CDRA Homeweek • Effectiveness analysis content workshop Significant Change (VSO) Initiative (IICD/ COSTECH) • CIET Social Audit Of all these approaches, the ‘Most Significant Change’ technique (MSCT) is especially ground-breaking and noteworthy. The approach, a qualitative and participatory monitoring method based on the systematic collection and interpretation of stories, has two main objectives: to identify change and to promote organizational learning. The next two sections (4 and 5) introduce and explain the key aspects of MSCT: Section 4 develops its core idea and objectives, and provides basic background information on MSCT two main pillars: monitoring and storytelling. Section 5 places the origin, tracks the evolution, and describes the implementation steps of MSCT. Additionally, this paper includes a comprehensive guide to implementing the ‘Most Significant Change’ technique, including an appendix with several practical examples. 78 Chetley, A. & Vincent, R. Learning to share learning: an exploration of methods to improve and share learning, 2003. Page 3. Page 42
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 “We now accept the fact that learning is a lifelong process of keeping abreast of change. And the most pressing task is to teach people how to learn.” Peter Drucker, management theorist 4. Introduction to the ‘Most Significant Change’ technique 4.1. MSCT in six words A question and stories. That’s the core of MSCT: A simple focused question that triggers a myriad of rich and thick stories. ‘Looking back over the last month, what do you think was the most significant change in the quality of people’s life in this community? ’79 Departing from an open question like the one above, MSCT presents itself as a (1) subjective, (2) qualitative, (3) complementary, and (4) participatory monitoring method that begins with and revolves around the systematic (5) collection and interpretation of (6) stories of change. (1) People are asked to give their opinion and to recognise and select changes affecting a certain field over a given period of time. In this way, MSCT explicitly refuses to use objective parameters. On the contrary, it embraces subjectivism and makes it pivotal to monitoring. MSCT, instead of ticking boxes in a list or calculating ratios, opens the door to the values, judgements and standpoints of many stakeholders. It is the people who have to make sense of their reality and define the meaning of events. (2) MSCT welcomes heterogeneity and qualitative content. The method is designed not to extract facts and figures and fit them in a mould but to keep the ‘cold facts’ with their wrapping. MSCT is interested in collecting the full version: rich and deep descriptions that depict an experience and whose significance is attached80 to the story. (3) MSCT doesn’t use predetermined indicators. The absence of defined indicators makes the monitoring method more free, dynamic and adaptive. Unexpected circumstances can be more easily tackled in a more flexible context. Yet, MSCT attempts neither to substitute nor to finish with the conventional 79 This example is taken from the 1994 CCDB Bangladesh project (see manual at the end of this paper). It is a ‘basis’ question which parts can be adapted to ask for more/less/different input. 80 This aspect will be extensively discussed in the following sections. Page 43
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 monitoring methods. Quite the opposite, MSCT tries to be a balancing tool to supplement the lack of information that may suffer traditional methods. (4) & (5) MSCT involves the collection and systematic participatory interpretation of stories of significant change81. Stories are collected at the field level and then follow a path upwards the organization. MSCT collects stories of change that occur at the field staff level. These changes are systematically reviewed by stakeholders of the organization, with each group (the field staff itself, management and Headquarters) selecting the most significant change to pass to the next level. Participation, as it has already been mentioned, arises from the essence of the concept: collection and interpretation is done by the people directly involved in the changes. (6) Finally, stories constitute the whole kernel of the process. And more specifically stories that describe significant change. __ Thus, MSCT requires and promotes intense participation and interaction of a wide range of actors throughout the organization. McClintock synthesises the main steps of the process82: (1) Identification of areas of interest for storytelling. (2) Development of a format for data collection (title, what happened, when, and why the change was considered relevant) (3) Selection of the stories at multiple levels, “on those accounts that best represent a program’s [or project] values and desired outcomes. 4.2. Monitoring for learning MSCT tries to fulfil two complementary (and inseparable) objectives: the strengthening of organizational learning and the effective monitoring of projects. Participation and stories sum up to define the essence of a truly innovative system. 81 An alternative description of the MSCT’s participatory character : The MSCT requests multiple stakeholders to draw meaning from experience through the writing and collection of stories –specifically of stories of change. Staff at all levels engaged in a project first selects the kinds of change they want to learn about –the areas of impact of a given project. Field staff and/or clients are asked the same questions in a systematic way and requested to write stories about personal experiences related to one of the determined areas of change, as well as an explanation of the significance of the stories. The stories and their explanations are methodically recorded and collected. Then, teams of employees at different organizational levels rigorously discuss, analyze, add meaning, filter and select the most significant stories and refer them to the next level until a single story per area of change is left. 82 Charles Mclintock. Using narrative methods to link program evaluation and organization development. The evaluation Exchange, winter 2003/2004. McClintock is Dean of the Fiedling Graduate Institute School of Human and Organization Development. Page 44
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Organizational learning: The bottom line of MSCT is to reinforce organizational learning83. Enough arguments, both theoretical and practical, make MSCT an inclusive instrument with attractive possibilities. MSCT encourages the organization, through the collective analysis of its objectives and capacity, to exercise a deep reflection on its own experience and the impact of its actions. Thus, MSCT attempts to broaden the utilization of monitoring processes from accountability purposes to organizational learning. Indeed, the method exhibits a strong potential for effective organizational learning: (1) Stakeholders have to extract meaning from actual events that have already happened –there are no predefined indicators. As a consequence, the gap between what was planned and what has been actually accomplished becomes evident; MSCT is able to raise awareness over the ‘performance gap’ and therefore, turns on a signal indicating the need to learn. (2) The demand for clients/field staff to place significance on their experience openly asks for individual reflection on personally experienced episodes. Direct experience and thoughtful consideration are at the core of the method. (3) The character of the selection process involves an open, cross-sectional and multi-level dialogue that can produce tangible benefits: At a minimum, it should generate a move towards a better understanding of each other’s values. At its best, it develops a consistent network among participants that constitute the basis for deeper interrelations -different participants get involved in developing a clear interpretation from several perspectives. (4) The selection of stories opens horizontal (among the members of a group) and vertical (the exchange of views among different levels within the organization) channels of communication, key for effective learning. 83 That was the initial purpose when Davies (Rick Davies, ‘inventor’ of the Most Significant Change technique. Davies is an independent Monitoring and Evaluation Consultant, Committee member at CIIR, and associate at the Centre for Development Studies, Swansea University.) described the method as an ‘evolutionary approach to facilitating organizational learning’. The early idea was to construct a mechanism to allow the effective supervision of the impact of development projects in uncertain circumstances and with unexpected results, and to learn from the experience. That premise remains valid today yet the technique has evolved, grown and matured since 1994 as a logical consequence of its implementation in several dozens projects. The application of the method has resulted in a wider scope, unanticipated benefits and uses –as well as unforeseen flaws. Today we are beginning to have a solid catalogue that documents the application of the MSCT across the globe: a collection of papers recording the use of MSCT in the Philippines and Central America, the implementation by Landcare and the Australian Department of Education, its complementarities to conventional monitoring methods as well as MSCT’s deficiencies, benefits and strengths. Page 45
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 (5) The personal and the social processes can promote necessary discussions, not only about specific stories but, more widely, about organizational values. Moreover, as ultimately participants must come to an agreement about the meaning and relevance of certain events within the context of an organization, MSCT can foster a dynamic shared vision. (6) The broad participation also helps in the dissemination of knowledge throughout the organization; instead of centralizing the information in the hands of a few, the involvement of several layers of staff guarantees knowledge sharing at multiple levels. The dissemination takes place in both a formal and informal way as the stories are documented, but the discussions that precede every selection process are clearly informal. (7) MSCT stimulates the development of individual and team skills. On the one hand, it forces storytellers to identify projects’ effects –and to develop their analytical skills and understanding of the mechanisms of projects. On the other, it encourages team work skills to reach the needed consensus for the selection of stories. Monitoring: As any other monitoring tool, MSCT aims at assessing the evolution of the activities of a project/programme and thus determining whether its terms are being met and whether the goals are likely to be achieved. Yet, MSCT is not a traditional monitoring system (see next chart) Differences between conventional monitoring and MSCT84 Issue Conventional monitoring Most Significant Change Technique Objective/ Conventional monitoring uses indicators. It has MSCT has abandoned pre-defined indicators. Instead subjective ‘objectivism’ as its central principle and a need subjective opinions, values and perspectives are valued to control or ignore differences and subjective and differences and interpretations are being explored. perspectives. Assumptions are made about Meaning is extracted out of events/changes that have future events/changes. taken place. Quantitative/ Conventional monitoring is merely quantitative MSCT is qualitative in content and uses “thick description” qualitative in content and efforts are done to homogenise which provides in depth information. Experience is selected experience rather than differentiate. (most significant) and the core is to define meaning of experience rather than to identify a central tendency. Collection & Information is collected in relation to the MSCT is open-ended and it asks broadly about happened interpretation prescription of change (indicators). Data is often changes. Data collection and interpretation are done by the interpreted out of context at senior levels in the people with direct experience with the changes. organisation. Static/ Conventional monitoring systems are frequently An MSCT approach is potentially dynamic and adaptive, dynamic static structures: indicators don’t change, same where the domains of change can be modified in relation to questions are asked. new and unexpected contexts. 84 Ringsing, Bettina. Page 89, Learning about advocacy. Monitoring as a tool for learning in Ibis South America. MSc Thesis, August 2003. http://groups.yahoo.com/group/MostSignificantChanges/files Page 46
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Before explaining the mechanics of MSCT, the next two subsections attempt to provide an essential background to understand two of the characteristics that differentiate MSCT from the rest and make it particularly suitable as an organizational learning tool: its participatory character and its preference for storytelling to record change. 4.3. Participatory monitoring There has been, during the last years, a strong trend within the NGO sector to enlarge the purposes of monitoring85 and, at the same time, to broaden the ways in which monitoring is conducted. On the one hand, and in order to satisfy multiple stakeholders’ demands, NGOs have realized that to use monitoring for mere accountability reasons was to waste the wealth of information that arises through the process. Subsequently, NGOs do not use the result of their monitoring exclusively for proving (and improving) their accountability but also to inform practice and management learning. On the other, a wave of critiques on scientific and external conventional monitoring systems, based mainly on their failure to comprehend the complexity, diversity and contingency of reality86, has induced the development of alternative and complementary monitoring methods. (1) Today monitoring is about sharing knowledge. The view of monitoring as a mechanism of ongoing review that also produces results employed simply to justify programmes is no longer sufficient. Everyone, from the World Bank to the United Nations through every NGO that tracks its projects, expects monitoring to contribute to organizational learning (see next chart, as an example) –and, ultimately, to enhance performance. Monitoring should provide, at least, facts and figures about the development of projects. That information, once analyzed, selected and systematized becomes part of the body of organizational knowledge available for future projects –it becomes generated knowledge. To say it in a nutshell, the organization learns from the experience. Monitoring provides the organization with practical instruments to share knowledge among multiple project stakeholders. Ideally, the learning extracted from the monitoring process will inform future 85 UN define monitoring as “a continuing function that aims primarily to provide the management and main stakeholders of an ongoing intervention with early indicators of progress…in the achievement of results” United Nations. Handbook on Monitoring and Evaluating for Results. United Nations Development Programme (UNDP), Evaluation Office, 2002. Page 6 86 Späth, Brigitte. Current State of the Art in Impact Assessment: With A Special View on Small Enterprise Development. August 2004. Swiss Agency for Development and Cooperation. Page 47
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 implementations, facilitate the avoidance of past mistakes and, thus, improve performance. Consequently, the scope of monitoring has expanded from simple accountability directed at external stakeholders to: a) supporting broader accountability (including a wider range of stakeholders –see the section ‘NGOs’-) b) building staff capacity c) informing decision-making and, above all d) enhancing organizational learning. UNDP’s monitoring purposes87 There are countless monitoring methods, ranging from complex, exhaustive and systematic ones to more informal, sometimes almost ‘ad hoc’, procedures. The next chart provides a selection of a comparative analysis of the most common types of monitoring in the development sector offered by the World Bank. However, it is not the intention of these lines to present a review of the different alternatives, but, instead to provide some essential background information on participatory monitoring –the kind of monitoring that informs the main object of this paper: The ‘Most Significant Change’ technique. 87 Idem supra 85 Page 48
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Selection of monitoring methods* Performance Indicators Logical Framework Formal surveys Rapid appraisal Participatory U - Setting performance targets - Improving quality of - Providing baseline - Providing rapid - Learning about local s and assessing progress project and program data against which the information for conditions and local e toward achieving them. designs—by requiring performance of the management decision- people’s perspectives - Identifying problems via an the specification of strategy, program, or making, especially at and priorities to early warning system to clear objectives, the project can be the project or program design more allow corrective action to be use of performance compared. level. responsive and taken. indicators, and - Comparing different - Providing qualitative sustainable assessment of risks. groups at a given understanding of interventions. - Indicating whether an in- depth evaluation or review - Summarizing design point in time. complex socioeconomic - Identifying problems is needed. of complex activities. - Comparing actual changes, highly and trouble-shooting - Assisting the conditions with the interactive social problems during preparation of detailed targets established in situations, or people’s implementation. operational plans. a program or project values, motivations, and - Evaluating a project, design. reactions. program, or policy. - Providing objective basis for activity - Providing a key input - Providing context and - Providing knowledge review, monitoring, to a formal evaluation interpretation for and skills to empower and evaluation. of the impact of a quantitative data poor people. program or project. collected by more formal methods. A - Effective means to measure - Ensures that decision- - Findings from the - Low cost. - Examines relevant d progress toward objectives. makers ask sample of people - Can be conducted issues by involving v - Facilitates benchmarking fundamental interviewed can be quickly. key players in the a comparisons between questions and analyze applied to the wider design process. assumptions and target group or the - Provides flexibility to n different organizational explore new ideas. - Establishes t units, districts, and over risks. population as a whole. partnerships and local a time. - Engages stakeholders - Quantitative estimates ownership of projects. g in the planning and can be made for the - Enhances local e monitoring process. size and distribution of learning, - When used impacts. management s dynamically, it is an capacity, and skills. effective management Provides timely, tool to guide reliable information for implementation, management decision- monitoring and making. evaluation. D - Poorly defined indicators - If managed rigidly, - Results are often not - Findings usually relate - Sometimes regarded i are not good measures of stifles creativity and available for a long to specific communities as less objective. s success. innovation. period of time. or localities—thus - Time-consuming if a - Tendency to define too - If not updated during - The processing and difficult to generalize key stakeholders are d many indicators, or those implementation, it can analysis of data can from findings. involved in a v without accessible data be a static tool that be a major bottleneck - Less valid, reliable, and meaningful way. a sources, making system does not reflect for the larger surveys. credible than formal - Potential for n costly, impractical, and changing conditions. - Household surveys surveys. domination and t likely to be underutilized. - Training and follow-up are expensive and misuse by some a - Often a trade-off between are often required. time-consuming. stakeholders to g picking the optimal or - Many kinds of further their own e desired indicators and information are interests. having to accept the difficult to obtain s indicators which can be through formal measured using existing interviews. data. Page 49
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 C - Low to high, depending on - Medium, depending - Medium to high. - Low to medium, - Low to medium. o number of indicators on extent and depth of depending on the scale Costs vary greatly, s collected, the frequency participatory process of methods adopted. depending on scope t and quality of information used to support the and depth of sought, and the approach. application and on inclusiveness of the how local resource system. contributions are valued. * Performance Indicators Measure inputs, processes, outputs, outcomes, and impacts for development projects, programs, or strategies. When supported with sound data collection analysis and reporting, indicators enable managers to improve service delivery. Participation of key stakeholders in defining indicators is important because they are then more likely to understand and use indicators for management decision- making. Logical Framework It helps to clarify objectives of any project, program, or policy. It aids in the identification of the expected causal links—the “program logic”—in the following results chain: inputs, processes, outputs, outcomes, and impact. It leads to the identification of performance indicators at each stage in this chain, as well as risks which might impede the attainment of the objectives. Formal surveys The collection of standardized information from a carefully selected sample of people or households. Surveys often collect comparable information for a relatively large number of people in particular target groups. Rapid Appraisal A quick, low-cost ways to gather the views and feedback of beneficiaries and other stakeholders, in order to respond to decision-makers’ needs for information. Participatory methods They provide active involvement in decision-making for those with a stake in a project, program, or strategy and generate a sense of ownership in the M&E results and recommendations. __ Source: Monitoring and evaluation: Some tools, methods and approaches. World Bank Operations Evaluation Department. (2) Rossman88 understands participatory monitoring as “a process of self- assessment, knowledge generation, and collective action in which stakeholders in a program …collaboratively define the evaluation issues, collect and analyze data, and take action as a result of what they learn through this process”. Hence, participatory monitoring (PM) entails the true and active engagement of people throughout the organization in every step of the monitoring process. This view is perfectly consistent with the understanding of organizational learning as an activity that needs to happen collectively –learning as a social and experiential process. 88 Rossman, Gretchen B. Participatory Monitoring & Evaluation. Centre for International Education, University of Massachusetts at Amherst, Page 50
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Indeed, PM methods promote that sort of values that nurture effective learning in the organization: better understanding of the project by those directly affected; empowerment of employees; potential for the development of capacities and skills, effective team building and creation of trust and, “a deep sense of meaningfulness to the job89 [as] the learning associated…is experiential”. Although non-participatory methods are still dominant, the incremental extension in the last decade of PM methods in the NGO sector might be seen as the symptom of an important tendency towards more democratic institutions. However, one must be realistic and careful in the approach to PM –and this applies, of course, also to MSCT. PM is not simply a different way (involving more people) of doing old things. It entails, more than anything else, the alteration of power dynamics within an organization and, as a result, it is more complex than normally assumed –and typically, its execution hides fierce tension. For this reason, before taking the decision to implement PM methods, two issues must be cautiously considered: a) its compatibility with organizational values, and b) the “reversals of power90” that it might imply. a) If the organization is not a participative one (with everything that it implies), the attempt to get through PM methods for the assessment of projects will surely generate resistance, open debates and demand extra efforts. However, that situation can be deliberately chosen as a healthy way of reviewing organizational values and structures. b) “The question ‘Who changes?’ calls us to attention. The point is not to what to change as much as how we change ourselves. Participation has little meaning unless we, and particularly those of us in positions of power, allow others to take part, to set agendas, take decisions, manage and control resources91 ”. In a non-participatory organization, the implementation of PM methods usually means that the ‘traditional’ relational structures of power and control are shaken –making uncomfortable many people in the organization; those that have to yield power, but also those that have to assume greater responsibility. Participation, negotiation, flexibility and a sincere commitment to learning, 89 idem supra 88 90 Aubel, Judy. Participatory Monitoring & Evaluation for Hygiene Improvement. Beyond the toolbox: What else is required for effective PM&E? A Literature Review, 2004. Environmental Health Project, U.S. Agency for International Development. 91 Robert Chambers. ‘Who changes’, 1998. Cited in Aubel, Judy. Participatory Monitoring & Evaluation for Hygiene Improvement. Beyond the toolbox: What else is required for effective PM&E? A Literature Review, 2004. Environmental Health Project, U.S. Agency for International Development. Page 51
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 leave the sphere of ‘guiding principles’ and become everyday tools within the organization. 4.4. Narrative with a purpose It seems clear that monitoring and evaluation (M&E) methods are no longer seen as mere reviews of performance but more as a continuous process towards improvement through the facilitation of learning. A new ingredient has been added recently, consciously, to participatory and qualitative forms of monitoring. According to McClintock, “narrative methods present a form of inquiry that has promise for integrating evaluation and organization development92“. Furthermore, storytelling is already part of monitoring, if only in a haphazard way: countless stories pop up in the informal (and formal) interviews that constitute the backbone of any M&E process. However, more often than not those stories that surround the ‘really important’ quantitative information get lost in the course of the process. As a consequence, there seems to be very scarce systematic use of narrative in development projects monitoring. Stories have only recently come to the forefront of both for and non-for- profit businesses. But stories have always been there, it is only the context that has changed. A purposeful use of the stories that emanate in every organization is a practical and priceless resource -one of those that belong to the core competences. Stories and storytelling provide “a [practical, economical] and valuable way of presenting and communicating knowledge93”. Stories, the natural and preferred means in which we make sense of reality, are also part of a profoundly social activity that can facilitate the flowing of knowledge across the organization. Thomas (et al.) go on to say that “…storytelling is useful in creating, capturing, and internalizing knowledge and…it accomplishes all of these simultaneously and not sequentially. Storytelling is also a representative knowledge socialization process” The methodical collection of stories and their systematic yields particularly useful interpretation results in participatory evaluation and organizational learning. Sharing experiences through narrative (1) generates emotional 92 Charles Mclintock. Using narrative methods to link program evaluation and organization development. The evaluation Exchange, winter 2003/2004. 93 Thomas, J.C., Kellog, W.A. & Erickson, T. The knowledge management puzzle: Human and social factors in knowledge management. IBM Systems Journal, 2001. Page 52
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 connections, (2) builds trust, (3) cultivate norms94, (4) bring meaning, and (5) functions as a disclosure mechanism95. (1) Stories flick a switch in adults: they describe unexpected situations that surprises and grasp attention, thus making them easier to remember than cold facts –and, consequently, easier to retrieve. (2) Stories are, ultimately, about an individuals’ set of values and competencies. When someone tells a story he/she is, implicitly, revealing his/her understanding of the world. Stories make the storyteller formulate a coherent explanation of reality which, in turn, provides important insights about his/her values. This exercise of transparency and honesty generates in others commitment and trust. (3) Equally, stories in an organizational context, convey the values and principles of an organization, bringing to the front values believed to be embedded (hopefully, would argue more than one manager) in the subconscious of the individuals. (4) As almost a logical outcome of the preceding three points, stories translate abstract concepts of planning and project design into specific faces and names. Also, stories are an ideal way to communicate and adapt to constant change, providing sense to chaos. (5) Finally, narrative is a preferred way to extract and transfer non- formalized knowledge already in place in every organization. It is considered a fundamental disclosure mechanism in the learning organization, especially when it comes to capturing highly contextualized implicit knowledge under conditions of uncertainty. “[T]he use of storytelling as a disclosure mechanism creates a largely self-sustaining, low costs means by which knowledge can be captured on an ongoing basis…Storytelling is a natural, organizational process in which the organization is managed as a complex ecology, through a series of low cost interventions96”. MSCT is one of the few approaches that systematically collect and interpret stories. MSCT facilitates the collective interpretation of the stories to bring meaning to monitoring processes and opens the door to a whole new concept in the utilization of stories. 94 Sole, D. & Wilson, D. Storytelling in organizations: the power and traps of using stories to share knowledge in organizations. LILA. Graduate School of Education. Harvard University. 95 Snowden, David. Snowden, David. Storytelling: an old skill in a new context, 2002. 96 Idem supra 90 Page 53
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 “Impact is where significant change have been brought about in the lives of poor women and men” Oxfam Great Britain, NGO 5. MSCT. Origin, history and process 5.1. Bangladesh, 1994 Corruption and poverty have reigned for decades in Bangladesh, one of the poorest countries of the world. The Government seems incapable of satisfying the essentials of an impoverished population that struggles to survive on $1 a day. External aid and NGOs are essential in such an environment providing indispensable funding, covering vital needs and empowering the people. One of those NGOs is CCDB (Christian Commission for Development in Bangladesh). Active since 1973, CCDB has earned high respect in the country and successfully developed numerous projects in rural areas to become a major institution in Bangladesh97. Though its initial operations were exclusively focused on emergency relief in a war-devastated country, CCDB has gradually become involved in a wider scope of development activities. Today CCDB spreads its efforts in programs that range from capacity building to advocacy and networking to community development. At the beginning of the nineties CCDB felt the need to enhance its contribution to the Bangladeshi society in two ways: improving the effectiveness of its own development activities and facilitating the participation of the people to develop sustainable growth. In 1994 CCDB started the Peoples Participatory Development Program (PPDP), a micro-credit project that involved 46,000 people, mostly women, in 785 villages. The PPDP aimed at providing assistance through “group based savings and credit facilities used to meet the needs of individual households, grant assistance given to the same groups on a pro-rata basis and intended for community level developments, and skills training, mainly for livelihood 97 “The organisation implements its development programmes in various regions of the country. At present, CCDB is working in 21 districts. Theses are: Manikganj, Rajshahi, Nawabganj, Naogaon, Natore, Pabna, Gopalganj, Barisal, Dinajpur, Rangpur, Bandarban, Rangamati, Faridpur, Jessore, Magura, Jhenaidaha, Khulna, Satkhira, Kushtia, Narail and Cox's Bazar. The programme beneficiaries of CCDB numbered 49 small organisations and 136,595 families in 2001. CCDB's programmes and budget are supported by 14 donor agencies from different parts of the world, including Europe, Australia and USA. Their annual contribution is approximately $3.5 million. The World Council of Churches (WCC), Geneva, co-ordinates a Round Table for CCDB.” http://banglapedia.search.com.bd/HT/C_0063.HTM Page 54
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 purposes”98. More than 500 groups were to devise and implement their own plans adjusted to their specific needs. It was easy to predict that each group would behave in a different way and asses its success or failure in a different manner. Accurately overseeing the impact of the plan was of paramount importance. However, traditional systems did not seem adequate to fulfil the demands that would arise from such a large-scale project and its unexpected results. 5.2. Indicators-based monitoring is not always enough Traditional monitoring (performance indicators, logical framework) appeared insufficient under the light of this new and vast project for a number of reasons. The most important one was the limitations of predetermined indicators: (1) Indicators are usually rigid: They respond to fixed parameters that do not change as the program changes. As a rule, indicators are defined at a moment in time and used for subsequent periods and are rarely modified unless dramatic external changes occur. (2) Indicators are complex and expensive: They require skills to correctly understand and interpret both the terms that are used and the objectives that indicators are supposed to measure. Thus, indicators ask for qualified and trained people which, in turn, make monitoring expensive. (3) Indicators present a systematic yet simplified reality: To facilitate analysis, indicators portray a ‘light’ version of complexity, necessarily avoiding the explanation of the reasons behind the facts and figures. (4) But, above all, predetermined indicators reduce the opportunity to learn from the unexpected: Indicators are defined in relation to the objectives meant to be achieved. In that sense, indicators only reflect the incidence of expected events of change. (5) And ownership of indicators is determinant: Who decides what factors and for what they are going to be used is essential to the definition of a logical framework. More than often, indicators are not decided by those who employ them but imposed by senior levels far from the field. CCDB felt that logical frameworks and performance indicators monitoring demanded too much effort and resources and did not fit well the open-ended nature of the PPDP. All these drawbacks of conventional monitoring systems made 98 1994, Bangladesh. Rick Davies's initial development of the MSC method. Page 55
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 CCDB reconsider the exclusive use of traditional forms of evaluation and look for superior alternatives. In 1994 Rick Davies was writing his PhD in organizational learning in NGOs. As part of the fieldwork for his PhD, Davies developed the ‘Most Significant Change’ technique (MSCT), initially conceived as an “evolutionary approach to facilitating organizational learning”. MSCT jumped from concepts to action as Davies contacted CCDB. Within the scope of the PPDP, Davies conducted an experiment on participatory monitoring in Western Bangladesh using MSCT as a non-indicator based approach. MSCT was born. 5.3. Travelling the world with MSCT Therefore, MSCT was fully implemented for the first time more than eleven years ago. The Christian Commission for Development in Bangladesh welcomed Davies’ idea and initiated a voluntary collaboration with him that produced fruitful results. No donor was involved in that first experiment. Last year, Care International, Oxfam and Ibis were among the dozen institutions that funded ‘The Most Significant Change Technique. A guide to its use’ –the ‘official’ MSCT manual. In between, the spread of MSCT has followed a decentralized path. On the one hand, the utilization of MSCT has not been restricted to its initial application in rural areas development projects: from community empowerment programs to learning methods and vast country programs, MSCT has proven useful in many different circumstances. On the other hand, the ‘users’ form also a varied community: It has been mainly, but not exclusively, put into practice by International development NGOs in developing countries. Yet Jess Dart has also contributed to its extensive use in Australia. It was first used in a Dairy program and is currently being implemented by several institutions like the Government of Australia, Department of Education which is moving into the third phase of the ‘Learning to Learn’99 program. Thus, in a decade, MSCT has gained the recognition and support of major international NGOS and some governments. 99 “The Initiative is funded by the Department of Education and Children’s Services (DECS), South Australia. Learning to Learn is changing the way teaching and learning is conceptualised and constructed for the children and students of South Australia.It provides leaders and teachers with extended learning opportunities to focus on generative learning – learning how to learn. Participants are supported to critically reflect on their current practices by immersing themselves in the latest research on learning and curriculum and conducting locally based curriculum inquiry.” http://www.learningtolearn.sa.edu.au , http://www.learningtolearn.sa.edu.au/learning_workroom Page 56
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Today, a mailing list100 and Davies’ monitoring and evaluation website (see footnote 51) constitute the main hubs that channel the exchange of ideas, follow the progression and discuss the application of MSCT. The growing online database is the most visible sign of the active use of the method. This technique is positively contributing to the fields of participatory monitoring and organizational learning, bringing some evident benefits to the organizations and communities that use it. Selection of applications of MSCT101 Year Country Organisation Program Informant 1994 Bangladesh CCDB People’s Participatory Rural Rick Davies Development Program 1994 Ethiopia SIDA Community Empowerment Program Terry Bergdall 1996 Multiple countries ITDG, UK Global program Helen Wedgewood 1996 India Aga Khan Foundation Aga Khan Rural Support Program Barry Underwood 1997 Australia Department of Primary Target 10 Jess Dart Industries, Victoria. 1998 Mozambique Oxfam 2000 Multiple countries VSO Global program 2001 Ghana DFID Brong Ahafo District Support Program Francis Johnston 2001 Mozambique MS Denmark Country program Peter Sigsgaard 2001 Zambia MS Denmark Country program Peter Sigsgaard 2001 Australia Department of Victorian Bestwool Jess Dart Communities 2002 Multiple countries MS Denmark Country program Peter Sigsgaard 2002 Laos ADRA Robyn Keriger 2002 Tanzania Country program Peter Sigsgaard 2002 Thailand STREAM Pat Norrish 2003 Papua New Guinea Oxfam New Zealand Oxfam New Zealand Bougainville Jess Dart Program 2003 Ghana CARE Country program Fiona Percy 2003 Central America Ibis Denmark Country programs Silke Mason Westphal 2003 Australia S.A Dept. of Education Learning to Learn Margot Foster 2003 Australia Landcare Landcare statewide Jess Dart 2003 Multiple countries Oxfam Australia Deb Elkington 2004 Australia Desert Knowledge CRC National body Jess Dart 2004 Australia Landcare North Central Landcare Jess Dart 2004 Australia Creatively Connecting Communities Jess Dart 100 http://groups.yahoo.com/group/mostsignificantchanges/ 101 Davies, Rick & Dart, Jessica. Page 76, The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use. (2005). http://www.mande.co.uk/docs/MSCGuide.htm Page 57
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 5.4. Overview 1 02 of MSCT process MSCT follows a systematic implementation process that can be separated into nine different steps. Not all of them are essential; some of them are discretionary and simply provide consistency to the methodology. However, there are three steps that cannot be avoided: the collection of significant change stories (4), the selection of the most significant of these stories by at least a group of stakeholders (5) and the feedback of the selection and the reasons behind to the relevant people (6) The steps to implement MSCT are the following: (1) Getting started (2) Defining the domains of change (3) Setting the reporting period (4) Collecting significant change stories (5) Selecting the most significant change story (6) Feeding back (7) Verifying the most significant stories of change, and (8) Conducting meta-monitoring and secondary analysis 4.7.1. Getting started Getting started can be challenging. That is why MSCT needs dedicated champions to succeed: influential, motivated and proactive people that understand, support and promote the use of the method in the organization. The contribution of senior staff is essential at the beginning to excite and motivate people but also to clarify ownership and management. In order to do so, senior staff must appoint a MSCT management team (or an individual) to lead the process: this team must encourage and support participants, provide all the needed information, facilitate and coordinate both the collection and selection of significant change stories and guarantee an adequate feedback loop. The most important thing to do at this stage is to convey a direct and simple message. First, a lot of emphasis should be placed on the easiness of the method. And second, the purpose of MSCT has to be clearly stated: Enough time has to be spent on explaining and clarifying what objectives are to be met and how they relate 102 For a detailed analysis of the implementation steps, please refer to the last chapter of this paper, 'The ‘Most Significant Change’ technique. User’s manual: a quick reference guide'. Page 58
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 to the rest of the organizational goals. And the best way to do so, it seems, is in a down to business workshop. Up to date, in most projects a one-to-three day workshop preceded the implementation of MSCT and a second ‘refresher session’ was organized in the middle of the process. Facilitators (usually MSCT team) steered the workshops. Some lessons can be extracted from these workshops: It is very effective to use a lot of practical exercises, drive participants through the complete process of participatory selection of the most significant story and to allow plenty of time for discussion. 4.7.2. Defining the domains of change Domains are topics, areas of change, usually related to key goals of an initiative, a project or an organization as a whole that storytellers are asked about103. Most of the organizations with experience with MSCT opted for explicitly matching the selected domains with chief areas of activity104. More interesting, however, is to link the selection of domains not to the organization’s activities but to specific organizational objectives, thus making the selection process a key strategic decision by aligning monitoring purposes and organizational goals. The domain selection can serve multiple purposes but, in any case, the choice should be determined by what the organization wants to learn. It has been a common practice among organizations to define an open domain: a domain that refers to changes in unspecified areas. Even in some cases, organizations have not defined domains at all at the beginning but let them materialize as the process developed. Finally, it is also useful to define a negative domain, a domain that accounts for what goes wrong. More than often, it is at least of equal importance to learn about what is not functioning and the reasons behind as to know about what functions. Besides its far-reaching strategic purposes, the selection of domains fulfils one basic immediate function: domains serve as categories to group large number of stories. Those who select the domains decide what should be learned. In a participatory method like MSCT the selection of domains should not be left to 103 In the sample question, ‘Looking back over the last year, what do you think was the most significant change that the educational program has had on your professional expectations ?’, the underlined ‘your professional expectations ’ represents the domain of change. 104 For instance, ADRA Laos defined 4 domains of change: Changes in people’s health, Changes in people’s behaviour, Changes that reflect an area to improve (negative), Changes in any other area (open). Page 59
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 senior managers alone: middle management and the staff directly involved in the monitoring process (and partners, in some cases) must also have a voice in such a selection. There are many procedures to come to an agreement over the most suitable domains. A top-down decision is probably the fastest and easiest way to select domains, yet also the less participatory one. A bottom-up process guarantees that the input of the field staff will be taken into account (however, such process should be carefully designed in large organizations). Dart and Davies advocate for the Delphi technique105. 4.7.3. Setting the reporting period Defining a reporting period is important to draw a timeline. However, establishing ‘a priori’ the optimal reporting period is simply not possible as each implementation of MSCT is unique. Although many organizations adjust the time to collect stories to already-in-place quarterly reporting periods, some organizations choose to report monthly and others do it annually, depending on their specific needs. 4.7.4. Collecting ‘significant- change ’ stories The open question previously studied is the key instrument to generate significant change stories. The collection of stories identifies those ones that express significant changes within each domain and entitles several important aspects: whose stories and what information to collect, what method and reporting format to use and issues about confidentiality. The monitoring purposes serving organizational goals determine the selection of not only the domains of change, but also the storytellers. The objectives to be met decide the appropriateness of the people106. At a minimum, the description of the story should be collected, the significance of the story for the storyteller and information about who collected (or wrote) the story and when. The story itself should be a brief and concise description 105 Initially conceived as a technology forecasting method, the Delphi technique is widely used today in many sectors as a tool to reach consensus about an opinion or view. This technique, if used effectively, can be highly efficient and generate new knowledge. http://www.is.njit.edu/pubs/delphibook , http://www.iit.edu/~it/delphi.html 106 If an organization wants to learn about the impact that its programs has on people’s lives, the storytellers by default will be the field staff collecting the stories of the beneficiaries. If an organization wants to measure the quality of its partnerships, the narrators will be the staff members of each one of the partners. If, finally, an organization needs to asses the firmness of its values, every stakeholder will be given the opportunity to write his/her story. Page 60
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 of events107. The significance of the story for the storyteller is a crucial part of MSCT. On the one hand, the storyteller makes sense of his/her own story. On the other hand, that meaning is made explicit. And, finally, it is important to record some basic information about the storyteller. The design of a simple standard format helps to easily collect the information. The availability of resources (time and people, basically) and the amount of effort that the organization wants to place in MSCT inevitably determines the choice of method. Three methods are preferred: interviews, group discussion and individual questionnaires. The latest allow for thoughtful stories and usually generate richer and thicker descriptions. The reason for the collection of the stories as well as how the stories are going to be used must be very clearly explained to the storytellers. They must also be asked for explicit consent if the organization intends to use the stories later on for whatever purposes. 4.7.5. Select ing the ‘most- significant change’ story Dart and Davies call the selection process ‘summary by selection’: It is a hierarchical, participatory, iterative and documented process in which each level selects the most significant of the stories submitted by the lower level and passes it on to the next level (see chart in next page). Documentation is one of the most important characteristics of the whole process. Each group justifies its selection by discussing, agreeing upon and finally attaching to the selected story a comment that explains the reasons for the selection and its significance for the group. The group must argue why the story has been selected linking the argumentation to the monitoring and organizational objectives under assessment. The group must also reflect on recommendations that address the reasons for change, the role played by the organization and the implications for future actions. Documentation becomes of the utmost relevance as it is a guarantee for transparency and accountability, links documented impact and strategic decision and opens strategic planning to all the levels of the organization. 107 It is very useful to also ask for additional information such as: A title makes easier the subsequent classification and selection of stories; Recommendations/lessons learned have the potential of bringing fresh perspectives to the organization and help to draw out implications of the story; Reflections on the influence of the organization can provide priceless information that serve as a basis for a more detailed and comprehensive analysis in the next step. Page 61
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 The selection of stories must be open to a wide range of actors across the organization involving, at its best, everyone in the line of decision of the specific program/project. In any case, selection is easier when storytellers don’t have to decide over their own stories. Selection criteria and methods are diverse and numerous. The preference for a specific method will depend on aspects such as the size of the group, the time available and the necessity to reach a consensus. Diagram of MSCT selection process Most significant change stories (1 MSC story per domain) Selection level 2 MSCS Committee Selection level 1 Selection level 1 Selection level 1 MSC Group 1 MSC Group 2 MSC Group 3 (1 MSC story/domain) (1 MSC story/domain) (1 MSC story/domain) Stories from storytellers (1 MSC story per storyteller) Significant change story per domain 4.7.6. Feeding back t he results of the selection process Many organizations that have applied MSCT have had a negative experience with the feedback stage. Organizations, although aware of its relevance, very often failed to provide adequate feedback. However, it seems evident that consistently feeding back the results to lower levels has the potential to produce long-term benefits: It shows commitment and involvement at all levels. It improves organizational communications, promoting intensive dialogue up and down the hierarchy of the organization. And it makes judgements explicit to everyone. Page 62
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 A comprehensive feedback should include the stories that have been selected for each domain, the reasons for their selection (linking them to the organizational objectives to be met) and the procedure for the selection. Emails, community newsletter and written formal reports are all optimal means for communicating results. 4.7.7. Ve rifying the most sign ifican t storie s of change Verification of stories of change is not an essential step in MSCT. Nevertheless, follow-up structures can serve two important purposes in large organizations: Story-checking can provide an argument for accountability to external parties such as partners and funders. And it can also be an incentive for storytellers to improve the quality of stories, when they know that the changes described in the stories might be double-checked. Yet verification can have undesired consequences and demands a supplementary organizational effort. A careless verification management can originate distrust among the participants in the monitoring process leading to discouraging effects. Moreover, a verification management process needs to be put in place deciding who is to conduct the analysis and what and how is to be checked. 4.7.8. Condu cting meta- monitoring The purpose of this complementary step is to keep record of all the stories that have been collected within the organization. Meta-monitoring of stories of change builds a purposeful database that facilitates the access to relevant information about policies’ impacts; exhibits the efficacy of the monitoring method and reflects the progression of MSCT. Meta-monitoring, ‘the monitoring of the monitoring’, has the potential to play a key role in organizational planning. Meta-monitoring should record whether the most-significant-change stories selected and the recommendations attached to them have had an impact on organizational policies. Keeping track of this sort of information helps to not only asses the effectiveness of on-going projects/programs but also whether the organization is learning from the insights that provide most- significant-change stories and effectively incorporating that learning into its new actions. Page 63
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 6. Moving forward “Alice thinks about what Looking-Glass House must be like, wondering aloud to Kitty if there might be a way to break through to the other side of the mirror. All of a sudden, Alice finds herself on the mantle, staring into the mirror. She magically steps through the mirror into Looking-Glass House. On the other side of the mirror, Alice looks around and finds that the room she is standing in resembles the mirror image of the room in her own house. However, several parts of the room look quite different108”. I must admit that when I first thought of this paper I was at this side of the mirror –and I only saw the reflection. After a whole (and intense) year of business education it seems almost an unavoidable place to be. I was fully convinced, and I am still, that NGOs (and, by extension, all kinds of civil society organizations and the public sector) can deeply benefit from a wide range of practices, methods and tools of the corporate world. Hence, the first intention of this paper was to analyze some of those techniques and ‘offer’ them to the NGO sector. However, I somehow, little by little, found myself in the mantle. NGOs are certainly making theirs many business instruments, adapting and transforming them to their specific needs–and they are learning fast. Yet, out of necessity, they are also developing their own innovative and solid methods –which, in addition, are many times economical, easy-to-use and effective. Resourcefulness and creativity don’t understand of mirror sides and jump from reality to reality. The argument of the ‘two worlds’ is rapidly diluting. NGOs adopt results- based management procedures and private firms are more and more involved in development projects. The different businesses, different role, different context justification to different methods is questioned today more than ever. NGOs, businesses and the government understand and demand the benefits of solid partnerships –as an example, at the time of this writing (December 5th), an international conference in Berlin to find better ways to offer universal health insurance in developing countries has begun. Organized by the GTZ, a major German NGO, it brings together worldwide representatives from the corporate world, the civil society sector and the public administration. 108 Lewis Carrol. Alice’s Adventures in Wonderland & Through the Looking-Glass : Chapter I: Looking- Glass House. Page 64
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Effectiveness, efficiency and efficacy are musts for both the profit and the nonprofit sectors –and learning from experience and using that knowledge to inform future practice is the only way to get there. NGOs must fully grasp the advantages of some business principles –Peter Drucker109 urged nonprofit organizations to “convert donors into contributors” and develop a stronger sense of “community and common purpose”. Businesses must learn from NGOs how to better asses and fulfil the needs of their clients, build sincere employee commitment and open the organization to them. Already fifteen years ago, another Peter, Senge, spoke of the leader as a designer, a teacher and a steward. This paper has attempted to describe a method used b y NGOsq that shows a great potential. Ultimately, the ‘Most Significant Change’ technique tries to nurture one of our biggest assets, the foundation for much of the rest: trust. The enthusiasm expressed by those who have experienced MSCT speaks for itself. The next step is to look at Kitty, break through to the other side of the mirror, and, fearlessly, try MSCT outside the development world. Or even better, to unfold the mirror. 109 Peter Drucker, Managing the Nonprofit organization, 1990. Page 65
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 7. References Civil Societ y - Clayton, Andrew, Oakley, Peter & Jon Taylor. Civil Society Organizations and Service Provision, October, 2000, UNRISD (United Nations Research Institute for Social Development). http://www.unrisd.org/80256B3C005BCCF9/httpNetITFramePDF?ReadForm&parentunid=19A B2640214382A380256B5E004C94C5&parentdoctype=paper&netitpath=80256B3C005BCCF9/ (httpAuxPages)/19AB2640214382A380256B5E004C94C5/$file/intrac.pdf - Bashyam, Leo. The role of the Northern development NGO. Chapter 10.10, The Companion to development studies, Desai, Vandana and Potter, Robert B., Oxford University Press, 2002 - Eikenberry, Angela M. and Drapal, Jodie. The marketization of the nonprofit sector: civil society at risk? Public Administration Review, March/April, 2004. - Glasius, Marlies. London School of Economics and Political Science. Civil society, http://www.fathom.com/feature/122536/ - London School of Economics, What is civil society? http://www.lse.ac.uk/collections/CCS/what_is_civil_society.htm - Salamon, Lester M. Global Civil Society. An overview. The Johns Hopkins University, Institute for Policy Studies, Centre for Civil Society studies,2003, http://www.jhu.edu/~ccss/pubs/pdf/globalciv.pdf - Seckinelgin, Hakan. Civil society as a metaphor for western liberalism, Civil Society Working Paper 21, January 2002, http://www.lse.ac.uk/collections/CCS/pdf/CSWP21.pdf - United Nations. Handbook on Non-Profit Institutions in the System of National Accounts, 2003. - Van Roy, Alison. Strengthening civil society in developing countries. Chapter 10.5, The Companion to development studies, Desai, Vandana and Potter, Robert B., Oxford University Press, 2002 N GOs - Anderson, Kenneth. International NGOs: A Law Unto Themselves, www.ngowatch.org - Blood, Robert. Should NGOs be viewed as political corporations? Journal of Communications Management, 7th July 2004 - de Clerck, Paul. Interview - Friends of the Earth's Paul de Clerck on lobbying transparency. July. 26th 2005, www.euractiv.com - Desai, Vandana. The role of non-governmental organizations. Chapter 10.6, The Companion to development studies, Desai, Vandana and Potter, Robert B., Oxford University Press, 2002 - Clark, John D. NGOs and the state. Chapter 10.8, The Companion to development studies, Desai, Vandana and Potter, Robert B., Oxford University Press, 2002 - Entine, Jon. Capitalism’s Trojan Horse: Social Investment and Anti-Free Market NGOs, www.ngowatch.org - Johns, Gary. NGOs Are Here To Stay, www.ngowatch.org - Johns, Gary. The NGO Challenge: Whose democracy is it anyway? www.ngowatch.org - Lewis, David. NGO: questions of performance and accountability. Chapter 10.11, The Companion to development studies, Desai, Vandana and Potter, Robert B., Oxford University Press, 2002 - Lloyd, Robert. The Role of NGO Self-Regulation in Increasing Stakeholder Accountability, July 2005. http://www.oneworldtrust.org/documents/SelfReg%20(final)July05.pdf - Peeters, Marguerite A. Participatory Democracy In The New Europe : A Critical Analysis, www.ngowatch.org Page 66
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 - Rabkin, Jeremy. Why the Left dominates NGO Advocacy Networks, www.ngowatch.org - Riggs, David and Huberty, Robert. NGO Accountability: What the US Can Teach the UN, www.ngowatch.org - Spar, Debora & L. La Mure, Lane T. The Power of Activism: Assessing the Impact of NGOs on Global Business. California Management Review, Apr 1, 2003. - SustainAbility. The 21st Century NGO: In the Market for Change, Second edition, 2003, www.sustainability.com - The Economist. A rigged dialogue with society, Oct 21st 2004 - Townsend, Janet G. Chalenges for NGOs. Chapter 10.14, The Companion to development studies, Desai, Vandana and Potter, Robert B., Oxford University Press, 2002 - Willetts, Peter. What is a Non-Governmental Organization? City University, London. http://www.staff.city.ac.uk/p.willetts/CS-NTWKS/NGO-ART.HTM Organ izational learning - Britton, B. Organisational Learning in NGOs: Creating the Motive, Means and Opportunity, 2005. http://www.intrac.org - Canadian Centre for Management’s development. Action-research roundtable on the learning organization. An inventory of learning tools –working paper, 2000. - Chetley, A. & Vincent, R. Learning to share learning: an exploration of methods to improve and share learning, 2003. http://www.healthcomms.org - Davies, Rick. Learning circles and loops: Time for more sophisticated representations, 2004. http://www.mande.co.uk - Garvin, David. Building a learning organization. Harvard Business Review, 1993. - Hailey, J., James, R. & Wrigley, R. Rising to the challenges: assessing the impacts of organisational capacity building, 2005. http://www.intrac.org - Mausolff, Cristopher. Organizational learning in response to performance monitoring: a social action approach. School of Business and Public Administration, California State University, Bakersfield - Nevis, Edwin C., DiBella, Anthony J. & Gould, Janet M. Understanding organizations as learning systems. Organizational Learning Centre, MIT Sloan School of Management. http://www.solonline.org - Pesce, F. and Vanelli V. Organizational learning: an evaluation model. Institute of Social Research (IRS), Bologna - Italy, 2002. - Roper, Laura & Petit, Jethro. Development and the Learning Organisation: an introduction. - Senge, Peter M. The Leader's New Work: Building Learning Organizations. Sloan Management Review, 1990 MIT Sloan School of Management. - Smith, Mark k. , 2005. Several articles at the Informal education website: http://www.infed.org - Learning theory. - David A. Kolb on experiential learning. - Paulo Freire. - Social Capital. - Communities of practice. - Organizational learning. - Learning organization. - Peter Senge and the learning organization. - Van Brabant, Konrad. Organizational and institutional learning in the humanitarian sector: opening up the dialogue, 1997 Page 67
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Narrat ive - Barry, David. Strategy retold: toward a narrative view of strategic discourse. Academy of management review, 1997. - Denning, Stephen. The narrative angle. The seven most valuable forms of storytelling. A managers handbook on narrative. - Downing, Stephen. The social construction of entrepreneurship: narrative and dramatic in the coproduction of organizations and identities, 2005. Baylor University. - Jameson, Daphne. Narrative discourse and management action. The journal of business communication, 2001. - Snowden, David. The art and science of Story or ‘Are you sitting uncomfortably?’ Part 1: Gathering and harvesting the raw material, 2005. http://www.cynefin.net - Snowden, David. The art and science of Story or ‘Are you sitting uncomfortably?’ Part 2: The Wett and Warp of purposeful storytelling, 2005. http://www.cynefin.net - Snowden, David. Storytelling: an old skill in a new context, 2002. http://www.cynefin.net - Thomas, J.C., Kellog, W.A. & Erickson, T. The knowledge management puzzle: Human and social factors in knowledge management. IBM Systems Journal, 2001. http://www.research.ibm.com - Sole, D. & Wilson, D. Storytelling in organizations: the power and traps of using stories to share knowledge in organizations. LILA. Graduate School of Education. Harvard University. M onitoring - Aubel, Judy. Participatory Monitoring & Evaluation for Hygiene Improvement. Beyond the toolbox: What else is required for effective PM&E? A Literature Review, 2004. Environmental Health Project, U.S. Agency for International Development - Barton, Tom. Guidelines to Monitoring and Evaluation. How are we doing? CARE Uganda, http://www.kcenter.com/care/dme/default.htm - Blagescu, Monica, de Las Casas, Lucy & Lloyd, Robert. Pathways to Accountability. The GAP Framework, 22 Sep 05. http://www.oneworldtrust.org/documents/Pathways%20to% 20Accountability,%20The%20GAP%20Framework.pdf - Dart, Jessica. A Story Approach for monitoring change in an agricultural extension project. http://www.latrobe.edu.au/aqr/offer/papers/JDart.htm - Davies, Rick. Monitoring and Evaluating NGO Achievements, 12 April 2000, http://www.mande.co.uk/docs/arnold.htm - Fletcher, Gillian, Magar, Veronica & Noij, Frank. Learning by Inquiry: Sexual & Reproductive Health. Field Experiences from CARE in Asia, 2005. CARE International. - Global Environmental Facility. Monitoring and Evaluation Policies and Procedures, 01.2002, http://www.gefweb.org/M_E_Policies_and_Procedures.pdf - Guijt, Irene, Arevalo, Mae & Saladores, Kiko. Participatory monitoring and evaluation; Tracking change together. PLA Notes (1998), Issue 31, pp.28–36, IIED London. - IFAD. A Guide for Project M&E, http://www.ifad.org/evaluation/guide/ - Kovach, Hetty, Neligan, Caroline & Burall, Simon. Power without Accountability, 2003, http://www.oneworldtrust.org/documents/GAP20031.pdf - McClintock, Charles. Using narrative methods to link program evaluation and organization development. The evaluation Exchange, winter 2003/2004. http://www.healthcomms.org - Rossman, Gretchen B. Participatory Monitoring & Evaluation. Centre for International Education, University of Massachusetts at Amherst, http://www.umass.edu/cie/Themes/participatory_evaluation.htm - Späth, Brigitte. Current State of the Art in Impact Assessment: With A Special View on Small Enterprise Development. August 2004. Swiss Agency for Development and Cooperation. www.sdc.admin.ch/themes Page 68
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 - The Business Communicator. The Top Ten Measurement Mistakes. March 2004, Melcrum, London. http://www.sinicom.com/Sub%20Pages/pubs/articles/article61.pdf - United Nations. Handbook on Monitoring and Evaluating for Results. United Nations Development Programme (UNDP), Evaluation Office, 2002. - White, Sarah & Pettit, Jethro. Participatory Approaches and the Measurement of Human Well-being. August 2004. United Nations University, http://www.wider.unu.edu/publications/rps/rps2004/rp2004-057.pdf - World Bank operations evaluation department -evaluation capacity development. Monitoring and evaluation: some tools, methods and approaches. http://lnweb18.worldbank.org/oed/oeddoclib.nsf/24cc3bb1f94ae11c85256808006a0046/a5efb b5d776b67d285256b1e0079c9a3/$FILE/MandE_tools_methods_approaches.pdf - Zadek, Simon & Raynard, Peter. Stakeholder Engagement: Measuring and Communicating Quality, Accountability Quarterly, http://www.accountability.org.uk/uploadstore/cms/docs/AQ19_Lead%20article.pdf M ost Significant Change technique - ADRA (Robin Keryger). ADRA Laos, MSC Guide, July, 2004. http://groups.yahoo.com/group/MostSignificantChanges/files/ - CARE. CARE Guidelines on MSC. Ghana, 2004. http://groups.yahoo.com/group/MostSignificantChanges/files/ - Dart, Jessica. Stories for Change: A systematic approach to participatory monitoring. http://www.clearhorizon.com.au/site/papers/Dart-2000b-Stories_for_change.pdf - Davies, Rick & Dart, Jessica. The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use. (2005). http://www.mande.co.uk/docs/MSCGuide.htm - Davies, Rick. Two Years on: VSO's experience with Most Significant Changes monitoring. July 15, 2003. http://groups.yahoo.com/group/MostSignificantChanges/files/ - Exchange, a networking and learning programme on health communication for development. Most Significant Change: An evolutionary approach to monitoring that facilitates organisational learning, Exchange (UK Department for International Development) lunchtime discussion 8 October 2002. http://www.healthcomms.org/comms/eval/le02.html - Exchange, Most Significant Change, Exchange Lunchtime Discussion 3 February 2005. http://www.healthcomms.org/comms/eval/msc.html - http://groups.yahoo.com/group/mostsignificantchanges/ Online discussion group: The most significant change technique. - Guijt, Irene. Handbook on evaluation programme, policy and practice. Assessing lived experience by monitoring the Most Significant Changes. Brazil, 1998, http://groups.yahoo.com/group/MostSignificantChanges/files/ - Le Cornu, Rosie, Peters, et al, Exploring Perceptions of ‘Significant Change’ in Reforming Schools, NZARE/AARE Joint Conference, Auckland November 30th – December 3rd, 2003. ‘Learning to learn’ program, Government of South Australia, Department of Education, http://www.learningtolearn.sa.edu.au/learning_workroom - ‘Learning to learn’ program, diverse material, Government of South Australia, Department of Education, http://www.learningtolearn.sa.edu.au/learning_workroom - Nutley, Sandra & Simons, Helen. Practical postmodernism or the systematic use of anecdotes. The Evaluator, Spring 2004, http://groups.yahoo.com/group/MostSignificantChanges/files/ - Ringsing, Bettina. Learning about advocacy. Monitoring as a tool for learning in Ibis South America. MSc Thesis, August 2003. http://groups.yahoo.com/group/MostSignificantChanges/files/ - Willetts, Juliet. Most Significant Change Pilot Project. Evaluation Report. For ADRA Laos, June 2004. Institute for Sustainable Futures, University of Technology, Sydney. http://groups.yahoo.com/group/MostSignificantChanges/files/ Page 69
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 The ‘Most Significant Change’ technique User’s manual: A quick reference guide 1 10 The ‘Most Significant Change’ technique (MSCT) needs careful introduction and facilitation; this manual is designed to help in those tasks. It is intended to be a reference guide that contains the most relevant information for the application of MSCT and to be an independent yet not a standalone document: it should be used as a basis to conduct a formative workshop. This manual I have written is based on Jessica Dart and Rick Davies’s MSCT guide following very closely its structure. However it differs from Dart and Davies’s mainly in two aspects: first, it contains neither any reference to supporting information nor theoretical background111. And second, it incorporates many insights deriving from the analysis of the most recent implementations of MSCT112 and my own observations and evaluation of the method113. 1. What is MSCT? 1.1. Definit ion MSCT is a (1) subjective, (2) qualitative, (3) complementary, and (4) participatory monitoring method that begins with and revolves around the systematic (5) collection and interpretation of (6) stories of change. (1) People are asked to give their opinion and to recognise and select changes affecting a certain field over a given period of time. In this way, MSCT explicitly refuses to use objective parameters. On the contrary, it embraces subjectivism and makes it pivotal to monitoring. MSCT, instead of ticking boxes in a list or calculating ratios, opens the door to the values, judgements and standpoints of many stakeholders. It is the people who have to make sense of their reality and define the meaning of events. 110 Based on: Dart, Jessica "Self - help guide to implementing the Most Significant Change Technique (MSC)" (2004) and Davies, Rick & Dart, Jessica. “The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use.” (2005). 111 Some chapters of the ‘original’ guide have not been considered to write this manual, specifically, the sections dealing with the issues of validity and voice in the MSCT and how the MSCT compares to other approaches and epistemologies. Furthermore, this manual does not include sections studying the history of the use of the MSCT (included in the MSCT chapter of my Master Thesis) and new directions for the MSCT –as the ‘original guide’ does. Page 70
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 (2) MSCT welcomes heterogeneity and qualitative content. The method is designed not to extract facts and figures and fit them in a mould but to keep the ‘cold facts’ with their wrapping. MSCT is interested in collecting the full version: rich and deep descriptions that depict an experience and whose significance is attached114 to the story. (3) MSCT doesn’t use predetermined indicators. The absence of defined indicators makes the monitoring method more free, dynamic and adaptive. Unexpected circumstances can be more easily tackled in a more flexible context. Yet, MSCT attempts neither to substitute nor to finish with the conventional monitoring methods. Quite the opposite, MSCT tries to be a balancing tool to supplement the lack of information that may suffer traditional methods. (4) & (5) MSCT involves the collection and systematic participatory interpretation of stories of significant change. Stories are collected at the field level and then follow a path upwards the organization. MSC collects stories of change that occur at the field staff level. These changes are systematically reviewed by stakeholders of the organization, with each group (the field staff itself, management and Headquarters) selecting the most significant change to pass to the next level. Participation arises from the essence of the concept: collection and interpretation is done by the people directly involved in the changes. (6) Finally, stories constitute the whole kernel of the process. And more specifically stories that describe significant change. 1.2. The heart The starting point and central part of MSCT is an open question: ‘Looking back over the last month, what do you think was the most significant change in the quality of people’s life in this community?’115 The question forces the respondent to articulate a rich yet focused answer. Breaking the question in its different parts makes it easier to understand how this happens: 112 This guide is the result of the analysis of the reviews, assessments and opinions of those who have used the MSCT to monitor the impact of their projects –and from my own judgement. They expand the original function and reveal welcomed benefits. Australia, Bolivia, Ecuador, Brazil, United Kingdom, Laos and Ghana is the geography in which we will be moving; Rick Davies, Jessica Dart, Ibis Denmark, Voluntary Service Overseas, Oxfam, The Australian Department of Education and Care International some of the ‘users’. 113 Thus, this manual is a quick reference guide, but also a critical evaluation emanating from the study of recent applications and a personal interpretation of the method. 114 This aspect will be extensively discussed in the following sections. 115 This example is taken from the 1994 CCDB Bangladesh project. It is a ‘basis’ question which parts can be adapted to ask for more/less/different input. Page 71
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 (1) ‘Looking back over the last month,’ The respondent has to limit the answer to a given period of time. –A month, in this example. But, obviously, it can be a quarter, a year or whatever time spam best suits the monitoring purposes. (2) ‘what do you think was’. The interviewee is not asked to search for objective facts; he/she must exercise their own judgement and assess the situation under their own perspective. (3) ‘the most significant change’. The evaluation has to be selective. He/ she cannot comment on everything that has happened. The respondent has to focus exclusively on changes, analyze them and select that single one that, under his/her eyes, is the most significant one. (4) ‘in the quality of people’s lives’. The interviewee has to narrow even further his/her response. The answer cannot refer to significant changes in general terms but must address a specific topic, in this case, ‘the quality of people’s lives’ – This is what is called the ‘domain of change’. ‘Domains’ should be adapted to objectives to be met such as ‘the learning process’, ‘professional life’, or ‘sanitary system’ (5) ‘in this community’. Finally, in the same way that there is a temporal boundary, there is also a physical frontier. The respondent has to limit his/her answer to a precise framework. 2. Implementation steps MSCT follows a methodological and systematic implementation process. This process can be separated into nine different steps. Not all of them are essential; some of them are discretionary and simply provide consistency to the methodology. However, there are three steps that cannot be avoided: the collection of significant change stories (4), the selection of the most significant of those stories by at least a group of stakeholders (5) and the feedback of the selection and the reasons behind to the relevant people (6) The steps to implement MSCT are the following: (1) Getting started (2) Defining the domains of change (3) Setting the reporting period (4) Collecting significant change stories (5) Selecting the most significant change story Page 72
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 (6) Feeding back (7) Verifying the most significant stories of change, and (8) Conducting meta-monitoring 1. Gett ing started The successful implementation of MSCT rests upon a truly open culture: one that believes in trying new approaches and that is not afraid of failure. The organization must have a desire to try innovative and unconventional approaches that encourage new learning. Similarly, the organization must also understand how to benefit from failure and use it as a trigger for improving future performance. 1.1. The Champions, owne rs hip and manage men t Such a culture doesn’t blow in the wind of the organization; it is embodied in the people. Yet some people really incarnate the values of an organization; they are the champions. MSCT needs dedicated champions to succeed: influential, motivated and proactive people that understand, support and promote the use of the method in the organization. Thus, MSCT demands the commitment of those with the power to decide and the enthusiastic involvement of a few people within the organization to inform and motivate. The contribution of senior staff is determinant at the beginning to excite and motivate people but also to clarify ownership and management116. In order to do so, senior staff must appoint a MSCT management team (or an individual) to lead the process. MSCT management team117 must have a sound understanding of MSCT to fulfil its multiple roles in the process: (1) Answer questions about the technique. (2) Prepare and facilitate workshops to build capability of the staff. (3) Maintain the momentum and enthusiasm to permanently encourage the collection of stories. (4) Ensure the systematic collection of stories. 116 The question of ownership and management were key issues in several projects: Ibis in South America, ADRA in Laos or Guijt's work in Brazil. 117 The MSCT management team can be formed of staff members or hiring a external consultancy providing a team that works closely with an internal champion. Page 73
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 (5) Facilitate and prepare the collective workshops in which the stories are to be selected. (6) Conduct secondary analysis. 1.2. Clarificat ion of purposes The most important thing to do at this stage is to convey a direct and simple message. First, a great emphasis should be placed on the easiness of the method: MSCT does not require any theoretical knowledge of monitoring or organizational learning whatsoever and can, therefore, be very easily used by anyone. Second, the purpose of MSCT has to be clearly stated. Enough time has to be spent on explaining and clarifying what objectives are to be met and how they relate to the rest of the organizational goals. And the best way to do so, it seems, is in a proactive workshop. 1.3. Some comme nts on building staff capacity From Australia to Ecuador to Laos participants in MSCT projects express a deep and quick positive impression of the method after their first experience. Yet the beginnings can be demoralizing: scepticism and lack of interest are the main obstacles that can arise at the start. One way to overcome them is to have a champion disseminating the technique across the organization. Another way, complementary to the first one, is to adequately train the staff to guarantee the effective use of MSCT and thus assure promising results. Having sufficient time and resources, a workshop followed by a hands-on training period in which participants can experiment with the method in a pilot project is the preferred approach. However, the conflict with other professional duties usually forces to reduce the initial training of the staff to a weekend workshop. The rest of the learning occurs as the project progresses. Up to date, in most of the cases a one-to-three day workshop preceded the implementation of MSCT and a second ‘refresher session’ was organized in the middle of the process. Facilitators, usually MSCT manager, steered the workshops. Some lessons can be extracted from those workshops: (1) It is very effective to use a lot of practical exercises: a) ‘Exchange’, hosted by Healthlink and supported by the UK DID chose the learning-by-doing option118. 118 DID (Department of International Development) .Participants did a short exercise: “Turn to your neighbour and ask them: • What was the most interesting thing that happened to you yesterday? (Get the details) Page 74
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 b) Ibis Denmark used meta-plan cards119 to define the expectations of the participants. (2) It is equally important to drive participants through the complete process of participatory selection of the most significant story120. (3) Time for frequent questions and discussion121,122 is essential. (4) It helps to better understand MSCT to compare it to other monitoring methods and case-study methodologies. • Then ask them why they thought this was the most interesting. • Then let your neighbour ask the same questions to you. • Come to a decision about which of the two stories you think is most interesting, and identify why you both think so. • You may have a number of reasons. • When asked to, tell the whole group the story you chose, and why you did so.” 119 The meta-plan technique is a visualization and systemisation method based on the use of written cards. It is usually used as a tool to help participants structure their expositions, as well as offering conceptual support; for the presentation of new subject matter (in order to structure and to determine participants’ previously acquired knowledge); especially at the beginning of a class, seminar, etcetera (“warming up”) and at the end of a workshop for self evaluation. However, this is not the only technique. The International Facilitators Association offer dozens of them in its website: http://www.iaf- methods.org 120 See appendix: Dart and Davies’ ‘facilitation guide for store collection’ 121 Jess Dart, in her 2004 MSCT guide suggests some questions for most of the steps: (1) How will you get ‘buy in’ from the people who will be involved in creating/selecting SC stories? - Where can you begin – is there a small pilot that you can test first? - Who are the best people to capture the first SC stories from? (2) Will you use domains? - Who will select domains? Top down/bottom up? - How will the domains be selected? (4) Who will tell the SC stories? - Who will collect the SC stories? - How will they be collected? - How/when will they be documented? - How will you ensure that the collection process is ethical? - How often will SC stories be collected? (5) Who would benefit from reading & selecting SC stories? - How could the selection process work? - Map out a possible structure: - How will you select the SC stories? - How will you make sure that everyone in the group is happy with the choice? - How/ who will record the reasons for choosing the story? (6) Who needs to get feedback on selected SC stories and reasons for selection? - How will feedback be communicated? - What will the feedback cover? (Comments on all SC stories or just those selected?) (7) Will you verify any of the SC stories? If yes – which SC stories? If yes – what aspects will be verified? - Who will verify them and how /when? Page 75
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 2. De fin ing the domains of change Domains are areas of change usually related to key goals of an initiative, a project or an organization as a whole. Jess Dart defines domains of change as ‘broad and often fuzzy categories of possible changes’. And they can be just that, but they can also be far more. As a matter of fact, most of the organizations that have already implemented MSCT opted to explicitly match the selected domains with chief areas of activity123. More interesting, however, is to link the selection of domains not to the organization’s activities but to specific organizational objectives, thus making the selection process a key strategic decision by aligning monitoring purposes and organizational goals. In that sense, learning about the changes would automatically mean learning about how those goals have been met –or missed. The domain selection can serve multiple purposes but, in any case, the choice should be determined by what the organization wants to learn about. 2.1. Type s of domains As a result, each organization will establish the domains of change that best fulfil its learning needs. Domains have aimed very frequently at learning about changes in the lives of individuals but about many other aspects as well, like the sustainability of institutions (CCDB), mutual partnerships in collaborative projects (Oxfam New Zealand), organizational performance (Ibis Denmark) or for different levels of management to learn about what’s happening in the organization (Brazil). (8) How will you monitor and store the SC stories? - If you will use a spreadsheet – what information will you record? - Who will do this? 122 Rick Davies compiled a collection of questions that emerged from the assessment of VSO’s use of the MSCT (Davies, Rick. Two Years on: VSO's experience with Most Significant Changes monitoring. July 15, 2003). These questions are the result of a policy review and are not specifically designed for an initial workshop. However, I consider that they cover many aspects that may very well come up in the mentioned workshop. Thus, it can be useful to have in mind some of them when conducting the workshop: - How not to be biased by the way in which the story is written? - Are we looking for representative or exceptional stories? - There is a lack of a clear understanding of what the MSC stories are for, at each level (volunteer, programme staff and VSO corporate). - Lack of ownership and understanding of the MSC process at the country programme office level - Why does the Significant Change story have to be 'pure' and unedited?, Surely it’s the change that is important not the way it is written? Can't we edit and add background and answer some of the missing information? - When and where volunteers should write the MSC stories? 123 For instance, ADRA Laos defined 4 domains of change: Changes in people’s health, Changes in people’s behaviour, Changes that reflect an area to improve (negative), Changes in any other area (open). Page 76
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 It has also been a common practice among organizations to define an open domain: a domain that refers to changes in any other area than those selected. This is an open door for story collectors to record (and for storytellers to write about) those changes that don’t fit in any of the categories but that, nonetheless, are relevant and worth being registered. Even in some cases, domains have not been defined at all at the beginning but instead as the process was going on, allowing them to ‘come out naturally’. This option has the potential to disclose truly unexpected change, as no limitation is imposed on the collection of stories other than broad ‘change’. However, this alternative might result in a too-big unclassified number of stories of change. Besides its far-reaching strategic purposes, the selection of domains fulfils one basic immediate function: domains serve as categories to group large number of stories – objective that is not accomplished if there are no domains, obviously. Finally, it is also useful to define a negative domain, a domain that accounts for what goes wrong. These years of MSCT implementation have seen a tendency for people to report mostly only the positive sides of projects leaving very little space for criticism. But more than often, it is as least of the same importance to learn about what is not functioning and the reasons behind as to know about what goes well. Furthermore, lack of self-criticism has been reported as a major flaw of MSCT in some of its applications124. Encouraging the eliciting of negative stories can be really beneficial for the organization: It is a clear-cut manner to identify what the program/project at hand should but is not addressing from the perspective of the people within the organization, beneficiaries or partners. 2.2. Sele ct ion of domains: Who and How It has been said that domains define the perimeter of what the organization wants to learn about. Therefore, those who select the domains decide what should be learned. It makes sense that senior managers decide the domains, if the domain selection is to be linked to strategic objectives. Yet in a participatory method like MSCT this decision should not be left to senior managers alone: partners, middle management and the staff directly involved in the monitoring process must also have a voice in the selection of domains. Furthermore, the active participation of multiple stakeholders legitimates the decision and assures consistency with the overall goals. There are many procedures to come to an agreement over the most suitable domains. A top-down decision is probably the fastest and easiest way to decide the 124 Ibis Denmark in South America, Guitj’s work in Brazil. Page 77
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 selection of domains, yet it is too the less democratic one–or, at least, the one that requires less participation. A bottom-up process guarantees that the input of the field staff will be taken into account (however, such process should be carefully designed in large organizations). Dart and Davies advocate for the Delphi technique125. This technique calls for the consultation of multiple stakeholders (staff members and management, but also partners and donors) using an iterative survey and feedback approach to allow participants to express and review their opinions based on the answers of others until a satisfactory consensus has been reached. This technique is very useful when many people are involved in the selection of domains and/or they cannot physically meet for discussion. In the use of MSCT in the Australian ‘Learning to learn’ program, the participants designed a ‘light’ version of the Delphi technique adapting it to their less-demanding needs of their specific project. 3. Setting the reporting period Defining a reporting period is important to draw a timeline and set a horizon. However, establishing ‘a priori’ the optimal reporting period is simply not possible. MSCT, as it is being described in this manual, provides the basis for a participatory approach to monitoring. Yet, the ‘standard format’ does not exist. Each organization has to adapt and modify the method to its own needs and purposes. Therefore, each implementation of MSCT holds particular characteristics very closely related to the nature of a project and specific organizational goals. In that sense, each application of MSCT is unique, and so is the reporting period. Having said that, some observations can be highlighted based on previous implementations: (1) Higher frequency usually translates into higher learning of the participants. Nevertheless, too higher a frequency can lead to a costly time- consuming method and exhausted participants. (2) No matter how detailed planning is, frequency is normally higher at the beginning and decreases as the process develops. (3) Although many organizations have adjusted the time to collect stories to already-in-place quarterly reporting periods, some organizations choose to report monthly and others to do it annually, depending on their specific needs. 125 Initially conceived as a technology forecasting method, the Delphi technique is widely used today in many sectors as a tool to reach consensus about an opinion or view. This technique, if used effectively, can be highly efficient and generate new knowledge. http://www.is.njit.edu/pubs/delphibook , http://www.iit.edu/~it/delphi.html Page 78
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 4. Colle ct ing ‘s ign ificant- change’ storie s ‘Looking back over the last month, what do you think was the most significant change in the quality of people’s life in this community?’ The open question previously studied is the key instrument to extract significant change stories. As we are beginning to see, all revolves around this question and all is condensed in it. The richness of the assessment that derives from the monitoring process depends on the quality126 of the stories that the question generates. The collection of stories identifies significant-change stories within each domain. The collection of stories entitles several important aspects: whose stories and what information to collect, what method and reporting format to use and issues about confidentiality. 4.1. Whose stor ies to collect The monitoring purposes serving organizational goals determine the selection of not only domains of change but also storytellers. The objectives to be met decide the appropriateness of the people. If an organization wants to learn about the impact that its programs has on people’s lives, the storytellers by default will be the field staff collecting the stories of the beneficiaries127. If an organization wants to measure the quality of its partnerships, the narrators will be the staff members of each one of the partners. If, finally, an organization needs to asses the firmness of its values, every stakeholder will be given the opportunity to write his/her story. 4.2. What information to colle ct 126 Here quality is considered under the light of the adequacy of a story to the monitoring objectives and does not refer to whether a story is well-written. Many participants have expressed their concern about the issue that well-written stories usually get selected although they are not necessarily the ones that better represent most significant changes. That is a bias certainly difficult to overcome. 127 The term ‘Beneficiaries’ refers to anyone ‘targeted’ by a program. Davies points out that, although stories narrated by beneficiaries are usually rich and valuable, they are difficult to elicit. Ideally beneficiaries should share, select and report their stories but very often it is not feasible to provide them with the training to do so. One exception, however, is the ADRA Laos experience. In addition, it must be said that when training beneficiaries is easy to do or simply is not a need, beneficiaries should be the storytellers, e.g. when an educational organization wants to asses the impact that an education program has on the professional/academic life of the students. Page 79
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 At a minimum, it should be collected the description of the story itself, the significance of the story for the storyteller and information about who collected or wrote the story and when. (1) The description of the story: The story itself should be a brief and concise (typically no more than two paragraphs) description of events –what happened, what and where. It is important that the story is documented as it is told (to keep the judgement of the storyteller) when the person documenting the story is other than the storyteller him/herself. Besides the story itself, asking for additional information enriches and facilitates the later analysis: a) A title: A headline forces the storyteller to concentrate his/her thoughts in one single sentence and makes the subsequent classification and selection of stories easier. b) Recommendations/lessons learned: Recommendations give the storyteller the opportunity to suggest courses of action and have the potential of bringing fresh perspectives to the organization. Comments about what the storyteller has learnt help to draw out implications of the story. c) The influence of the organization: Reflections on why the change has happened and the direct role that the organization has played can provide priceless information that serve as a basis for a more detailed and comprehensive analysis in the next step (selection of the most significant story). (2) The significance of the story for the storyteller: This is a crucial part of MSCT. On the one hand, the storyteller has to make sense of his/her own story. On the other hand, that meaning is made explicit and thus unveils the importance of the story –otherwise, the significance might not be evident. (3) Information about the storyteller: And, finally, it is important to record the basics about who, when, in what context. 4.3. Reporting format s and methods An uncomplicated standard format helps to easily collect the information128. The format facilitates the categorization and classification of the information that needs to be collected. Although in many occasions the ‘first collection’ is done using physical formats (a sheet of paper, a recording tape), it is desirable for ulterior analysis to transfer the information to digital media (spread sheet, word editor, online database…). 128 The appendix provides several examples of formats used by different organizations. Page 80
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 The availability of resources (time and people, basically) and the amount of effort that the organization wants to place in MSCT inevitably decides the choice of method. The more active the collection process the more demanding and resource- consuming it becomes for the organization. Three methods are preferred: (1) Interviews: They are a good way to capture stories from selected people within the organization. Interviews probably lack the reflection of a written discourse but have the advantage to trigger new thoughts through dialogue. It is recommended to follow a semi-structure script and record the thoughts of the interviewee as they are expressed. (2) Group discussion: Sharing stories in a group multiply the interview ‘trigger effect’. As an example, the ‘Target 10’ Dairy program group discussion provided numerous and valuable stories. (3) Individual questionnaires: Handing a pro-forma questionnaire to individuals lacks the pros of interviews and group discussions but exhibit important advantages: It allows for more thoughtful stories, individual voices are easier to identify and usually generates richer and thicker descriptions. 4.4. Ethics The reason for the collection of the stories as well as how the stories are going to be used must be very clearly explained to the storytellers. They must also be asked for explicit consent if the organization intends to use the stories later on for whatever purposes. 5. Selecting the ‘most sign ifican t change’ story 1 2 9 Diagram of MSCT selection process Most significant change stories (1 MSC story per domain) Selection level 2 MSCS Committee 129 The selection of the ‘most significant change’ story (MSCS) is the consequence of a discussion group meeting. The group must plan and estimate the time available in advance to guarantee the success of the meeting –that a single MSCS per domain is selected and documented. To make it feasible, no more than 10 stories per domain and per session should be discussed at the lower level. The facilitator(s) play a determinant role in these meetings. Page 81
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Selection level 1 Selection level 1 Selection level 1 MSC Group 1 MSC Group 2 MSC Group 3 (1 MSC story/domain) (1 MSC story/domain) (1 MSC story/domain) Stories from storytellers (1 MSC story per storyteller) Significant change story per domain The chart represents the pyramidal-like selection process of the ‘most significant change’ story (MSCS): Four domains of change have been selected in this example. E.g.: capacity building, communication policy, improvements (negative domain) and an open domain. (a) Storytellers collect (write) their respective MSCS within a given domain and submit their stories to a selection group at the next level. (b) Each group selects one MSCS per domain, resulting in 3 MSCS per domain (3 different groups haven been appointed in this exmaple) and submit the stories to the MSCS committee. (c) The committee selects a definitive single ‘most significant change’ story per domain. At each selection level, feedback is provided to the lower level. Dart and Davies call this process ‘summary by selection’: It is a (1) hierarchical, (2) participatory, (3) iterative and (4) documented process in which each level selects the most significant of the stories submitted by the lower level and passes it on to the next level: (1) Hierarchical: It responds to a hierarchical structure because it is designed in such a way that upper levels decide over decisions made by lower levels. (2) Participatory: Participation results from the wide involvement of people across all organizational levels –from field staff to middle and senior management- and from the fact that every selection process is not done by single individuals but by a group of people. (3) Iterative: Selected stories at one level are reviewed again (and selected or not) at the next level. Therefore, one story selected at the lower level does not go Page 82
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 directly to the top but must overcome subsequent filtering processes. Furthermore, the iterative filtering progressively reduces the original block of stories (until a single one per domain is selected) making the selection lighter and easier for the upper levels. (4) Documented130: This is one of the most important characteristics of the whole process. As we may recall from the previous step, each storyteller makes sense of his/her story writing about the significance of the story from his/her own standpoint. This is required to elicit the subjective significance of the story. In a similar way, each group justifies its selection(s131) by discussing, agreeing upon and finally attaching to the selected story a comment that explains the reasons for the selection and its significance for the group. Therefore, a mere ‘this story has been selected because’ is not sufficient. The group must argue why the story has been selected linking the argumentation to the monitoring and organizational objectives under assessment. The group must also reflect on recommendations that address the reasons for change, the role played by the organization and the implications for future actions. Thus, each level (except the first one) receives the selected stories with at least two ‘significance reflections’ –the one from the storyteller and the one from the previous level selection group. This documentation becomes of the utmost relevance at least for three reasons: a) Because it is a guarantee for transparency and accountability. Documentation demands a thoughtful procedure and represents an obstacle to arbitrariness. b) More importantly, it is the only evident and explicit link in the complete process between documented impact and strategic decision. Without this documentation phase MSCT runs the risk of simply identifying changes -but not knowing then what to do with the results. c) It opens strategic planning to all the levels of the organization giving a voice to everyone and testing what has been decided at senior levels against what lower levels consider important. 5.1. Sele ct ion of the most significant change story: Who and How Just as the procedure for the selection of domains reflects the true participatory spirit of the organization, the selection of stories must be open to a wide range of actors across the organization. At the minimum, the selection should 130 Ringsing argues that not only the selection arguments should be documented but also the discussion to reach consensus as well –the own process of looking for an agreement. 131 In the common case that there is more than one domain of change. Page 83
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 be done by those with responsibility over the storytellers, closest relationship with them or directly involved in the program that affects them as beneficiaries. At its best, everyone in the line of decision of the specific program/project should participate in the selection process. In any case, selection is easier when storytellers don’t have to decide over their own stories. Selection criteria and methods132 are diverse and numerous. The preference for a specific method will depend on aspects such as the size of the group, the time available and the necessity to reach a consensus. Yet despite the chosen method, some conclusions can be drawn from experience: (1) This step in the process always starts with a group in front of a pile of documents that has to be reduced to a single story of change per domain. (2) The first decision to take is whether the selection criteria and methods are to be decided before or after reading the stories. The first option speeds up the process but can condition in some way the selection. The second option makes the process more open to unpredicted results, hopes for the selection method to surface from the discussion and is favoured by Dart and Davies as the optimal one to aid learning. (3) Regardless of the previous comments, the discussion group workshop always follows a similar pattern: a) Everyone reads the stories. b) All stories are read aloud and listened. c) The group opens a discussion to select the most significant story of change per domain. d) The group makes a decision. e) The decision is documented and justified. What to do with the stories that have not been selected All stories that have been discarded must be documented and kept. Each story should be attached a comment explaining the reasons for refusal and stored for future analysis –specifically systematic content analysis, see step 8. 6. Feeding back the result s of the se le ction process to all stakeholders 132 Unanimity, majority and minority voting, iterative voting, scoring, pre-scoring (and then group vote) and secret ballot are among the methods suggested by Dart and Davies. Page 84
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Feedback is the weakest knot in any monitoring process133, considered in many cases as an add-on to the truly significant decision-making phase. Many organizations that have applied MSCT have had a negative experience with the feedback stage. Organizations, although aware of its relevance, very often failed to provide adequate feedback. A comment134 made by a participant in a MSCT training workshop organized by Ibis Denmark in October 2004 clearly illustrates the problem: ‘Downward accountability is called feedback –lucky you if you can get it.’ However, it seems evident that consistently feeding back the results to lower levels has the potential to produce long-term benefits: It shows commitment and involvement at all levels and fights the undesired feeling of one-way upward accountability. It improves organizational communications, promoting intensive dialogue up and down the hierarchy of the organization. And it makes judgements explicit to all participants, encouraging reconsideration of previous decisions. 6.1. Content and form A comprehensive feedback to lower levels should include: (1) The stories that have been selected for each domain. (2) The reasons for their selection, linking them to the organizational objectives to be met, and (3) The procedure for the selection. From face-to-face verbal communication to formal reports, many channels are open to effective feedback. Nevertheless, it is important to keep two considerations in mind: First, a timely response is sometimes as important as the content of the message; lower organizational levels will be anxious to know the results of the selection process, even more if they know when the decision has been taken. Therefore, an effort should be made to communicate agreements as soon as they have been reached. Second, formalized feedback ensures the easy and handy access to decisions in the future. This might prove especially useful to track the progression of the organization. Verbal feedback is not feasible in most of the cases for obvious reasons135. Emails, community newsletters posted in the organization’s intranet/blog and 133 Monitoring processes are not the only ones to suffer malfunctioning downward feedback. Poor downward communication is a malaise in almost every organization. 134 Page 35,Davies, Rick & Dart, Jessica. The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use. (2005). 135 And, besides, if results are fed back only verbally, recording the whole feedback process might become a difficult task, probably missing important insights. Page 85
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 written formal reports appear as optimal means of communication: A brief email sent to the people involved in the process, soon after the selection decision has been made, reinforces the sense of participation. A community newsletter in an open and active forum creates a productive mechanism for constant dialogue. And the publication of all selected stories under one cover is a useful catalogue which may serve many practical purposes –from its use as marketing material to communication tools. 7. Ve rifying the most significant storie s of change Verification of stories of change is not an essential step in MSCT. Certainly, there might be no need at all for verification in small organizations or small-scale projects where the veracity of the changes is easy to check. Nevertheless, follow-up structures can serve two important purposes in large organizations. On the one hand, story-checking can provide an argument for accountability to external parties such as partners and funders -a solid verification process brings legitimacy to MSCT. On the other hand, such a process can also function as an incentive for storytellers to improve the quality of stories; they might place a greater effort in the accuracy and description of events when they know that the changes described in the stories might be double-checked. Yet verification can have undesired consequences and demands a supplementary organizational effort. Dart and Davies mention a testimony recorded by Sigsgaard during the implementation of MSCT in Mozambique: ‘…We found that the word ‘verification’ should not be used in external communications to refer to such further investigation (follow-up of the stories of change). The word was too much connected with control’. A careless verification management can originate distrust among leading to discouraging effects. Moreover, and as it has been mentioned, a verification management process needs to be put in place deciding who is to perform that role and what and how is to be checked. ‘What’ should be limited to the most significant change stories per domain: attempting to verify every selected story is too time-consuming –and, probably, useless. Additionally, only the truthfulness of the change described in the story should be object of verification; interpretations (significance of the story) cannot be false. ‘Who’ should come up from the members of the group that selected the most significant stories of change, perhaps the one more directly involved in MSCT. And ‘How’ the actual verification happens should be similar to the way in which the selection of significant stories of change is documented136. 136 See page 38. Page 86
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 8. Conducting meta- mon itoring 1 3 7 The purpose of this complementary step is to keep record of all the stories that have been collected within the organization despite whether they have been selected or not. Why take the trouble to do that? The easy answer is that all collected stories are already there, so classifying and putting them aside is not much of an effort. The more satisfactory answer relates meta-monitoring of stories of change to building a purposeful database that facilitates the access to relevant information about policies’ impacts, shows the efficacy of the monitoring method and reflects the progression of MSCT. Meta-monitoring, ‘the monitoring of the monitoring’, focuses on the examination, categorization and study of the attributes of the stories and potential to play a key role in organizational planning. If MSCT is to be used as an effective tool informing strategic decision-making, then the most important information to keep track of refers to the impact of the stories. In particular, meta-monitoring should record whether the selected most-significant-change stories and the recommendations attached to them have had an impact on organizational policies. Keeping track of this sort of information helps to not only asses the effectiveness of on-going projects/programs but also whether the organization is learning from the insights that provide most-significant-change stories and effectively incorporating that learning into its new actions. In addition to providing the basis for such fundamental information, meta- monitoring can also be used to trace the evolution of stories and storytellers: how many stories are written in each reporting period, who is writing them and whose stories are being selected. These facts and figures provide evidence about the real acceptance overtime of the monitoring method among different organizational levels at the same time that helps to find a paradigm for successful stories. 137 Dart and Davies recommend to not only ‘meta-monitor‘ the process but also to conduct a secondary analysis as a part of a ‘summative evaluation’ (SE). SE is concerned with a program's overall effectiveness: it is normally used to determine the extent to which a project met its goals and whether the project's accomplishments are the result of the services/activities provided. It can be very useful to asses both the outcome of a program and ways to improve it. Although SE demands the use of statistical information and research and analysis skills (which in practice can result in the work of a specialist) the results of the analysis can be very useful for further enhancement. Page 87
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 The ‘Most Significant Change’ technique Appendix Story collection formats ‘Significant-change’ stories Case study Facilitation Guide for story collection workshop Page 88
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Story collection formats 1. Learning to learn . Australia, 2003 1 3 8 Storytellers: Students and teachers Title of Story: Name of Person Narrating Story: School / Preschool: Date: Domain (select one): Student engagement: Student outcomes: Teacher/leadership engagement: Classroom pedagogy: Whole school reform: What Happened? Why do you think this was a significant change? What evidence do you have? How do you know that this has made a difference? Please sign to approve use of your story in Learning to Learn’s documentation. 138 http://www.learningtolearn.sa.edu.au/learning_workroom Page 89
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 2. CARE GoG MSC FORMAT. Ghana, 2004 1 3 9 Storytellers: Field staff Name of Project: Title of case study/story (Give your case study a name, like a newspaper headline): Name and Location of individual/group/institution concerned: Date/Month in which it occurred: Theme for the story: (State which theme out of: 1. Empowerment/livelihoods, 2. Institutions, 3. Power relations 4. CARE innovation/process 5. Other) Describe the Change that occurred. (one or two paragraphs, be concise and specific – what was the start situation, what changed?) Give the significance of story (Explain WHY you chose this story e.g.: WHY it is important for the project/programme, HOW the event relates to the theme, WHY this change is of more significance than any others that occurred for this theme) If there is any relationship to the project Log Frame then state what this is? (E.g. the event provides information relating to project outputs, outcomes, indicators, impact, assumptions etc) Recommendations (What should the project/CARE do differently because of this story, and who should do it) 139 Page 4. CARE Guidelines on MSC. Ghana, 2004. Page 90
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 3. Landcare (The Mallee Landcare Support St rat egy) 1 4 0 Storytellers: Field staff Background: The Mallee Landcare Coordinators and Facilitators would like to capture stories of significant change that may have resulted from their work with Landcare in this region. This will help us to improve what we are doing, enable us to celebrate the successes together as well as being accountable to our funders. The stories and information collected from these interviews will be used for a number of purposes including: (1) To explore what Coordinators and Facilitators together with Landcare groups in the Mallee have achieved already (2) To help Facilitators and Coordinators understand what people in Landcare value, and support more of these sorts of outcomes (3) To acknowledge and publicise what has already been achieved. Contact details: Name of storyteller*: Name of person recording story: Location: Date of recording: * (If they wish to remain anonymous, don’t record their name or contact details – just write ’landholder’ or some similar description.) Questions: Tell me how you (the storyteller) first became involved with Landcare and what your current involvement is: From your point of view, describe a story that epitomises the most significant change that has resulted from Landcare in the Mallee Why was this story significant for you? How, (if at all) has the work of the Landcare Facilitators and/or Coordinators contributed to this? 140 Page 92,Davies, Rick & Dart, Jessica. The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use. (2005). Page 91
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 4. AD RA FORM AT. Laos, 2004 1 4 1 Storytellers: beneficiaries MSC Story of Change General Information Project: Domain: Person reporting: Date of report: (date/month/year) Name of Village and Name of District? Information from Villager What is your name? What is your age? Topic of change: When did the change happen? (Approximate month and year) What happened? ¡ How did the change happen? Why did the change happen? Did ADRA help with this change? (Yes or No) If Yes, how? Is this change important to the villager? Why? Information from Staff What are your observations about the change? Why do you think this change is important? What should ADRA do based on this story? Management Information PMC selected or not selected: Why did the PMC select this story? PMC recommendations: VTE selected or not selected: Why did the Vientiane committee select this story? VTE feedback on PMC recommendations: 141 Page 23, ADRA. ADRA MSC Guide, July, 2004. Page 92
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 ‘Significant-change’ stories 1. CARE GoG. Ghana, 2004 1 4 2 Storytellers: Field staff Name of Project: Wenchi District Livelihood Security and Natural Resource Management Project Title of Case Study/Story: Agubie and Asuogya Zone B save millions of cedis Name and location of group: Agubie in Wenchi District-Brong Ahafo Region Date/month in which it occurred: January - March 2003 Theme: Promoting inclusive civic and individual empowerment Agubie is a small community in Wenchi district with a population of about 500. Bush fires have been a recurrent nuisance to the residents of Agubie for over ten years. Indeed for ten years, annual bushfires during the dry season had burnt community farmlands, rendering them infertile and unsuitable for agricultural production. Farm produce, forest produce and wildlife was burnt each year resulting in increased poverty. The devastating effects of bushfires notwithstanding Agubie community had no concerted effort to combat bushfires. Community members felt that fire prevention and control was the ‘business’ of the community fire volunteer squad and hence there was little or no participation of the wider community including the traditional authority in bushfire management. CARE’s WEDLAN project assisted Agubie to develop a plan to combat bushfires. Under this plan fire volunteer squads of Agubie and neighbouring communities collaborated to undertake anti bushfire educational campaigns and to mobilize the entire community to put out any fires that broke out. This strategy was successfully implemented and during the dry season of 2002/2003 there was zero incidence of uncontrolled bush fires in Agubie and surrounding communities. Farm produce worth hundreds of millions of cedis was protected. According to one farmer “in the previous years I had to convey my maize to Wenchi for storage but this year I stored it in the community and saved the money I would have spent on transport. I have that money to take care of my family”. He also said that farm produce such as cassava, plantain, cocoyam etc. that 142 Page 6. CARE Guidelines on MSC. Ghana, 2004. Page 93
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 was normally left unharvested (due to poor storage properties) on the farm until needed for food did not get burnt. In the past such produce would have been destroyed by bushfires. Significance of the story (Project Facilitator): Ability of Agubie fire squad to organize the zonal fire volunteers structure which was not functional towards a common goal Ability of the group to take an informed decision on all inclusive bush fire prevention and control High commitment and enthusiasm exhibited by the firer volunteers, community leaders and the entire members of the communities Ability of the people to take charge of their development Significance of the story (Project Manager): The story shows the success of a community based initiative to combat uncontrolled bush fire with limited external input. The success achieved was due to concerned communities’ own collective effort – there was no need to call the Ghana National Fire Service. Also of significance is the fact that combating those fires that did start was not left to the fire volunteers alone, the general community got involved. This is a demonstration of improved community organization and community ownership of the bushfire management strategy developed. Page 94
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 2. AD RA. Laos, 2004 1 4 3 Storytellers: beneficiaries ADRA Laos MSC Pilot Project Domain: CHANGES IN PEOPLE’S BEHAVIOR Villager or Observation: Story reported through interview with villager. Project: ACTION Health Person reporting: Phetduangchan Date of report (Day/Month/Year): 12 November 2003 Name of Village and Name of District: Boungvai village, Saisettha District What is your name? (Mr./Mrs. first name and Mr. Bounthieng family name) How old are you? 44 years old Basic idea of change: Medicine better than sacrifice. WHAT happened? Using medicine WHEN did the change happen? (give a Since 2003 specific month) Details of the change: (HOW and WHY did Before when my family got sick we had to do the change happen?) sacrifice of animals. Because we had health education, when my family gets sick we will go to the hospital instead of doing sacrifice. We can save our animals to do other work. How did ADRA help with this change? Provided health education Only for stories from villagers. When we use medicine it’s better than sacrifice Why is this change important to the because we can keep our animals. We don’t VILLAGER (person in story)? have to kill a buffalo that is worth a lot of money. As a project staff member, why is this change Before they believe in spirit, and they don’t important to you? Is it positive or negative? know about disease. PMC reason for selection: The villagers use medicine and go to the hospital. Vientiane Committee reason for selection: This story is about a very big change in beliefs as well as behavior – it shows that the project is having a big impact for this person. ADRA Australia reason for selection: We felt this story showed how ADRA's intervention demythologized their religion and promoted a more economically viable medical alternative. 143 Page 18, ADRA. ADRA MSC Guide, July, 2004. Page 95
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 3. Bougainville: Osi Tanata (NGO) 1 4 4 Storytellers: beneficiaries Significant Change Story Do you the storyteller: • want to have your name on the story (tick one): Yes o No o • consent to us using your story for publication: Yes o No o Name of person recording story: Wilson Name of storyteller: Sebastin Kakau Project and location: O’Kerry organic project – cycle 3 Date of recording: 23th of March When did it happen: Over 1 year Title of story: “Growing big” Domain: Changes in people’s lives Tell me how you (the storyteller ) first became involved with Osi Tanata, and what your current involve me nt is: I used to be a member of a community project. But I Ieft the community project in anticipation of disputes that might occur within the community project. However, upon hearing that Osi Tanata was giving training to grass roots, I attended some of the Osi Tanata training of project management and book keeping and TOT. From your point of view , describe the most significant change that has resu lted from your involveme nt with Osi Tanata (training/support or funding) After the training I went back to my village and mobilized my family members to venture into organic gardening. I decided to set up my own family project on organic gardening. Despite not having funding from any agency I ventured into setting up this small project with only the knowledge that I got from Osi Tanata. We set up our organic garden growing cabbages, capsicums, greens, tomatoes, aibika, chillies and other things. 144 Page 97,Davies, Rick & Dart, Jessica. The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use. (2005). Page 96
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Currently I am thankful for what I learned from Osi Tanata, and am using it. Today my project is progressing well. We have sold many of their produces from their organic farm. For example, for a bed of cabbage, he is getting around K100. Now they have spent the money to buy clothes and many other basic needs. Apart from generating income the families and the surrounding villages have enough surpluses to feed their family and others. Also some of the money is being used to start other projects such as a trade store. Why is this significant to you? It is significant to me because at first I had no knowledge to run a project. Today I have a good project running and the income from this project is being used to sustain the livelihood of my family. Page 97
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Case study 1 45 CARE Bangladesh’s HIV/AIDS pr ogram Background CARE Bangladesh has been particularly active in embracing a rights-based approach to programming. In the process, leadership and staff realized that existing monitoring and evaluation (M&E) frameworks do not really allow for monitoring or evaluating qualitative changes related to empowerment. The Most Significant Change methodology was introduced to CARE Bangladesh’s HIV/AIDS program so staff members could try using this approach to capture qualitative changes and learn from the process of creating change. How was a more reflective approach developed? Volunteer outreach workers, CARE Bangladesh field-based staff, and staff from partner NGOs asked street- and brothel-based sex workers: “What is the most significant change that you have experienced during the past three months?” The question was open-ended and made no reference to the CARE Bangladesh project underway in the five areas. This was to ensure that the storytellers could decide for themselves what change was important. Any link to the project would have shown an assumption on CARE’s part that sex workers should consider CARE’s work important. The sex workers told their stories, which were written down by the outreach workers (who also were sex workers) and staff. Several Bangla dialects were spoken, so the story collectors took pains to ensure they used the right dialect with the women. A total of 36 stories were collected and taken to the CARE Bangladesh M&E team. Team members read each story and identified those they felt were most significant. Then the team returned to the story collectors and verified their understanding and analysis of the stories. The team felt that the most significant stories described how women reduced inequities in their personal and social relationships, including: • Being able to dress as they like, which included wearing footwear (previously prohibited) 145 Fletcher, Gillian, Magar, Veronica & Noij, Frank. Learning by Inquiry: Sexual & Reproductive Health. Field Experiences from CARE in Asia, 2005. CARE International. The following case study is directly taken from the ‘Learning by inquiry’ report, pages 16, 17 and 18. I wanted to keep the original text as it shows CARE’s own assessment of the MSCT and, what’s probably more important, the benefits (and challenges) of the method, according to the staff that implemented it. Page 98
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 • Winning access to education institutions for their children • Having the right to a burial place • Having greater ability to negotiate with clients on safer sex practices • Winning the right to open a bank account • Gaining freedom after years of bonded labor • Increasing involvement in self-help groups, rallies, etc. • Winning administrative and local support against a brothel eviction What were the benefits of working this way? A senior program manager said the process of collecting and analyzing the MSC stories gave staff insight into what the sex workers felt was important. He said stories about the women wearing shoes or securing education for their children, for example, had given staff valuable “food for thought” about the women’s perspectives and the everyday inequities they face. This was true of staff members who collected the stories, as well as those who analyzed them. In addition, the team reported that the volunteer outreach workers had increased their self-confidence and sense of engagement with CARE. They felt their skills in building relationships with sex workers were important to the entire MSC process. Outreach workers were also able to see that the experiences of disadvantaged and vulnerable women were important to CARE as an organization. Challenges Collecting the stories required skill in both Bangla and its dialects. And writing the stories down verbatim required a great deal of awareness and confidence. The story collectors had to put aside their own understanding of development and really listen to what the storytellers were saying or writing. It was critical that these narratives reflected the storytellers’ real voices; development- speak concepts such as such as empowerment or participation could be introduced during analysis, if appropriate. There was also a danger that collecting stories, then taking them elsewhere to be analyzed, could actually be disempowering for the women involved. CARE Bangladesh will assess this danger as it proceeds with the MSC pilot. Working qualitatively is also challenging for many staff members who expect that community research means collecting quantifiable data or basic demographics. For some staff members, there was a strong temptation to introduce existing ways of working into the MSC, which for them represented a completely new way of monitoring and evaluation. Page 99
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Because this exercise was a pilot, CARE Bangladesh was unsure how to share information from the stories within and across projects. This is being addressed as the pilot progresses. Similarly, the M&E team must figure out how to incorporate MSC into the M&E system. This will involve a reevaluation of reporting existing frameworks, which will take time and effort. In general, finding time and energy to think about adopting new ways of working, while still meeting old objectives and timelines, is a major challenge for all staff. Did anything changed? This work is still in its early stages. A second round of MSC story collection was underway as this report was being written. The M&E team had identified the need for technical assistance to provide support in the early stages of MSC capacity development within the country office. However, CARE Bangladesh is already considering ways to amend its reporting formats within its HIV program to build qualitative principles of MSC into everyday work. In addition the MSC process has enabled staff at different levels, including those in the field and on M&E teams, to really focus on what is happening in sex workers’ lives. One of the best results of the workshop was CARE Bangladesh’s willingness to use MSC throughout the country. The country office is drawing on national and international expertise to mainstream MSC methodologies for organizational change, learning and evaluation in a multi-year, countrywide women’s empowerment initiative. Common themes: Benefits • Usually, staff members appreciated opportunities to reflect on their work, question their own assumptions and develop a deeper empathy with communities. • Staff members realized they do not have to be experts, with an answer for every situation or inequity. • Reflective practice was key to increasing solidarity and empathy with the community - gradually moving away from “them and us” (i.e., project clients passively receiving services from project staff) to “we.” • Comparing MSC stories with key objectives helped identify project gaps and high light opportunities for deepening the quality of the work. • Donors were interested in qualitative changes in the field. Page 100
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 • Reflective practice enabled staff members to question the everyday realities of their work and to understand it better. It also helped them better understand their own position, particularly by thinking about their values and attitudes and how these may influence their work. Challenges • Country offices must avoid being “terrorized by logframe”; traditionally, logframe designs have focused on quantitative indicators and clearly defined services, which can make developing more reflective, participatory work a real challenge. • At both the country office and regional management unit level, CARE needs to step up advocacy efforts with donors to provide for more reflective, less formulaic ways of working. • A more qualitative, questioning approach should not mean prying into the lives of project participants. Staff members need to ask themselves: Are their questions vital to the project or are they just curious? It is easy to become a voyeur. • Learning by inquiry asks staff to head into largely uncharted territories by adopting a much more open approach to project design, implementation and M&E. Early on, it can be overwhelming. • For reflective learning to take place, it needs to be integrated into all of CARE’s day-to-day work. Reflective learning is a highly useful process to use throughout the project cycle, from initial design to final evaluation. Page 101
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Facilitation guide 146 The facilitator writes all the titles of the stories on the whiteboard, grouped by domain. They leave a space next to each story for comments e.g. Domain Title (examples) Comments 1 Develop analytical skills of the field staff Moving story, beginning middle and end. Attribution to project is questionable. Great story, not sure if it is about the project. 2 Increase participation of stakeholders in the monitoring process 3 Enhance mutual understanding among partners 4 Reafirm the values of the organization 1. The facilitator invites volunteers to read out all the stories belonging to the first domain of change. After each story ask: • What is this story really about? • What is your opinion of the story? 2. The facilitator writes any comments next to the title on the white board as above. 3. When all the stories have been read out for the first domain, ask people to vote for the story that they find most significant. Voting can be done by a show of hands. 4. When the votes have been cast, if there is a range of scores, encourage participants to discuss why they chose the story they chose. Ask questions such as: • Why did you choose this story above all other stories? • But some of you chose a different story – can you explain why you didn’t choose this story? • What do you think of the stories in general? 5. Next to each story makes notes of the reasons why they were and were not selected. 146 Page 104,Davies, Rick & Dart, Jessica. The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use. (2005). Page 102
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 6. Once everyone has heard why certain stories were voted for above others, the facilitator may call a second vote, this time there may be more consensus. If there is still no consensus about which story to choose, facilitate a discussion on the options with the group and come to an agreement, for example: • Choose two stories to reflect the range of views • Decide that none of the stories adequately represents what is valued • Choose one story but add a caveat explaining that not all people voted for this story because… 7. Move onto the next domain. Page 103
  • Stories that count. Narrative to monitor and learn. Guillermo Idáñez. December 2005 Stories that count. Narrative to monitor and learn. Guillermo Idáñez, MBA Graduate. HHL –Leipzig Graduate School of Management __ Guillermo.idanez@mba.hhl.de +49 (0) 176 211 76834 Page 104