GDNet Year 2 M&E Report 2013

4,455 views
4,360 views

Published on

This document provides the annual progress report (Year2 Jan-Dec 2012) and update to GDNet’s Baseline and M&E Framework. The M&E report for 2012 is structured according to the GDNet logframe – with separate chapters from the Purpose-level down through Outputs 1 to 4. From our experience, knowledge matters, partnership matters, and skills and capacity matters and our recognition of this has guided the strategic direction of GDNet throughout 2012 ad the development of a number of activities highlighted in this report.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
4,455
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

GDNet Year 2 M&E Report 2013

  1. 1. GDNet M&E Report 2013 – Year 2 Robbie Gregorowski and Jodie Dubber Revised August 2013
  2. 2. 1 Table of Contents Introduction ............................................................................................................................................................ 2 GDNet Year 2 M&E Summary ................................................................................................................................. 4 Outcome Level ........................................................................................................................................................ 8 Output 1 - Southern research better informed by current ideas and knowledge................................................ 13 Output 2 - Researchers better able to communicate their research to policy ..................................................... 22 Output 3 - Knowledge networking between researchers and with policy actors increased ................................ 32 Output 4 - Lessons about knowledge brokering best practice in the global south learnt and communicated .... 37 Annex 1: GDNet user base annual web survey questionnaire –Year 2................................................................. 44 Annex 2: GDNet user base annual web survey results –Year 2 ............................................................................ 45 Annex 3: Long list of cases .................................................................................................................................... 76 Annex 4a: Year 2 cases of knowledge into use in the policy process ................................................................... 79 Annex 4b: Update on Baseline and Year 1 cases – Cases of knowledge into use in the policy process ............... 89 Annex 5: Output 3 indicator 1 – GDNet ‘user base’ interaction – log................................................................... 99 Annex 6: GDNet Social Media Annual Stat Report for 2012 ............................................................................... 129 Annex 7: Output 3 indicator 2 – Researchers interactions with the policy domain – log................................... 133 Annex 8: Output 4 indicator 1 – Generation of best practice lessons – log........................................................ 135 Annex 9: Output 4 indicator 2 – Communication of lessons – log...................................................................... 146
  3. 3. 2 Introduction This document provides the annual progress report (Year2) and update to GDNet’s Baseline and M&E Framework. The document is structured according to the GDNet logframe – with separate chapters from the Purpose-level down through Outputs 1 to 4. Each Chapter is structured as follows:  Year 2 summary – A clear summary statement of progress for each output indicator for comparison against the baseline and Year 1, and the relevant milestone. The statement is followed by a more detailed elaboration of the Year 2 M&E data generated and an analysis of its implications.  M&E approach summary – A very brief explanation of the approach and method adopted to generate the data for each output indicator. Readers should refer to the 2011 GDNet Baseline and M&E Framework for a more detailed account of how the M&E framework was designed and the methods adopted.  Data management plan – Setting out the on-going M&E roles and responsibilities within the GDNet team.  Evidence base – Providing detailed summaries of the relevant data used to support each output indicator – typically web statistics, web users survey, log templates, and interviews. Unless otherwise stated, Year 2 refers to the period January to December 2012. The GDNet M&E baseline was established in December 2010. GDNet’s M&E is reviewed and reported on an annual reporting cycle according to the calendar year January to December as follows: Logframe M&E Framework Baseline Baseline – est. December 2010 Milestone 1 (2011) Year 1 – January to December 2011 Milestone 2 (2012) Year 2 – January to December 2012 Target (2014) Year 3 – January to December 2013 with the potential to extend to July 2014 to cover GDNet Programme completion The essence of the Year 1 M&E report was to highlight the progress the GDNet team had made in generating a robust knowledge and evidence base relating to the provision of knowledge services and capacity support to Southern researchers. Following on from this, the essence of the Year 2 M&E report has been to investigate GDNet’s progress in the ‘ownership’ of evidence base - synthesising the knowledge they hold and developing best practice lessons. Overall it is the belief of the M&E expert that GDNet is demonstrating a maturing institutional capacity as a leading knowledge brokering institution for Southern research. Note from GDNet on their Year 2 strategy and their approach to monitoring, evaluation and learning From our experience, knowledge matters, partnership matters, and skills and capacity matters and our recognition of this has guided the development of a number of activities in year two: the Connect South campaign, the new ThinkNet platform, and consolidation of our team's skills and composition. Based on our reflections and learning about the landscape, we see that we still occupy an important niche and meet an important need: engaging with the South and supporting southern research uptake. GDNet has been maturing as an organisation that offers knowledge services and we are investing in open data plans that will be available towards the end of 2013 and will enable us to offer related products by the end of next year that incorporate knowledge aggregation, and expanded
  4. 4. 3 regional knowledge portals. 2012 saw a focused approach to encourage stakeholders to engage with services provided by GDNet be them knowledge services or the research communications capacity building program. During the course of the year the GDNet team developed a strategy to use digital tools to increase the outreach and uptake of southern research systematically throughout the four output areas as indicated by the logframe. After the launch of 23 thematic portals together with 7 regional portals, GDNet's Knowledge Services continue to become an advocate for southern researchers and seek to understand more about their needs and the barriers to access knowledge and the uptake of southern research. Two desk reviews were commissioned to look closer at those areas; ‘Implementing a gender audit of an online knowledge service: The experience of GDNet’ and ‘Southern Research uptake’. Under GDN’s new management, both parties are progressing integrated interaction - bringing both websites under one online platform and building a combined approach to Open Data by early 2014 as key milestones. Interim measures to streamline branding and enhance content exchange and outreach have been rolled out. Finally, as an example of GDNet’s maturing institutional capacity and self-belief as a knowledge broker for Southern researchers, 2012 saw the launch of the Connect South campaign (www.ConnectSouth.org); an initiative to encourage development research stakeholders to create a more enabling environment for southern researchers. The aims of this campaign are reflected in the GDNet’s four key areas of activity.
  5. 5. 4 GDNet Year 2 M&E Summary Outcome Diverse research and policy audiences make better use of development research from the global south Indicator 1 Southern researchers use of other southern research in own research Baseline Significant use of Southern research in Southern researchers’ own research - 64% of GDNet researchers use Southern research to a great or moderate extent and GDNet’s most popular KB publications on average draw on research which is 40% from the global south and 60% from elsewhere Year 2 Significant use of Southern research in Southern researchers’ own research – 64% of respondents use Southern research to a great or moderate extent. No apparent shift in use over time. Appropriate milestone Increase to be defined once baseline set, including incidence of researchers citing GDNet as a source of information. (2012) GDNet Ownership  Sherine Ghoneim Tools & Frequency  GDNet users web survey - annual  Bibliometric sampling exercise –citation analysis - annual Indicator 2 Cases of knowledge into use in policy processes Baseline Eight cases of knowledge into use in the policy process selected, developed and validated by GDNet registered researchers Year 2 Five new cases of knowledge into use in the policy process selected, developed and validated by GDNet registered researchers. Appropriate milestone At least five more examples of knowledge into use in policy processes plus annual updates on existing cases to provide an annually expanding and evolving portfolio of cases. (2012) GDNet Ownership  Sherine Ghoneim Tools & Frequency  GDNet users annual web survey – case identification - annual  Awards & Medals Finalists Most Significant Change Technique – case selection  Informal case study telephone interview – case development and validation Output 1 Southern research better informed by current ideas and knowledge Indicator 1 Level of use of, and satisfaction with GDNet research-orientated online services Baseline High general use – an average of 23,617 visitors (33% from Global South) resulting in 8,359 recipients of Funding Opportunities newsletter, and 1144 JSTOR sessions to access online journals. Moderate satisfaction– Knowledgebase online papers rated extremely useful by 38% and moderately useful by a further 32% of respondents to the GDNet users’ web survey. Access to online journals rated extremely useful by 41% and moderately useful by a further 29% of respondents Year 2 Significant continued increase in headline level of use on baseline and Year 1, with the GDNet website receiving an average of 34,709 visitors per month in Year 2 with 32% coming from the Global South. Level of satisfaction is broadly maintained with users expressing that it is important that GDNet continues to provide its core range of services – newsletters, access to online journals, knowledge base journal
  6. 6. 5 abstracts, and researcher profiles. Appropriate milestone 10% year-on-year increase in use 5% year on year increase in satisfaction (2012) GDNet Ownership  Shahira Emara Tools & Frequency  GDNet web statistics – collected quarterly & analysed annually  GDNet users web survey - annual Indicator 2 Level of use of, and satisfaction with themed services Baseline Themed services not yet established Year 2 Relatively low initial use of themed services combined with moderate satisfaction expected to improve in a group of engaged users as themed services become established and further rationalized. Appropriate milestone Establish baseline use of themed services at end of first year. Establish baseline satisfaction of themed services at end of first year. (2012) GDNet Ownership  Shahira Emara Tools & Frequency  GDNet web statistics – collected quarterly & analysed annually  GDNet users web survey - annual Output 2 Researchers better able to communicate their research to policy Indicator 1 Researchers’ confidence and ability to communicate their research – immediately following capacity building effort Baseline On a self-assessment scale where 0 = not at all confident and 5 = very confident, the average GDNet researcher is moderately confident (2.8 out of 5) to communicate their research to policy before any training/capacity building has been provided by GDNet. Year 2 On a ‘before and after’ self-assessment scale (where 0 = not at all confident and 5 = very confident to communicate their research to policy), the average capacity building participant increases from 2.0 before to 3.0 (a 50% increase) after a GDNet capacity building effort. Similarly in terms of ability, the average participant increases in ability from 1.8 before to 2.7 (a 50% increase) afterwards. Appropriate milestone Consistent 30-40 % increase in confidence at the end of each workshop regardless of starting point. Expect to see year-on-year improvement on value-added in workshops. Consistent 50–60 % increase in ability at the end of each workshop regardless of starting point. Expect to see year-on-year improvement on value-added in workshops (2014) GDNet Ownership  Zeinab Sabet Tools & Frequency  Research communications capacity building questionnaire – per event Indicator 2 Researchers’ confidence and ability to communicate their research – sustainability of capacity building effort Baseline First set of cases of researchers’ ability to communicate their research to policy being developed. Year 2 Second set of six ‘pledge’ cases developed from GDNet workshops 7-9 held during Year 2 reflecting the sustainability and application of GDNet’s capacity building effort. Four of six Year 1 cases developed with 3 and 12-month follow up. Appropriate milestone Rich portfolio of examples of researchers’ communications confidence and ability across a range of sectors and regions. (2014)
  7. 7. 6 GDNet Ownership  Zeinab Sabet Tools & Frequency  Research communications capacity building questionnaire – cases developed from 3-month pledge follow up  12-month pledge follow up (introduced in Year 1) in process Output 3 Knowledge networking between researchers and with policy actors increased Indicator 1 GDNet ‘user base’ interaction Baseline Very limited ‘user base’ interaction – online collaborative workspace piloted Year 2 GDNet have continued to increase user base interaction in Year 2, demonstrating a maturing ability to deploy a range of tools and platforms (particularly social media) to engage a broad spectrum of GDNet users. Appropriate milestone 10 % year on year increase in use of electronic user base network. Good face-to-face user base interaction at conference and at 2 RNPs/Regions. (2013) GDNet Ownership  Zeinab Sabet / Sherine Ghoneim Tools & Frequency  User base interaction log – on-going / following each round of interaction Indicator 2 Researchers interactions with the policy domain Baseline Very limited interaction – GDNet facilitation of researchers interactions with the policy domain not yet established Year 2 GDNet have logged five distinct interactions between Southern researchers and the policy domain as well as catalysing several more through their research communications capacity building support. GDNet can also demonstrate some significant new knowledge about how best to facilitate researcher – policy domain interaction. Appropriate milestone At least one research-policy interface per year in one region plus one online space. (2013) GDNet Ownership  Zeinab Sabet / Sherine Ghoneim Tools & Frequency  Interaction log - on-going / following each round of interaction Output 4 Lessons about knowledge brokering best practice in the global south learnt and communicated Indicator 1 Generation of best practice lessons Baseline Generation of best practice lessons not yet established Year 2 GDNet routinely generates lessons about knowledge brokering best practice in the Global South. As well as synthesising lessons across the programme, GDNet has also initiated a small but significant and expanding stream of work documenting the needs and demands of Southern researchers as well as potential best practice in meeting these needs and demands, based on their own experience and expertise. Appropriate milestone Two GDNet best practice products. (2012) GDNet Ownership  Sherine Ghoneim / Shahira Emara Tools & Frequency  Capacity building event reflection – on-going / following each knowledge brokering expert involvement
  8. 8. 7  Synthesis of event reflection best practice - annual Indicator 2 Communication of lessons Baseline Communication of best practice lessons not yet established Year 2 In Year 2 GDNet has stepped up both its communication of knowledge brokering lessons (with seven discrete examples reflected in the communications of lessons log) and its capacity to be reflective in terms of the team enhancing their ability to communicate these lessons (as demonstrated by the reflective learning lessons) Appropriate milestone One conference paper or one formal published paper. (2012) GDNet Ownership  Sherine Ghoneim / Shahira Emara Tools & Frequency  Inventory log of communications activities - on-going / following each round of communication Indicator 3 1 Instances of GDNet incorporating new thinking or innovation into its practices as a result of participation in knowledge brokering fora Baseline No instances of GDNet incorporating new thinking or innovation into its practices as a result of participation in knowledge brokering fora Year 2 Despite it being too early for the team to conduct a meaningful reflection of the output 4 indicator 3 log, the log itself contains a small number of illustrative examples of GDNet behaving as a learning programme by incorporating new thinking or innovation into its best practices. Appropriate milestone GDNet Ownership  Sherine Ghoneim / Shahira Emara Tools & Frequency  Inventory / log of incorporating new thinking or innovation into GDNet practices 1 Following the GDNet DFID Annual Review held at the end of April 2013 and the subsequent discussion of the logframe it was agreed to drop Output 4 indicator 3.
  9. 9. 8 Outcome Level Indicator 1 - Southern researchers’ use of other southern research in own research Year 2 summary – Significant use of Southern research in Southern researchers’ own research – 64% of respondents use Southern research to a great or moderate extent. No apparent shift in use over time. GDNet user base web survey results – Surveyed using the same format as the baseline and Year 1, a number of questions in the web survey provide an indication of the level of use of Southern research. Further details and analysis of the Year 2 web survey are provided in Annex 2. Asked to what extent Southern researchers use Southern research in their own work, 64% of respondents claimed that Southern research was used to a great or moderate extent (See Annex 2 question 27). This is precisely the same percentage figure as the baseline and Year 1. When asked to describe the type of research that they read, the most common response researchers gave is that they do not distinguish between Northern and Southern research (34%) (See Annex 2 question 28). However, the next biggest group (26%) believe they read more Northern than Southern research, followed by 25% who believe they read the same amount of Southern and Northern research. These results are very similar to the baseline and Year 1 results. As with the baseline, what emerges from the results of the web survey is a nuanced picture of use – significant use of Southern research by Southern researchers but perhaps no more significant than their use of Northern research. There appears to be very little apparent change in this trend over time – between the baseline and Year 2. The Year 2 web survey provided a number of interesting insights into the nature and perceptions of Southern research by Southern users and how they tend to combine use of Northern and Southern research in a complementary manner (See Annex 2 question 28). The following responses illustrate this: “Development of my own discipline takes place primarily in the North. I read books and articles written in the North to keep to date on these theoretical developments. I also read Southern research to understand how the theory is being applied in the region. As a matter of fact, I do a lot of South-North/North South research.” “I believe one type of research feed the other, and vice versa. While Northern research is strongly grounded on theories and a quite rectangular approach of thinking, Southern research addresses the complexity of the developing world, with apparently backward behaviours rooted in a mosaic of cultures that bring forth the impetus to challenge mainstream reflection paths and theories. The difficulties encountered in trying to apply such theories in Africa e.g., due to the lack of appropriate data or research context, give scientists in the South the opportunity to contribute to developing new theories or to provide insights to modifying existing ones.” “Northern research arenas have demonstrably failed to impact measurably on issues like poverty and inequality, much less their source cause, corruption. Southern research offers more outside-the-box thinking, often due to necessity mothering invention.” “Southern research quality is picking up, albeit slowly. The northern research in general is more structured and is supported by generous funding. Also, the clustering of researchers also benefits the northern research much more than the southern ones. But I would reiterate, a proper choice of southern researchers, where matching of quality, capacity and the research topic is supreme, could produce excellent result provided generous funding support comes.” As highlighted in the Year 1 report, these responses hint at a potentially important niche for GDNet to more proactively fill – raising the profile of the best Southern research so that it is perceived as on a par with Northern research in terms of quality but also highlighting the feature of Southern research that defines it from more traditional Northern or Western research – it’s applied and practical nature, grounded in local contexts and addressing issues where there is strong demand or a clear evidence gap. To a certain extent GDNet already
  10. 10. 9 does this through its tagline to ‘showcase’ Southern research but it is not clear that GDNet’s remit covers an explicit focus on enhancing the perception of Southern research based on its unique features. Note on the GDNet web survey in response to comments made in the GDNet DFID Annual Review – see Annex 2 for a more detailed introduction to the web survey: The GDNet web survey has been conducted annually since the baseline and provides a range of data to support outcome, output 1 and output 2 indicators. Whilst the data is statistically robust in terms of absolute numbers of respondents (721 in Year 2), the response rate relative to total GDNet members is relatively small (5.4%). This is not an unusually low response rate for a survey of this nature to what is a diffuse and generally passive user group. However 5.4% response rate in Year 2 is down from 8.2% in Year 1 which suggests GDNet’s members may be suffering from ‘survey fatigue’. In addition and as explicitly set out in the baseline M&E report, all web surveys of this nature suffer from response bias whereby those motivated to respond tend to be those with particularly strong feelings (positive and negative) towards the services offered. The DFID Annual Review team also mentioned in their feedback on the Year 2 M&E report that the survey methodology should be improved in order to generate responses from a more representative sample of GDNet members. Responding to this feedback, the M&E team will investigate ways to improve the survey in terms engaging a more representative sample in Year 3. This of course depends on how to accurately define a ‘representative’ GDNet user, given that the vast majority of GDNet’s users are likely to be ‘passive’ – using GDNet as one of a selection of research accessing and knowledge sharing tools periodically and ad hoc – and therefore accessing a valid sample of ‘representative’ users who by their nature tend not to respond to web surveys could be a challenge.. However, options to improve the web survey may involve sending survey to randomly selected user profiles or emailing every Xth member to ask them to participate in a short telephone interview which would complement the findings of the broader web survey. M&E approach summary Purpose level indicator 1 draws on perceptions of use of Southern research gathered from the GDNet user base web survey results conducted annually. Data management plan Robbie Gregorowski / ITAD  On an annual basis – Repeat analysis of the annual GDNet user base web survey. Sherine Ghoneim / GDNet  On-going – Interpretation of the findings of the annual GDNet user base web survey and application to better understand and improve the services GDNet offers Evidence base See Annex 1 for the GDNet user base web survey questionnaire including the new questions added in Year 2. See Annex 2 for a summary of the results of the Year 2 GDNet user base web survey.
  11. 11. 10 Indicator 2 - Cases of knowledge into use in policy processes Year 2 summary –Five cases developed through the Year 2 case selection process. GDNet Researchers- Cases of Knowledge into Use in Policy Processes – Year 2 CASE 3 - Yugraj Singh Yadava, Bangladesh, Small-scale fishermen Subject: Low-cost group insurance for fishermen in Bangladesh Context: To better understand how to introduce safety nets to mitigate the risks faced by small-scale sea fishermen and their families. Impact: Group insurance scheme successfully piloted and continues to expand to cover 2.5-3% of Bangladeshi coastal fishermen. Policy influencing factor: BOBP-IGO took on the role of an ‘honest broker’ or impartial agency, connecting the fishermen’s associations to the insurance provider. CASE 2 - Dominique Babini, Argentina, Open access to scholarly research Subject: Dominique Babini is the Open Access Advocacy Program leader at CLACSO. Context: how to better understand and provide open access to research outputs as a global public good. Impact: The program has advocated that APCs are unreasonable, unnecessary and unsustainable / unaffordable for Southern research producers and demonstrated the potential of managing OA scholarly communication globally as a commons .Policy influencing factor: Replication and further influence over the OA agenda relates to creating and mobilising networks of like-minded people across the research and science policy sector in the global south. CASE 1 - Harilal Madhavkan, India, Ayurvedic medicine Subject: Understanding local health system and the protection of indigenous knowledge in medicine in Kerala. Context: To better understand how the state can support converting indigenous knowledge into sustainable livelihoods strategies. Impact: The action-research led to establishment of Confederation of Ayurvedic Renaissance in Kerala (CARe Keralam) consortium which helps to develop a medicinal plant linkage with community cooperatives and potential tribal medicinal plant collectors. Policy influencing factor: Related to the point above, action-research intended to provide solutions to practical problems should look for solutions that already exist in the community / problem-area itself. Solutions don’t necessarily need to be imported. CASE 4 – Glady Kalema-Zikusoka, Uganda, Community health worker family planning Subject: Assessing the feasibility of community health workers giving three monthly contraceptive injections. Context: Many rural communities live far away from health centres making modern family planning beyond the reach of these communities. The research aimed to demonstrate that community health workers could be sufficiently well trained to safely provide Depo-Provera contraception injections. Impact: .The training led to a doubling of the rate of uptake of contraceptive injections in Bwindi communities and ultimately provided the evidence base for a policy change allowing trained community health works to provide contraceptive injections. Policy influencing factor: Engaging key policy makers in the Ministry of Health from the outset of the program was critical in delivering a policy change. CASE 5 – Nikica Mojsoska-Blazevski, Macedonia, Research on ways to enhance social safety nets Subject: A detailed analysis of the social assistance and benefit system in Macedonia with a focus on addressing disincentives to work. Context: Macedonia requires measures to decrease the dependency on welfare of those who are able to work. The research identified a number of areas where legislation could be altered to improve incentives to work. Impact: As a result of the research labour legislation was amended to distinguish between active unemployed persons and those not actively searching for a job. This resulted in halving the number of registered unemployed in Macedonia. Policy influencing factor: Research has to be at least in part demand-driven by policy makers and involve a field- based component to assess the practical side of how a policy, law or measure will be successfully implemented.
  12. 12. 11 The Year 2 case selection process produced five cases following the case selection process held with the GDN Awards and Medals Finalists at the GDN Annual Conference in Manila in June 2013. The latest round of cases adds to the eight developed in each of the previous two years to provide GDNet with a deeper understanding of the nature of Southern research and some very informative illustrations of how Southern research can inform policy and practice. Drawing on the Year 2 cases as well as the understanding GDNet has gained across all three rounds uncovers some interesting themes and initial conclusions about the nature of Southern research: Policy influencing factors  An emerging theme apparent in many of the cases across all three rounds is the extent to which Southern researchers set out to use research to solve distinct development problems in a practical and pragmatic manner. Several of the themes which became the subject of the research in cases identified had very little in the way of a prior robust, empirical research or evidence-base. Both the Uganda community health workers case where people lack basic access to effective contraception and the Bangladesh small-scale fisheries case where fishermen and the families lack affordable insurance illustrate how the application of action-research has provided workable solutions to very ‘tangible’ problems.  Drawing on Southern research addressing pressing developmental problems, it is apparent that nearly all cases are clearly ‘demand-led’. That is they all respond to the direct demand of the primary stakeholders for research to address a problem or constraint they face. These primary stakeholders, rather than simply being the subjects of the research, are engaged in a very participatory manner as stakeholder partners in the research process itself. Many of the cases illustrate the researchers go one step further and from the outset engage policy makers as well as the primary stakeholders from the outset. In this way policy makers are drawn into the research process as it develops.  Involving decision makers in from the outset is just one way in which Southern research tends to take a more innovative, informal and opportunistic approach to research dissemination. Southern researchers conducting ‘action research’ seem at ease with engage a wider range of stakeholders – local communities, politicians, civil servants, and the media (amongst others) throughout the research process. This is in direct contrast to Western research which tends to engage decision-makers at the end (if at all), disseminating research findings often through a relatively ‘formal’ and established dissemination and communication processes – presenting at conferences and disseminating research findings through formal journal peer-review processes.  Similarly the cases continue to highlight that Southern researchers use a wider and more innovative variety of tools to generate ‘evidence’ to support their research. Several cases highlight the use of documentary evidence (photos and video footage) combined with more traditional research methods such as key informant interviews to communicate the research to a wider audience of stakeholders. In this way Southern researchers explicitly draw in the media, civil society organisations, NGOs, and the private sector to put ‘pressure’ on decision-makers to legislate for change. Put simply, the cases highlight that some Southern researchers are particularly adept at translating their research findings into formats appropriate the meeting the needs of multiple stakeholder and audience groups and are adept at employing a wide range of formats, platforms and channels to broad sets of stakeholders. GDNet’s role and contribution  GDNet can play a simple but critical role in sharing innovative research, connecting researchers in one region or country with other researchers so that knowledge and learning in one context can effectively be transferred and replicated in similar contexts elsewhere.  The use of evidence in the most appropriate format – using photos and videos combined with more formal research techniques such as surveys and interviews - is one of the areas where Southern research can be considered more effective and advanced than more traditional Western research – Southern research appears better at bringing in innovative technology such as the use of visual and social media to generate more substantial impact. There is a potential role for GDNet in sharing the lessons and experience of how best to combine the two forms of research as well as potentially providing training in the use of more innovative research and documentation techniques for Southern
  13. 13. 12 researchers – building the capacity of Southern researchers to present their research in the most appropriate format for a particular stakeholder audience.  Overall, the case selection process has identified that GDNet engages a wealth of innovative, informed and highly motivated researchers. The simple process of producing the cases has provided a showcase for a number of these researchers. GDNet could use the on-going case selection process to provide a platform expressing GDNet’s lessons and learning on the key success factors in producing effective, policy-influencing work – this would both help raise the profile of innovativeness of Southern research (something that more traditional Western research may learn from) and help bring the lessons and success stories to a wider audience. M&E approach summary The aim for this indicator is to develop a robust and credible portfolio of cases of knowledge into use on a year on year basis – updating progress with existing cases and developing new ones. The process for developing new cases involves 3 stages which are repeated annually:  Stage 1 – Case identification from GDNet Registered Researchers - A broad number of ‘cases’ (approx. 35-50) are identified from responses to the annual GDNet user base web survey.  Stage 2 – Most significant selection and validation panels – Engaging a group of GDN A&M Finalists at successive GDN Annual Conferences, the next of which is being held in Manila in June 2013 to review and select the ‘most significant’ cases. The A&M Finalists panel is followed by a second panel of GDNet key stakeholders including GDNet staff and independent research communications experts to further review and select the most significant cases down to a shortlist of 8-10 cases.  Stage 3 – Development and Validation of Most Significant Cases - The authors of the 8-10 selected cases are contacted by the ITAD consultant and each invited to an informal telephone interview to discuss and develop their case in more detail. Data management plan Robbie Gregorowski / ITAD  On an annual basis – Facilitation of the case selection process and development of new cases. Zeinab Sabet / GDNet  On-going – more detailed case follow-up and lesson learning if required. Sherine Ghoneim / GDNet  On-going – Extraction and synthesis of lessons to enhance GDNet’s role and contribution. Evidence base A detailed explanation of the process designed to identify the cases can be found in the Baseline and M&E Framework report.  Annex 3 provides the long-list of cases gained from the web survey.  Annex 4a provides the full write ups of the five new cases developed in Year 2.  Annex 4b provides the updates and revalidated write ups of the cases developed at the baseline and Year 1. Note – engaging researchers to update their cases perhaps unsurprisingly has proved to be a challenge with the majority of researchers failing to engage in the follow up process in the years subsequent to them developing the case. A summary of the limited responses (despite chasing) is provided in the table below.
  14. 14. 13 Cases of knowledge into use in the policy process – follow up status Name Case Country Baseline 2012 2013 2014 Gohar Jerbashian Prevention of maternal and neo-natal mortality Armenia Y Y CHASED Wassam Mina Investment in Gulf Cooperation Countries UEA Y Y Y Marcio da Costa Unequal educational opportunities Brazil Y Y Y Rajarshi Majumder Obstacles for Out of School Children India Y SENT SENT Pamela Thomas Decline of immunisation and maternal child health care service deliveryVanuatu Y SENT SENT David Rojas Elbirt Provision of 'Watsan'products Bolivia Y Y SENT Tohnain Nobert Lengha Obstacles to Diasbled Children in Education Cameroon Y SENT CHASED Sarah Ayeri Ogalleh Community tree planting Kenya Y SENT SENT Year 1 Cecil Agutu Laws and policy in sugar sub-sector Kenya Y CHASED Davidson Omole Nigerian Stock Exchange Nigeria Y CHASED Martin Oteng-Ababio Digital Waste Management Ghana Y CHASED Constancio Nguja Civil Society Advocacy Mozambique Y Y Brigitte Nyambo Integrated Pest Management Technology Ethiopia Y CHASED Waweru Mwangi Card-less ATMsystem Kenya Y CHASED Francesco Pastore Mongolian Youth education and employment Mongolia Y CHASED Hasina Kharbhih Child Labour Rights India Y Y Year 2 Augustin Fallas Santana Outsourcing clinical trials by pharmaceutical industry Costa Rica N Dr Palitha Ekanayake Classification of rural roads Sri Lanka N Harilal Madhavkan Traditional medicine industry India Y Llam Dorji Youth policies Bhutan N Dominique Babini Digital Open Access Argentina Y Dr Damilola Olajide Regulation in the payment card industry Nigeria N Tran Tuan Mental Health Care in Vietnam Viet Nam N Rajdeep Mukherjee Low cost insurance for fishermen in Bangladesh India Y Nadia Afrin Shams Builidng Inclusive Information Knowledge System Bangladesh N Gladys Kalema-Zikusoka Integrated biodiversity, health and community development Uganda Y Nikica Mojsoska Blazevski Gender Wage Equality in Macedonia Macedonia Y First phone interview Followed up in?
  15. 15. 14 Output 1 - Southern research better informed by current ideas and knowledge Indicator 1 - Level of use of and satisfaction with GDNet research-orientated online services Year 2 summary – Significant continued increase in headline level of use on baseline and Year 1, with the GDNet website receiving an average of 34,709 visitors per month in Year 2 with 32% coming from the Global South. Level of satisfaction is broadly maintained with users expressing that it is important that GDNet continues to provide its core range of services – newsletters, access to online journals, knowledge base journal abstracts, and researcher profiles. Summarised below are the key web statistics currently generated on the use of GDNet online services, averaged for Year 2 – January to December 2012. GDNet Stats Report Sheet 2012 Month Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12 Totals mth Avg lst yr Av % Inc Traffic Total no. of Hits 123,923 147,090 130,585 105,353 174,091 130,230 182,277 143,612 129,379 159,097 137,889 187,988 1,751,514 145,960 130,984 11.43% Total no. of Visits 42,333 41,275 41,715 36,309 45,491 31,560 44,573 38,437 41,810 51,511 53,203 50,621 518,838 43,237 41,324 4.63% Total no. of Visitors 38,641 35,984 34,214 30,586 32,949 21,409 34,279 30,301 32,920 39,323 43,359 42,548 416,513 34,709 29,416 18.00% Total no. of Visitors from the South 12,798 11,965 11,594 10,216 10,999 6,927 11,744 10,650 10,635 11,599 12,319 12,585 134,031 32% 39% -17.49% Total document downloads from KB 12,924 8,235 18,351 11,643 8,761 9,164 17,907 10,864 10,495 15,712 10,674 12,572 147,302 12,275 11,972 2.53% Research Paper abstract views 29,379 20,159 32,022 23,787 32,255 22,048 29,205 22,851 22,144 31,721 38,667 28,924 333,162 27,764 KB pages stats(res+doc+org abstracts) 69,754 50,315 70,176 58,041 62,936 46,946 62,266 55,845 57,047 73,493 98,234 76,262 781,315 65,110 % of KB stats to overall website stats 56% 34% 54% 55% 36% 36% 34% 39% 44% 46% 71% 41% 45% Newsletters New recipients - Funding Opps 34 16 31 28 16 5 22 8 10 7 12 18 207 51 Total recipients - Funding Opps 8,068 8,102 8,118 8,149 8,177 8,193 8,198 8,220 8,228 8,238 8,245 8,257 8,257 New recipients - Research in Focus 42 21 36 32 18 3 17 2 0 1 2 1 175 15 47 Total recipients - Research in Focus 36184 36226 36247 36283 36315 36333 36336 36353 36355 36355 36356 36358 36358 Reseachers No. new researchers' registrations 236 249 219 211 195 126 133 135 84 118 108 111 1925 160 53 IMP ** No. of accepted res profiles/month 42 21 37 33 18 8 23 10 10 10 14 21 248 21 53 -60.96% No. of researchers update their profiles 120 88 93 84 80 51 77 77 61 65 77 73 946 79 113 Total no. of researcher profiles 12,321 12,359 12,378 12,410 12,427 12,435 12,457 12,466 12,476 12,486 12,500 12,521 No. of researchers with research papers 5/ 1745 0/ 1745 1/ 1746 2/ 1748 0/ 1748 0/ 1748 1/ 1749 1/1750 1/ 1751 0/ 1751 0/ 1751 1/1752 No. of researchers accessing online jrnl 85 88 96 98 90 80 79 81 71 90 71 83 1,012 84 105 -19.30% Organizations No of applications to register as an org 0/ 524 0/ 524 1/ 525 2/ 527 0/ 527 0/ 527 0/530 0/530 1/ 531 1/ 532 1/ 533 1/534 67 No. of new organisation profiles/month 19 23 24 52 42 20 87 36 18 27 24 25 396 33 15 122.52% Total no. of organization abstracts 4,791 4,814 4,838 4,890 4,932 4,952 5,039 5,075 5,093 5,120 5,144 5,169 No. of new organisation with documents 18/ 1893 19/ 1912 22/ 1934 51/ 1985 39/ 2024 20/ 2044 32/ 2076 22/2098 17/ 2115 27/ 2142 22/ 2164 25/2189 306 NA Documents No. of new online research papers/month 114 174 121 234 200 143 218 158 158 152 196 158 2015 168 83 102.72% Total no. of document abstracts 17,843 18,017 18,138 18,372 18,572 18,715 18,933 19,091 19,249 19,401 19,597 19,755 Total New content generated 175 218 182 319 260 171 328 204 186 189 234 204 2,670 223 150 48.33% 224 *MS: 150/ month *MS: 175/month **only 14% of registrations are accepted. We receive 166application; accept only 24/month Level of use – The level of use of GDNet’s research-orientated online services continues to rise. At the headline level, GDNet received an average of 34,709 visitors per month in Year 2. This represents an 18% increase in the number of visitors over Year 1. This exceeds the logframe-defined milestone of 10% increase in year on year use. However, the percentage of visitors coming from the Global South has fallen from 39% in Year 1 to 32% in Year 2 2 . It is not entirely clear why there has been a fall in the percentage users coming from the Global South between Year 1 and Year 2. One potential explanation may relate to GDNet’s proactive social media efforts 2 Established from users’ IP addresses.
  16. 16. 15 throughout 2012 which may have engaged a higher proportion is Northern / Western users who are more familiar with and have better access to social media channels. Similarly positive progress has taken place in terms of the average monthly document downloads from the GDNet KnowledgeBase (KB). Average monthly document downloads were approximately 4,000 at the baseline, 11,900 during Year 1, and rose further to 12,275 during Year 2. This is a pleasing statistic as it represents an increase in the ‘quality of use’ of GDNet’s online services. As referenced in the Year 1 report, quality of use (developing a core of ‘involved’ users and focussing on their uptake of knowledge) has been a focus of the GDNet team throughout Year 2. With this in mind, the GDNet team have endeavoured to develop a small set of ‘quality of use’ indicators which will be followed up on in Year 3. These include:  The total research paper abstract views which in Year 2 totalled 333,162 and averaged 27,764 per month; and,  The % of Knowledge Base hits as a proportion of overall GDNet site hits which averaged 45% in Year 2. Taken at face value both of these results are pleasing as they indicate that there is high absolute usage of the GDNet research paper abstracts (a key GDNet value-added service) and that almost half of all GDNet visitors use the Knowledge Base – a quality of use indicator. In Year 3 GDNet will hope that both overall usage of the KB and its usage as a proportion of overall visits increases as GDNet users continue to appreciate the value- added services provided. The number of recipients of GDN Newsletters continues to rise although at a significantly lower rate than in Year 1: an average of 15 new recipients per month receive the Research into Focus newsletter (down from an average of 51 in Year 1) and an average of 17 new recipients receive the Funding Opportunities newsletter (down from an average of 42 in Year 1). Although this represents a significant fall in the rate of new recipients, it is not deemed too significant as GDNet’s strategy focussed on quality involved usage is not based on newsletter recipients who, to a certain extent, represent a slightly out of date and less involved means of interaction with users. The GDNet Team have been quick to embrace technological progress in user engagement in Year 2. This is illustrated by the fact that GDNet now maintains several complementary platforms alongside the website – a blog, Twitter feed, YouTube channel and LinkedIn page 3 – to support interactive user engagement through cross-posting. The implications of maintaining multiple platforms are discussed in the next section drawing on the web survey results. Level of satisfaction – Satisfaction with GDNet’s research orientated online services is assessed based on the web survey findings, in particular question 14a which asks GDNet users to rate GDNet services according to their usefulness. A summary of the Year 2 results with the Year 1 results in brackets is provided below. Answer Options Extremely Useful Moderately Useful Somewhat Useful Not at all Useful Lack Access to Service Not aware of service Research in Focus newsletter 28.6% (26.3%) 38.2% (28.2%) 21.0% (22.9%) 2.8% (5.8%) 1.0% 8.5% (16.8%) Funding Opportunities newsletter 36.9% (37.0%) 30.2% (29.2%) 20.7% (20.3%) 2.9% (3.8%) 1.5% 7.8% (9.7%) Monthly GDN newsletter 27.8% (32.5%) 40.4% (33.2%) 21.8% (21.0%) 2.8% (4.7%) 1.1% 6.1% (8.6%) GDNet Knowledgebase - Online papers 27.8% (36.2%) 33.6% (28.2%) 22.9% (20.4%) 3.3% (4.7%) 2.7% 9.7% (10.5%) GDNet Knowledgebase - Researchers' profiles 15.1% (20.5%) 29.2% (29.4%) 34.3% (28.9%) 6.7% (8.3%) 3.4% 11.3% (12.9%) GDNet Knowledgebase - Organisations' profiles 13.3% (15.7%) 27.6% (32.0%) 36.5% (29.8%) 7.2% (7.7%) 2.5% 12.9% (14.9%) Online journals 35.4% (42.0%) 27.6% (23.2%) 16.9% (17.9%) 4.6% (4.3%) 3.8% 11.7% (12.6%) Regional window portals 15.1% (17.6%) 30.2% (25.8%) 28.0% (21.9%) 6.1% (9.5%) 3.1% 17.5% (25.2%) GDN announcements (competitions, conferences, scholarships, jobs, etc.) 38.0% (43.3%) 29.8% (26.9%) 19.5% (20.4%) 2.7% (2.7%) 1.2% 8.8% (6.7%) Thematic Windows 14.9% (15.8%) 27.8% (25.1%) 26.3% (22.4%) 7.8% (9.7%) 3.0% 20.2% (26.9%) GDNet Feeds (RSS or email) 10.5% (13.1%) 22.2% (24.6%) 28.5% (25.6%) 11.4% (10.6%) 4.7% 22.7% (26.1%) GDNet YouTube channel 5.0% (5.1%) 14.7% (14.1%) 25.3% (22.7%) 16.5% (15.8%) 6.8% 31.7% (42.3%) GDNet Twitter 4.8% (3.8%) 13.0% (12.0%) 23.5% (23.4%) 18.9% (17.5%) 7.4% 32.3% (43.3%) GDNet Community Groups 6.9% (12.0%) 20.1% (25.8%) 25.3% (29.7%) 13.0% (9.0%) 5.4% 29.3% (23.4%) 3 GDNet blog - http://gdnetblog.org/ GDNet Twitter – https://twitter.com/Connect2GDNet GDNet YouTube – http://www.youtube.com/user/gdnetcairo GDNet LinkedIn - http://www.linkedin.com/company/gdnet
  17. 17. 16 In broadest terms, GDNet has maintained overall high levels of satisfaction with key services in line with both the baseline and Year 1 figures. However, GDNet is aiming for a 5% year on year increase in satisfaction as defined by the output 1 indicator 1 milestone. This has not been achieved but may be considered unrealistic given the stable and established nature of GDNet’s key services – newsletters and access to online journals. It is possible to draw a number of headline conclusions from the results:  In line with both the baseline and Year 1 findings, GDNet users seem satisfied with the key services provided by GDNet. The KnowledgeBase online papers rated extremely useful by 28% (Year 1 - 36%) and moderately useful by 34% (Year 1 - 28%) of users. This represents a slight decrease in satisfaction on both the baseline and Year 1 figures but still represents a high overall level of satisfaction.  Other services rated as extremely useful include: GDN announcements (38%); Funding Opportunities newsletter (37%) and, accessing the online journals (35%). As has been clear from the outset, a priority for Southern researchers is accessing information on potential funding opportunities and funding sources. This binding constraint is well known to the GDNet yet is one that cannot be simply overcome through the provision of information and knowledge. Lessons on the use and satisfaction of GDNet research-orientated online service derived from the GDNet Gender Audit The starting point for the GDNet Gender Audit was analysis of the member survey responses from 2011. While the main reasons for registering with GDNet were the same for the male and female members (gaining funding information, data for their research and to access online journals) there were differences in relation to use and satisfaction. The primary reason for using GDNet for women is email newsletters, while for men it is the online database of papers; male GDNet members are nearly twice as likely to visit the GDNet website on a weekly basis than female members and are also more likely to have used GDNet’s social media products than female members. This analysis and the subsequent review of literature about the gender digital divide suggests the need for separate milestones for men and women for use of and satisfaction with GDNet's services, and even for individual services. From sharing the Gender Audit with others in the sector, it has also become clear that there is little value in reporting on the ratio of men to women in terms of use of services unless a relevant benchmark is available e.g. the proportion of men to women among southern researchers. On the basis of the Gender Audit, a question was added to the 2012 survey to help GDNet identify if there are different content needs for men and women. The full report produced by GDNet consultant Cheryl Brown can be found here - http://cloud2.gdnet.org/cms.php?id=implementing_gender_audit_of_online_knowledge_service The Year 1 M&E report highlighted two issues for GDNet to pay further attention to in Year 2. First, a number of services were less highly rated by survey respondents. Amongst GDNet’s ‘core’ services, the KnowledgeBase researcher and organisational profiles and Regional Window portals are deemed to be moderately and somewhat useful by the majority of users. This remains the case in Year 2. In order to better understand this an additional question was added to the Year 2 survey which asked responders how important it was to them for GDNet to continue to offer certain services. A summary of the results of question 14b is provided below:
  18. 18. 17 Answer Options Very important Quite important Not important Response Count Access to JSTOR and Project Muse online journals, based on eligibility criteria 547 (73%) 157 (21%) 39 (5%) 743 A searchable database of researchers for you to make contact with 453 (61%) 237 (32%) 41 (6%) 731 A webpage on the GDNet website for you to share your contact details, research interests and papers 427 (58%) 259 (35%) 50 (7%) 736 Opportunity to participate in GDNet Online Community Groups 349 (48%) 294 (40%) 84 (12%) 727 The results are interesting as they tend to indicate that whist access to journals is seen as a core GDNet service, researcher and organisational profiles are also seen by GDNet users to be very important, with a strong implication that they would be missed if they were closed. This is in slight contrast to the findings of question 14a which hinted that these services were deemed only slightly useful. Second, the results in both Year 1 and Year 2 point to only limited use and satisfaction with GDNet’s newer ‘Web 2.0’ tools and platforms (GDNet’s feeds, YouTube channel, Twitter, and Community Groups) which are considered only somewhat useful by between a fifth and a quarter or respondents in both Year 1 and Year 2. Positively, the Year 2 results indicate a small but significant increase in GDNet user awareness of these services. For example, 42% of users were unaware of GDNet’s YouTube Channel in Year 1. This has fallen to 32% in Year 2. Statistics on the channel itself tend to reflect relatively strong usage with a total of 29,866 views as of 19 March 2013. Similarly, 43% of users were unaware of GDNet’s Twitter feed in Year 1. This has fallen to 32% in Year 2. Whilst these statistics point to increased awareness amongst GDNet’s users, the GDNet team would also be keen to point out that high satisfaction and use of services of this nature is unlikely across GDNet entire, rather amorphous and passive user base. The ‘90 – 9 – 1 rule’ is generally accepted by knowledge brokering experts whereby 90% of knowledge network or platform members are passive and engage ad hoc and periodically, 9% of users are passive but visit regularly, and only 1% can be considered engaged users who constructively contribute to the platform or network. Therefore it is sensible of GDNet to develop a strategy that aims to engage the 9% and 1% of regular users. Therefore, the role GDNet’s Web 2.0 tools and services play with this smaller but active ‘core’ user base is what is important as well the how the tools allow GDNet to engage with different users in different contexts and at different times – at conferences and training events etc. With this in mind a more nuanced interpretation of the web survey results tends to indicate that the GDNet team is getting more sophisticated at engaging and catering for the needs of different users across their user base, combining established core services to general users as well as a range of services for more engaged users. There are a small number of anomalies between the webstats and the web survey, highlighting some issues that require further interrogation and explanation by GDNet during Year 3:  More detailed and nuanced interrogation and investigation on gender-based demand and use of GDNet services – see table above.  Free access to online journals - Since the programme was conceived, but particularly during Year 2, there has been a big change in policies on open access. Most recently the World Bank and the Welcome Trust both launched open access policies; JSTOR has opened free access to Africa, but now restricts some access to India and Pakistan). In contrast, GDNet access to JSTOR and Project Muse online journals is viewed by users as a core service (73% of users believe it is very important that GDNet continue to provide free journal access) yet the web stats indicate that there is an average of only 84 online journal sessions through GDNet per month. GDNet should further investigate this anomaly in order to establish the continued value of free access to online journals in an era of increasing open access.  Visitors from the Global South - The percentage of visitors to GDNet from the Global South appears to be between 30 and 40% and may be falling (32% in Year 2 compared to 39% in Year 1) according to
  19. 19. 18 the web statistics, yet the researchers from the Global South are main target audience for GDNet. This has a number of implications: o GDNet should be more explicit about how the web stats identifying visitors from the Global South are defined and generated. Generating location statistics based on IP address is notoriously unreliable. The web survey results seem to suggest that a higher proportion of GDNet users are based in the Global South than the web stats indicate. Web survey question 3 lists the top 8 countries in which users live with the USA (no. 4 with 2.7%) as the only Western country and India (18.3%) and Nigeria (6.7%) first and second respectively. o In terms of increasing users from the Global South, GDNet initiated the Connect South campaign during Year 2. 4 The web survey shows that one third of users have heard of the campaign which is reasonable given the campaign was established during Year 2. During Year 3 GDNet should devise a couple of questions to add to the web survey in order to assess the impact of the campaign as well as any new knowledge it has generated in terms of GDNet better profiling GDNet users and their needs, perhaps disaggregated according to general and engaged users as well as any difference between Southern users in different locations. Informal feedback from the GDNet team indicates that, through their experience working with different sets of researchers in different countries, the team is increasingly aware of subtle but important differences in capacities and knowledge needs across the Global South. As far as possible GDNet should express these differences and tailor their services accordingly.  Use and value of thematic windows – The thematic windows, launched and piloted in Year 1 as a key value added service for engaged users, are dealt with separately under output 1 indicator 2 below.  As mentioned in the note under the Outcome section, the 2013 DFID Annual Review raised a concern over the validity of any findings and conclusion drawn from the GDNet Annual Web Survey which had a response rate of only 5.4%. Responding to this feedback, the M&E team will investigate ways to improve the survey in terms engaging a more representative sample in Year 3. M&E approach The M&E of the level of use of, and satisfaction with research-orientated online services combines GDNet’s monthly web statistics with data generated from the annual GDNet user base web survey. Data management plan Karim Sobh  Design, testing and monthly production of standardised GDNet web statistics report. Shahira Emara  Monthly collection and quality assurance of web statistics Robbie Gregorowski  On an annual basis – assess level of use of research-orientated online services over previous 12 months through analysis of web statistics and through the annual GDNet users web survey, and report on findings against baseline and lesson learnt to GDNet. Evidence base A detailed explanation of the process used to generate the web statistics and GDNet user base web survey can be found in the Baseline and M&E Framework report.  Annex 1 provides an outline of the Year 2 web survey questionnaire.  Annex 2 presents the results and a brief analysis of the Year 2 web survey responses. 4 http://www.connectsouth.org/
  20. 20. 19 Indicator 2 - Level of use of and satisfaction with themed services Year 2 summary – Relatively low initial use of themed services combined with moderate satisfaction expected to improve in a group of engaged users as themed services become established and further rationalised. GDNet piloted a beta version of 11 themed services on 13 July 2011, over half way through the Year 1, and launched the full set of 23 themed services on 11 November 2011. Therefore the full set of 23 themed services have been established and further piloted throughout Year 2. Analysis of the web survey and web statistics indicates that:  Overall use of the thematic windows is relatively low in comparison to GDNet’s more established core services. However, this is to be expected of a service is designed to cater for the thematic needs of more engaged users. The pie chart below presents the average monthly number of hits to the top five thematic windows (Agriculture, Education and training, Environment and climate change, Health and, Information and Communications Technology). Analysis of the limited webstats that GDNet is able to generate that relate directly to themed service usage indicates that these five themes are clearly the most popular – these 5 themes account for 30% of total themed service visits across all 23 themes. By contrast the least visited five themes (Labor & Social Protection, Private Sector Development, Law and Rights, Transport, and International Affairs) account for only 12% of total theme visits. Or looking at it another way – the top 5 themes receive on average three times as many hits as the bottom five. It is difficult for GDNet to define the precise level of use as disaggregation of different ‘pathways’ by which users navigate the site is not feasible. So, for example, a thematic user may view a number of research abstract highlighted through a thematic window. This activity will be counted as a Research Paper abstract view on the web statistics but the pathway to the view through the thematic window will not be identified. The implication is that the web statistics indicating high overall use of the KB pages as well as the high number of research abstract views are likely to some extent to be attributable to the thematic windows.  The web survey also points to a small but growing group of users who find the thematic windows useful. 43% of survey respondents find the thematic windows either extremely or moderately useful with a further 26% finding them somewhat useful. There is also some evidence of improved awareness by users of what is still a new GDNet service – 27% of users were unaware of the thematic windows in Year 1. This figure had fallen to 20% in Year 2.
  21. 21. 20 GDNet will use the experience gained from the extended piloting of the 23 thematic windows during Year 2 to rationalise their scope and enhance their utility to users. Effectively maintaining and facilitating 23 themes is not feasible given GDNet’s resources as the themes require continual content curation in order to remain relevant and ‘fresh’. Therefore, based on the usage statistics, GDNet is likely to reduce the number of themes from 23 down to approximately 10 during Year 3. According to the average monthly hits per month across all 23 themes as presented in the table below, a natural split seems to occur between theme 11 – governance and those below it. It is anticipated that focussing on a smaller set of themes will further enhance the level of use and satisfaction as the themes become established as a core value-added GDNet service.
  22. 22. 21 M&E approach Level of use themed services will be more intensively monitored in Year 3 using web usage statistics. Web statistics are likely to include:  Number of hits each thematic window front page.  Where functionality allows in the future, the quality of thematic window usage – Participants entering into online discussion, submitting content to micro-site (feasibility to be further discussed with GDNet following thematic window rationalisation anticipated in 2013) Data management plan Shahira Emara  Day-to-day – management and facilitation of themed services including generating web statistics on the level of use (reporting monthly but analysed quarterly). Robbie Gregorowski  On an annual basis - assess thematic service satisfaction through the annual GDNet users web survey as well as designing short web survey targeted at thematic micro-site users Evidence base  Annex 1 provides an outline of the Year 2 web survey questionnaire.  Annex 2 presents the results and a brief analysis of the Year 2 web survey responses.
  23. 23. 22 Output 2 - Researchers better able to communicate their research to policy Indicator 1 – Researchers’ confidence and ability to communicate their research – immediately following capacity building effort Year 2 summary – On a ‘before and after’ self-assessment scale (where 0 = not at all confident and 5 = very confident to communicate their research to policy), the average capacity building participant increases from 2.0 before to 3.0 (a 50% increase) after a GDNet capacity building effort. Similarly in terms of ability, the average participant increases in ability from 1.8 before to 2.7 (a 50% increase) afterwards. During the Year 2 period GDNet conducted 3 training events with successful completion participant numbers as follows: Workshop 7 GDN Awards and Medals Finalists Hungary 10 participants Workshop 8 GDNet-AERC policy brief training Kenya 19 participants Workshop 9 GDNet-AERC policy brief training Tanzania 13 participants Total 42 participants A summary of the ‘before and after’ confidence and ability scores generated across the three research communications capacity building events conducted by GDNet during Year 2 is provided below: Workshop 7 Awards & Medals Presentation Skills June 14-15, 2012 Budapest, Hungary Average before Average after Average increase Confidence 1.8 2.3 0.5 Ability 1.8 2.5 0.7 Workshop 8 GDNet-AERC Policy Brief Training Workshop July 2-3, 2012 Nairobi, Kenya Average before Average after Average increase Confidence 2.3 3.3 1.0 Ability 1.9 2.8 0.9 Workshop 9 GDNet-AERC Policy Brief Workshop 29-30 November, 2012 Arusha, Tanzania Average before Average after Average increase Confidence 2.0 3.3 1.3 Ability 1.8 2.7 0.9 Overall these results produce average before and after confidence and ability figure as follows:  Average before confidence score 2.0  Average after confidence score 3.0  Year 2 - average increase in confidence 1.0 (50%)
  24. 24. 23  Year 1 – average increase in confidence 1.3 (50%)  Baseline – average increase in confidence 1.2 (39%)  Average before ability score 1.8  Average after ability score 2.7  Year 2 - average increase in ability 0.9 (50%)  Year 1 – average increase in ability 1.6 (89%)  Baseline – average increase in ability 1.1 (38%) Confidence – Across the 3 GDNet events from June – November 2012, the average GDNet researcher is moderately confident (2.0 out of 5) to communicate their research to policy before any training/capacity building has been provided by GDNet. This average confidence figure rises to 3.0 immediately following a training / capacity building event. This equates to an average 50% increase in confidence immediately following a capacity building event. This matches the Year 1 figure. This figure demonstrates that GDNet’s training and capacity building activities are effective in terms of the immediate transfer of confidence to attendees. Ability – Across the same events, the average GDNet researcher is slightly less able than confident (1.8 out of 5) to communicate their research to policy before any training/capacity building has been provided by GDNet. This average ability figure rises to 2.7 immediately following a training / capacity building event. This equates to an average 33% increase in ability immediately following a capacity building event. This is significantly down on the Year 1 average ability increase GDNet was able to deliver. These results imply a number of broad conclusions:  GDNet continues to provide effective training and capacity building activities which demonstrate a significant and immediate transfer of confidence and ability to attendees. The GDNet team have defined an effective approach to training and capacity building and are competent and confident in its delivery.  Between Year 1 and Year 2 GDNet has not been able to deliver an increase in the magnitude of confidence and ability scores as they did between baseline and Year 1. Rather the magnitude of the increase in confidence transfer seems to have stabilised broadly in line with the Year 1 figures. The magnitude of the increase in ability transfer is considerably lower than the Year 1 figures. This divergence between the increases in confidence and ability is hard to explain. One possible explanation is that following the training, researchers are confident (in terms of knowledge transfer) but have not yet put the knowledge into practice (feel less able).  It is interesting to note that the magnitude of the increase in confidence and ability for the Awards and Medals Finalists (workshop 7) are significantly lower than the other two workshops. Discussing this with the GDNet team reveals a couple of explanations and conclusions: o As ‘eminent’ / ‘emerging’ Southern researchers, the Awards and Medals Finalists are likely to have more specific and advanced research communications needs and demands. Feedback from them tends to indicate that they would like specific training on using social media and other online platforms and tools to support research communications rather than broader training in drafting and presenting policy briefs etc. o One implication is that GDNet should attempt to tailor the training and capacity building events according to their understanding of the attendees. In particular, GDNet should pilot a more advance and challenging event for the next cohort of Awards and Medals Finalists at the 2013 GDN Conference in Manila. o More broadly, the GDNet team feel that they now have a better understanding of Southern researchers’ needs based on experience of conducting training in a variety of contexts. Put simply, researchers emanating from Asia and Latin America tend to want to focus more on the use of social media for research communications and enhancing their skills in this area. Researchers from Central, Southern and Eastern Africa tend to need support with key research communications skills such as synthesising key messages, drafting policy briefs and
  25. 25. 24 putting together presentations. This perhaps reflects their own challenges in accessing resources and technology. In future, the GDNet team will endeavour to tailor the content and objectives of each of the workshops according to their understanding of the nature of skills and capacity of specific sub-sets of Southern researchers. These and other lessons to have emerged from GDNet’s output 2 activities are reflected in the output 4 indicator 1 log template, indicating GDNet’s enhanced capacity as a learning programme. In response to the DFID annual review comments, as well as mean increase in confidence and ability we have also calculated the median increases which are presented in the table below: Workshop 7 Awards & Medals Presentation Skills June 14-15, 2012 Budapest, Hungary Median before Median after Median increase Confidence 2.0 2.4 0.6 Ability 2.1 2.8 0.6 Workshop 8 GDNet-AERC Policy Brief Training Workshop July 2-3, 2012 Nairobi, Kenya Median before Median after Median increase Confidence 2.3 3.4 0.9 Ability 1.9 2.8 0.9 Workshop 9 GDNet-AERC Policy Brief Workshop 29-30 November, 2012 Arusha, Tanzania Median before Median after Median increase Confidence 2.2 3.6 1.2 Ability 1.9 2.9 1.1 Overall these results produce average median before and after confidence and ability figure as follows:  Median before confidence score 2.2  Median after confidence score 3.1  Year 2 - median increase in confidence 0.9 (41%)  Median before ability score 2.0  Median after ability score 2.8  Year 2 - median increase in ability 0.9 (45%) When based on median figures rather than mean figures, the increase in confidence and ability is slightly lower (41% and 45% respectively compared to 50% on both when based on mean confidence and ability scores. Note on the GDNet capacity building M&E in response to comments made in the GDNet DFID Annual Review – The annual review report recommends that ‘The programme should consider strengthening its evaluation of training by for example testing written material produced before and after training, as self-perception of improvement may have little bearing on reality.’ The rationale underlying the capacity building M&E approach (and its basis in self-assessment) was elaborated in the GDNet M&E Baseline report and draws on established international best practice in the assessment of capacity building initiatives. The rationale is to link relatively
  26. 26. 25 short term changes as a result of a capacity building event (reaction and learning) to more significant changes in terms of behaviour and results. For the benefit of the DFID annual review team who may be unfamiliar with the approach set out in the baseline report, it states: ‘The approach to monitoring and evaluating researchers’ confidence and ability to communicate their research draws heavily on Kirkpatrick’s training evaluation model and employs a tailored questionnaire to assess confidence and ability before and after training. The four levels of Kirkpatrick's training evaluation model essentially measure: 1 - Reaction - is how the participants felt about the training or learning experience. 2 - Learning - is the measurement of the increase in knowledge - before and after. 3 - Behaviour - is the extent of applied learning back on the job - implementation. 4 - Results - is the effect on the business or environment by the participant.’ The DFID annual review suggests testing the written material produced before and after training as a more robust approach. Whilst we disagree with testing written material as a more robust approach to establishing changes in confidence and ability before and after a training event, GDNet will endeavour to apply such an approach to complement the established self-assessment process going forward. It is anticipated that such an approach will define a set of ‘key attributes of a policy brief’ criteria to be scored before and after a training event. In the interest of simplicity and in order to avoid subjective judgements it is anticipated that these criteria will be based on absolute statements, providing a before and after score out of six for each participant than can be averaged event by event: 1. Is the policy brief concise (less than 2 pages)? Y/N 2. Is the policy brief presented in an attractive and easy to read format? Y/N 3. Does the policy brief contain a concise summary of a particular issue? Y/N 4. Does the policy brief convince the reader that the problem must be addressed urgently? Y/N 5. Does the policy brief present one or more options to address the particular issue? Y/N 6. Does the policy brief appear to be based on a relatively robust research process or wider evidence base? Y/N M&E approach for Output 2 Indicator 1 and Indicator 2 GDNet activities under Output 2 revolve around a series of region-specific and thematic mentoring, capacity building and training workshops for a range of researchers / GDNet stakeholders on research communications and writing for policy relevance. Participants’ confidence and ability before, immediately after and 3-months after the workshop are assessed through a questionnaire and follow up email survey. This provides both an immediate before and after rating as well as a richer, qualitative assessment of the ‘impact’ of the training 3- months later through tracing ‘pledge’.
  27. 27. 26 Indicator 2 – Researchers’ confidence and ability to communicate their research – sustainability of capacity building effort Year 2 summary – Second set of six ‘pledge’ cases developed from GDNet workshops 7-9 held during Year 2 reflecting the sustainability and application of GDNet’s capacity building effort. Four of six Year 1 cases developed with 3 and 12-month follow up. It has always been the understanding of the GDNet team that increased confidence and ability immediately following a capacity building event is not particularly meaningful. Rather what is more important is a long term and sustainable increase in confidence and ability, and what this means for how these researchers do their jobs. Output 2 indicator 2 assesses this using the ‘pledge’. The long term sustainability and impact of GDNet’s capacity building efforts are assesses 3-months after each workshop through a ‘pledge’. Each participant is asked to respond to the following: Question – What will you do differently as a result of attending this workshop? Pledge – ‘Within the next 3 months I will…’ All 3 and 12 month pledge follow up data GDNet received to date is presented in chronological order by workshop / event in an Excel database. A sample of the most informative ‘pledge’ statements generated in Year 2 (workshops 7 to 9) is presented below. Workshop 7 – Awards and Medals Presentation Skills Pledge : I will be presenting my papers without any hesitation. 3-month follow up: After coming back from Budapest, I have shown improved communication skills both in terms of interpersonal as well as professional level. The personal guidance extended by your communication team was superb and excellent. This has helped me to modify my presentation to the audiences suitability and helped to formulate my oral presentation with the relevant wording and communications. I have submitted a paper for the presentation whose wording of the findings were critically appreciated by the climate change community in India. Further, I am presenting several papers and oral presentation in India pertaining to my works. My presentation on "Critical Concerns in Social Sector Budgeting in India" and also "Budgeting for Water Resources Development in India" have been greatly appreciated. Further, the communication skills learnt at Budapest meeting have helped to sharpen my advocacy skills which are bearing significant result in my access to governmental apparatus Workshop 8 – GDNet-AERC Policy Brief Training Workshop - Nairobi Pledge : I will submit my policy brief to AERC for publication; I will organize a national policy workshop where we will share our findings with policymakers and the media. 3-month follow up: l have submitted my policy brief to the AERC for publication. Concerning pledge number 2, we had the national policy workshop on September 13, 2012. The title of our project is "the effect of socioeconomic and national health insurance status on household demand for prenatal and postnatal health care in Ghana", policymakers from the Ministry of health, the national health insurance authority, NGOs, midwives, medical doctors and the media were invited and findings were shared with them. The dissemination took the form of a 20 minute presentation of the research findings, followed by a discussion. The participants were happy that we had undertaken such a study, and expressed their appreciation for the useful recommendations we made to them. My participation in your workshop helped me greatly in the selection of the participants (audience) for our workshop and also in deciding on the mode of communicating the findings and the policy recommendations.
  28. 28. 27 Pledge : Prepare policy messages from my previous research work and also help others doing so 3-month follow up: Let me start by saying that the Training on Policy Brief in Nairobi was more than useful to me and my department. When I returned from that training, I shared the materials with my colleagues at the Economic Policy Analysis Unit of the ECOWAS Commission. I also made a presentation that helped them grasp the fundamental message from your training. Since my arrival from Nairobi, we have been preparing Policy briefs from our in-house research. That really helped us to present our research findings to our Management team. To sum it up, your training was very timely and useful to all of us into research and policy analysis. I am really very grateful to you and the AERC for making this possible. Pledge : I will make sure that our policy brief and other briefs that I will write from other research papers I have done are out for dissemination; and will also try to write for newspapers in my country 3-month follow up: Pledge 1: I worked on AERC policy brief and submitted it. We have printed copies of the brief and disseminated them to policy makers and comments are positive. People appreciate the briefs and this means that u did a wonderful job. At the institute where am based as a research fellow (Policy analysis and development Research Institute (PADRI)) I am writing policy briefs from IDRC Project on Social Protection in Uganda. We are also thinking about training other researchers from Universities, NGOs and research organization in Uganda Under PADRI on how to write policy briefs using the experience from workshop. This is because we have realised that so many researchers in Uganda do not know how to write policy briefs. This is at thinking stage though. Pledge 2: I have not got any headway on this but we have opportunity to write for upcoming regional newspaper which will be running weekly in Uganda and Rwanda. We got slot to start writing topical issues on economy. Pledge : Try to disseminate my research outputs in line with what I have learned; give my first press-conference. 3-month follow up: I had planned for the country workshop this December, unfortunately, I had a poor response from policy makers because of timing and I am postponing it to February/march next year. I have not been able to give a press conference yet. I have not chickened out. I will do it at the appropriate time. Please, I feel like discussing this idea with you. I need help to be connected to people and organisations that can help me kick start it. Pledge : Select the targeted audiences; select the right media to access the audiences; get the message through in repeated efforts 3-month follow up: As you know in Africa everything is politics and politics driven. Research outputs are not seen as potential inputs for policy, rather as propaganda tools. Most of the time only favourable findings are welcome. To be frank I don't feel very free in communicating research outputs. I have to be cautious. One of my worries was that our research output was covering the period 1994 to 2004. Findings in this period may not be seen as relevant in Ethiopian context and in the eye of those who influence policy. They could be interpreted as picking the bad moment for defacing the performance. Therefore I had to extend the research coverage to cover the years after 2004 and bear the consequence. I have done that to a satisfactory level (with a little left) and I have planned a dissemination workshop for November 2/2012. I have made a preliminary selection of my audience including people from the media. Two of my pledges are on the way and the third is a sequel. Workshop 9 – GDNet-AERC Policy Brief Training Workshop - Arusha
  29. 29. 28 Pledge : I will organise a dissemination workshop of my research on the impact of foreign aid on growth factors in Sub- Saharan Africa. 3-month follow up: The policy brief Training Workshop was a real opportunity for me to understand major advances on research communications. The workshop familiarized me with a broad range of writing of Policy Brief. In fact, the workshop gave me tools to write a good policy brief. I will organize the dissemination workshop of my research at my Research Centre. Actually, the paper has been sent to AERC to the final evaluation. I am waiting comments and suggestions from AERC reviewers to improve the quality of paper. After taking in account suggestions and comments, i will organize the workshop dissemination. At this moment, i could not give you a schedule. Pledge : I will prepare a policy brief from my research project and a new article for consideration by the press/media 3-month follow up: I have in position to prepare both the policy brief and media article using the knowledge gained from the workshop. We have not published the media article because my research colleague advised that we could share it with the media after the dissemination workshop of our project to policymakers and academics within Uganda. I intend to distil policy briefs for some of my other research work and if acceptable I could share them with you or Andrew for review. Pledge : I will attempt to make my research more relevant. 3-month follow up: The greatest obstacle is really breaking the barriers to reach out to the intended research consumers, in particular those in policy arena. Nonetheless, efforts in this direction are on-going. As well as generating new pledges and follow up in Year 2 from workshops 7 to 9, GDNet has also engaged past GDNet training and capacity building recipients to provide 12-month follow up on their pledges. Due to an expected non-response rate over pledge, 3-month follow up, and 12-month follow up, the sample after 12 months is much smaller. Nevertheless, a significant number of attendees have responded with 12-month feedback on their pledges. A sample of these from workshops 1 to 6 conducted in the baseline and Year 1 is provided below: Workshop 5 - PEM Research Communications Workshop Pledge : Redo the policy brief and contact media on communicating the results 3-month follow up: We have redone the policy brief on water sector a couple of months after the workshop. We have communicated some of our recent results to donors and other stakeholders, however, as the on-going products are not yet finalized we have not prepared anything that can be disseminated to media. As indicated in Delhi workshop, we do plan to communicate some of our results to media institutions. So, half of the promise is done. We will be communicating the progress (including our communication outputs) to GDN. 12-month follow up: Last year we have made 6 town-hall meetings with around 300 citizens from Yerevan (capital) and 2 towns and several surrounding villages. That communication/dissemination/outreach mode was the primary mode for us during the last year as we were in the process of consultations during the finalization of our policy simulation papers. We have submitted our policy simulations at the end of the 2012 but we still in the process of final formulation based on the feedback from GDP technical advisers. Thus, the papers are not final yet to print and disseminate, therefore no media intervention was made as of today.
  30. 30. 29 Pledge : Set up a network among stakeholders related to policymaking process 3-month follow up: The process to set up the network is still going. However, there are several obstacles to implement the idea. Based on the plan, we will start the network creation process through an FDG that involves stakeholders related to education, health and water sector. The purpose of this FGD is to explore and gather information about any development issues related to education, health and water sector, particularly the issues that happen recently. Also this FGD process will try to set up a strategy on how each stakeholder can take role in policy changing. This FGD is not happened yet, since it is quite difficult to find a time to gather all those people. Besides that it is quite hard to find some people who have the same concern or interest with CEDS, mainly for this project. So far, the effort that I have done to start this network is by identifying the stakeholders and make a list of them. This process is done by conducting some informal meeting with some people. I hope the FGD can be held quite soon. I plan to set up the FGD in the middle of March. 12-month follow up: * The FGD was held around July last year. On that FGD we invited several people from different background, such as NGO workers, health expert, government budget expert, and academicians. At this first FGD we agreed to establish a network between academicians and CSO activists. After this FGD we held several regular discussions. Normally the discussion was held once in a month. * During the discussion process we found that creating network among those actors in not really easy, mainly in finding the common interest from all the participants. We thought there is a need to create some activities that involve all the participants. * Due to this reason, on the last discussion that was held on November last year, we agreed to held a workshop that invites different stakeholders who have concerns on public policy. We took health sector as the main issue for the workshop. The purpose of this workshop is mainly to design the advocacy strategy to make a change of a public policy, especially on health issue. Unfortunately, because of the workload each participant, this workshop still cannot be executed. So far, the curriculum of the workshop has been designed, we're still waiting for the right time to run the workshop. Perhaps next month. Pledge : Prepare a press release and a policy brief 3-month follow up: Following your guidelines during the October PEM Communications Workshop in New Delhi, we have prepared three policy briefs for each of the three sectors on education, health and water services. Policy briefs have not been disseminated because we are in the midst of preparation for a major conference to present these policy briefs to our target audience composed mostly of policy makers and sectoral stakeholders. The Conference will be held on March 13 of this year. A press release will come shortly prior to the Conference. We are also at the thick of preparation in launching our website to be able to disseminate the results of our study on Strengthening Institutions in Public Expenditures Accountability. The PEM Asia Workshop has significantly helped us in crafting the requirements of GDN especially in the formulation of policy briefs and technical presentations. 12-month follow up: 1. Prepared one (1) policy brief on the health sector using the BIA as tool of analysis. 2. Launched our PEM webpage linked to the Center for Research and Communication website. 3. Conducted Roundtable discussions with policymakers such as members of the Congress and Department officials and key stakeholders such as CSOs and private organizations interested in social responsibility programs in health, water and education. 4. On January 29, we have been invited to a roundtable discussion with Senator Ramon Magsaysay Jr--an auspicious chance to inform him of the results of our PEM research, focusing on the need for evidence-based research specifically on public accountability related to budget expenditures on the three social sectors of education, health and water. The BIA tool of analysis will be highlighted in this activity. 5. The March 2012 conference did not push through but on February 26-27 this year, we will finally conduct the Policy Conference on the Issues, Challenges and Initiatives facing the Education, Health and Water sectors. The
  31. 31. 30 conference seeks to raise the level of awareness on policy issues and concerns affecting the three social sectors. The research outcomes we shall present in the said conference can aid development policymakers in making informed judgments in formulating strategies and pushing for reforms in the three sectors. The conference will also discuss current initiatives in addressing policy issues/challenges which the participants may find appropriate and useful in their respective organizations and communities. Policymakers, members of POs and NGOs, the academe and selected members of media will be in attendance. Workshop 6 - GDNet-AERC Policy Brief Training Workshop Pledge : Review the way of writing key abstracts, presenting the results of my work to colleagues, policymakers and media 3-month follow up: I am currently at the Centre of Studies of African Economies (CSAE), Oxford University. And, today (29.02) (between 1-2.30 PM local time) I presented my paper titled "Trade liberalization, labour market reform and firm's labour demand: evidence from Cameroon" at the CSAE seminar. The feedback I received show that, relative to the past (i.e. before the workshop in Nairobi) I made a lot of progress regarding two points: (1) the abstract, and (2) policy implications of the results. You remember we had a group work on (i) the meaning of policy implications of the results, and (ii) how to present them. This shows that I made a lot of progress on writing the abstract as well as the presentation of the policy implications. However, and still from the feedback I received, I still have some problems on how to present the results, namely the background information. Any assistance from you or GDN is still welcome. 12-month follow up: Yes I attended the ATPS annual workshop in 2011 and i presented my paper co-authored with my colleague (Bayo Ajala) titled "Towards effective research uptake and innovative communication of research projects in Nigeria ". Yes the output has been published in the proceedings of the workshop which is available on the ATPS website. I work for the government and there are laid down protocols to sharing the results of our project. On my part l have been able to demonstrate part of what I have learnt in many of the projects I am involved in at work The pledges provide not only an insight in to the nature of the application of the capacity building but also a very clear link from training to increased confidence / ability to direct application by the researchers – the sustainability of the capacity building effort. A number of pledges point directly to higher order outcomes and possibly even impact (all be it small scale) as a result of GDNet’s capacity building efforts. For the purposes of GDNet M&E, some also point to researcher interaction with the policy domain as a direct result of GDNet support and hence should be captured and claimed under output 3 indictor 2. The follow up, particularly the 12-month follow up, also points to the complexity of any change pathway from enhanced research capacity to informed policy. This clearly highlights the limited extent of GDNet’s contribution as well as the extent to which change is dependent on multiple exogenous factors. That said, now that GDNet has 3 years of training and capacity building experience, there is an opportunity for GDNet to more systematically interrogate the pledge follow up. This will allow GDNet to synthesise any lessons that have emerged as well as trace the change pathways of any particularly interesting or insightful pledges - from the provision of training right through to the application / use of new skills and knowledge in terms of higher order outcomes or impact. Ultimately this may allow GDNet to start to make some claims about GDNet’s contribution to enhanced research uptake and evidence informed policy through training and capacity building. Data management plan Robbie Gregorowski / ITAD  Design and testing of workshop questionnaire template and results Zeinab Sabet / GDNet  On-going – defining confidence and ability statements in advance of each workshop,
  32. 32. 31  Facilitating questionnaire completion by participants at each workshop,  Recording results following each workshop in the results template,  Facilitating the 3-month ‘pledge’ email and telephone follow-up with a sample of participants (approx. 25%) following each workshop and completing the pledge follow-up template,  Facilitating the 12-month ‘pledge’ follow up with those who submitted 3-month follow up.  Synthesis of pledge results into a small number of cases on an annual basis,  Follow-up on training event feedback to extract learning for GDNet and feed this back into improved training and capacity building provision. Evidence base GDNet holds the capacity building workshop questionnaire responses, including the pledge statements and the 3-month follow up response in an Excel database designed by ITAD. It is not practical to include this as an annex but GDNet is happy to share the database with interested parties.
  33. 33. 32 Output 3 - Knowledge networking between researchers and with policy actors increased Indicator 1 – GDNet ‘user base’ interaction Year 2 summary – GDNet have continued to increase user base interaction in Year 2, demonstrating a maturing ability to deploy a range of tools and platforms (particularly social media) to engage a broad spectrum of GDNet users. GDNet user base interaction involves Southern researchers with whom GDNet has built a sustained engagement – through attendance at a capacity building event, conference, or membership of a community or thematic group. GDNet logs all its interaction with its ‘user base’ in a log template presented in Annex 5. The aim of the template is to set out ‘at a glance’ the nature of the interaction, the results that this interaction produces, as well as any lessons GDNet learns as a result of this interaction. The purpose of the log is to provide a ‘living’ document which GDNet staff can interrogate periodically in order to learn lessons on the nature of their interaction with their key set of stakeholders. The log is designed to be analysed and synthesised annually in order to establish the extent of user base interaction. Indicators of increased user base interaction will relate to sustained or even increased blog views and responses, sustained or increased subscriptions, views and ‘click throughs’ to GDNet social media such as Twitter and YouTube. It is anticipated that log will also include indicators of more strategic and in-depth user base interaction such as collaboration with specific partners to produce research communications products as well as panels and presentations at workshops and conferences. GDNet held their first team synthesis and learning retreat on 31 March 2013. A summary of sum of the key reflections, messages, and conclusions taken that were distilled during this event are presented below: GDNet Team user base interaction log synthesis On 31 March 2013 the GDNet team came together at their first learning retreat to identify, discuss and reflect on the lessons generated from implementing the programme over the previous year. This table presents the findings from the synthesis discussion of the output 3 indicator 1 log template on user base interaction. The GDNet team focused on discussing: the nature of the interaction GDNet has been facilitating throughout 2012 with its user base; interrogating the results of different types of interaction; and drawing out some reflections and lessons learned to be taken into consideration when undertaking future activities. A copy of the log template is presented in Annex X. This is designed to be a ‘living’ document which all GDNet team members are invited to input in and update. The key messages to emerge from the learning retreat are as follows:  Understanding of audience nature: GDNet have developed a better understanding of the nature of our different audiences. The log demonstrates that the level of engagement shown by a specific audience differs according to its nature; i.e. online users are definitely easier to engage at conferences and events than audiences that are unfamiliar with social media tools. Therefore, we have been trying throughout 2012 and through different activities to use different interaction tools (including but not limited to social media ones) to engage different types of audience.  Blog evolution: Although the use of social media was not defined in the original planning of the GDNet programme and logframe, there has evolved an increasing awareness of and dedication to its role and importance nowadays to the programme. This is due to the exposure it provides for the programme, as well as the global activities GDNet takes part of and which confirms social media as the most effective and powerful way to communicate today. Throughout 2012, the team has been working to better understand how to use social media as communication and interaction tools in GDNet’s different outputs and activities. As part of the integration process, GDNet are ensuring that team capacity is built and sustained accordingly. A GDNet Social

×