Web 2.0 and social media capacity building initiative - What have we learnt over the period 2011-2012? Results of an impact study
•Download as PPTX, PDF•
0 likes•676 views
Presentation of the findings of the external evaluation of CTA web 2.0 and social media training programme (2011-2012) - by Pier Andrea Pirani (Euforic Services), 28 March 2013.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (6)
Similar to Web 2.0 and social media capacity building initiative - What have we learnt over the period 2011-2012? Results of an impact study
Similar to Web 2.0 and social media capacity building initiative - What have we learnt over the period 2011-2012? Results of an impact study (20)
More from Technical Centre for Agricultural and Rural Cooperation ACP-EU (CTA)
More from Technical Centre for Agricultural and Rural Cooperation ACP-EU (CTA) (20)
Recently uploaded
Recently uploaded (20)
Web 2.0 and social media capacity building initiative - What have we learnt over the period 2011-2012? Results of an impact study
- 1. CTA 2011 – 2012 Web 2.0 and Social Media Capacity Building What have we learnt? What should you do? Pier Andrea Pirani 28 March 2014
- 2. Headlines Lots of data available Trainings work Recommendations on – Demographics – Training curriculum and format – Sharing and learning – Supporting organizational change – M&E
- 3. Assess programme against goals + expected results Context Programme goals 2011 2012 Output level 1. Increased awareness 2. Enhanced skills 3. Certified practitioners (UNITAR course) 1. Increased awareness 2. Training hubs 3. Institutional social media 4. Improved ICKM 5. Certified practitioners (UNITAR course) 6. Improved quality of CTA services
- 4. Context Programme goals 2011 2012 Outcome level Promoted adoption of digital tools Enhanced institutional capacity in the use of Social Media
- 5. Context Programme goals 2011 2012 Impact level Contributed to ACP sustainable ag and rural development Increased engagement of CTA beneficiaries to adopt ICTs
- 6. Results Impact Outcome Output 2011 Contributed to ACP sustainable ag and rural development Promoted adoption of digital tools 1. Increased awareness 2. Enhanced skills 3. Certified practitioners (UNITAR course) 2012 Increased engagement of CTA beneficiaries to adopt ICTs Enhanced institutional capacity in the use of Social Media 1. Increased awareness 2. Training hubs 3. Institutional social media 4. Improved ICKM 5. Certified practitioners (UNITAR course) 6. Improved quality of CTA services
- 8. Countries as of 2012 Source: Actual participants data
- 9. Who did you reach as of 2012? Women – 36% Youth (Under 35) – 56.3% Organizations – Research institutes, private enterprises, farmer’s organizations Sectors – Agriculture and Ag research, training and capacity development Source: Actual participants data
- 10. The training works 27.3% 42.0% 23.5% 7.2% 0.4% 6.8% 43.6% 49.2% <- [1] Basic [2] [3] [4] Advanced -> BEFORE the training AFTER the training Source: Impact survey
- 11. Satisfaction! 99% – Satisfaction with training 98% – New skills to work more effectively 98% – Inspired by web2.0 and social media potentials Source: End of event evaluation
- 12. Future plans of adoption 58% 64% 67% 69% 73% 74% 75% 79% 82% 0% 20% 40% 60% 80% 100% iMark RSS Feeds Skype Alerts Advanced Search Maps Wikis/Google Docs Social Networks Blogs Source: End of event evaluation
- 13. Actual adoption 53% 58% 60% 61% 66% 72% 75% 76% 84% 0% 20% 40% 60% 80% 100% Skype Google Drive Blogs Twitter Alerts Maps Advanced Search Linkedin Facebook Source: Impact survey
- 14. Adoption rates by type of event Source: Impact survey 0% 10% 20% 30% 40% 50% 60% 70% 80% Google maps Twitter Blogs Google Drive VoIP RSS Wikis F2F training Online training
- 15. Actual areas of improved work performance Source: Impact survey 61% 62% 69% 75% 82% 92% 0% 20% 40% 60% 80% 100% Info access & management Remote collaboration Communication Promotion & outreach Project management Linking with peers
- 16. Ripple effect 9% 28% 37% 40% 41% 71% 0% 10% 20% 30% 40% 50% 60% 70% 80% None Blogging/Writing iMark Internal trainings Partners' trainings Informal sharing Source: Impact survey
- 17. Common patterns and outliers Age U36 - Higher adoption rates for all tools Significant differences between two age cohorts – Twitter – 72% vs. 50% – Ning, Google Plus – 45% vs. 19% Gender Most popular applications - Facebook and LinkedIn Male respondents - Higher adoption rates – Wikis – 44% vs. 24% – Blogs – 65% vs. 48% – Dgroups – 29& vs. 16% Exceptions – Dropbox – Other social networking sites Source: Impact survey
- 18. Social media @ work 39% 41% 43% 44% 47% 48% 69% 0% 10% 20% 30% 40% 50% 60% 70% 80% More outreach Social media strategy Less comms costs Internal adoption Better info delivery More visibility More comms channel Source: Impact survey
- 20. Challenges Personal Lack of time to experiment Irregular internet access Preference for more traditional comms tools Institutional Limited resources (hardware and adoption) Lack of IT staff Resistance to change Lack of management support
- 21. Recommendations as of 2012 Demographics – Youth, female & Francophone trainees – Business persons, entrepreneurs, farmer organizations Training curriculum – Training of Trainers programme Sharing & learning – Documentation of success (and failure!) stories Organizational change – Sensitize senior and middle managers – Assess pilot with EAFF M&E – Focus groups for qualitative data & peer sharing
- 22. CTA 2011 – 2012 Web 2.0 and Social Media Capacity Building What have we learnt? What should you do? Pier Andrea Pirani 28 March 2014
Editor's Notes
- This presentation contains some of the findings from the evaluation of CTA web2.0 and social media training programme initiatives between 2011 and 2012. These fall under to types of events:Web2.0 and Social Media Learning Opportunities (face-to-face 5 days in country trainings)UNITAR online e-learning course (Tutored online course with certification upon successful completion)
- The report, currently being finalized, is quite rich. We have lots of data available and for the scope of this study three datasets have been used: All participants data at registration levelAll participants data at end of event evaluationOnline impact survey (24% response rate)Interviews with trainers and trainees from hosting organizations and CTA strategic partner organizations are currently in progress to finalize the findings of the study.In short, evidence indicates that the trainings work: trainees are satisfied with the training, they increasingly adopt social media to support their work and the training has started being replicated. The initial recommendation cover 5 main areas. Some recommendations are on the operational side of the programme, other more strategic. We’ll discuss more at the end of this presentation.
- 2011-2012 was the second phase of the programme, after the initial phase between 2008-2010. A previous evaluation is available. Building also on its findings, this new evaluation aims at assessing CTA programme against its stated goals and objectives.Activities in 2011 and 2013 fall under two different CTA strategic plans, hence there are some noticeable changes in the project goals and expected results. In particular, at the output level there is a move from creating awareness on web2.0 and social media to supporting institutional capacity and adoption of social media and web2.0 applications, especially to influence agriculture policy and supporting value chains development.
- This is also confirmed at the outcome level.
- Finally, the dimension of engagement is introduced at the impact level.
- If we use a simple traffic light system, we can see that most of the goals are met – green dots.In terms of developing institutional adoption, trainings are replicated and there’s a spin off in activities, so the trend is positive. Likewise for the institutional use of social media – it’s happening but at slower rate and in slower percentages than for personal adoption. At the impact level, this is probably too soon to tell – this takes time and it can be seen only in the long term. It is also better documented through stories and qualitative data in general, instead of the quantitative data that we use mostly in this study. In part this has stared, through the informal sharing on the web2fordev Dgroup and other online communities and the case studies and interviews we are collecting.
- Clearly in 2011-2012 the trainings took off. The number of initiatives CTA organised has more than doubled, likewise the number of trainees. Almost the majority of these attended one of the Learning Opportunities but CTA has also made more use of the online UNITAR training, sponsoring the participation of trainees selected from partners organizations. Increasingly, these are former trainees in the Learning Opportunities that have the potential or interest to act as trainers in the future.
- As of 2012, the training reached 16 ACP countries:-Africa (41 events, 13 countries)Caribbean (3 events, 2 countries)Pacific (4 events, 1 country)½ of trainees are from East Africa where the majority of the LOs took place
- Between 2011 and 2012, CTA has particularly favoured the participation in the training of specific profiles of trainees:Women, +8% over the two years and total of 36% of traineesYouth, +6% over the two years and 56.3% of total traineesTrainees from research institutes, private enterprises and farmer’s organization has increased over the two yearsMost of the trainees work in the agricultural sector. The majority is also working on training and capacity development – important element in the replicability of the training.
- In the impact survey, respondents are asked to rate their web2.0 and social media skills before and after the training. 1 is the lowest and 4 the highest. As we can see, before the trainings the majority of respondents indicate a level of 1 and 2. This change significantly after the trainings, with the majority of respondents rating their skills at 4 and 3, so in the high end of the spectrum. The modal value moves from 2 before the training to 4 after the training.Face to face events appear to perform better than online events.
- The way the training is organized perform incredibly well. It is significant to notice that the trainings work as an eye opener and that trainees are inspired and enthused with the skills and tools they learn and practice with.
- These are the results of end event evaluations, indicating the intentions trainees have of using the different tools. All the tools score quite high, with blogs on top of the list, followed by social networking sites and wikis/Google Docs, Google Maps, Google Advanced Search.
- The reality of actual adoption – from survey respondents, so only a subset of trainees – is different. Social networking sites are on top of the list – is this surprising?Maps and Google Advanced Search are also quite high. In both cases intentions and results are aligned.Less people do actually blog, when compared to intentions. Wikis have much lower adoption.It is also important to underline how the adoption rates for face to face events are in general higher than the ones registered for the online trainings. Indeed, the Learning Opportunities are characterised by intense hands-on exercises and peer learning and assistance, with participants sitting side by side and tending to help each other. In this sense, they appear to be more effective in equipping participants with skills and knowledge that they can turn into practice and active usage of new tools and approaches.
- Face to face training preform better: when the same tool is present in both curricula (face to face and online) the adoption rates from survey respondents are in general higher for the Learning Opportunities than for the online training with UNITAR.
- Specifically, where respondents fill they are getting better at? In our analysis, we have reduced the list of options to identify the main business functions a knowledge worker works on – reducing the list of options presented in the end of event questionnaire. More specifically, the analysis show that the outcome that has materialized more frequently amongst respondents regards the gains in efficiency in obtaining up-to-date information, followed by the documentation of work, using social media, and the use of more communication channels to promote the work done.
- According to the data, over 70% of survey respondents indicate having introduced web2.0 and social media to friends and colleagues working in the ARD sector. Very interestingly, over 40% of respondents say they have introduced social media to partner organizations. This is particularly impressive when considering that it almost doubles compared the first phase (2008-2010) of CTA web2.0 and social media capacity building programme.
- We have just picked a couple here that may inform our conversation at the end of the presentation. We focus here mainly on demographic profiles (age and gender) and see what are the common elements and the differences.
- When looking at social media adoption at the organizational level, survey respondents indicate different ways on which this happens. In particular organizations expand their visibility online and create more communication channels.
- The majority of respondents feel their organization has an intermediate level of social media adoption. However, there is still a 5% that indicates social media not being used in their organizations.
- These are some of the personal as well as institutional challenges that respondents have indicated as obstacles for broader social media adoption. Some challenges may be obvious, some other less. It would be interesting to see what CTA can do to influence some of these. For example, working with managers in partners organizations to sensitize them to the advantages of social media in the workplace.
- These recommendations come from both this study and the previous one – when considered still relevant and (we think) not fully explored. Others are also derived from our experience in running social media media trainings in all types of organizations and with all types of trainees.In terms of demographics, it is worth investing the most responsive group and in the groups that so far have been less exposed to the Learning Opportunities.The creation of a Trainingof Trainers programme would ensure a larger multiplying effect for the reach of the capacity development intervention and the enlargement of the pull of trusted trainers that could be activated by CTA or partners organizations, to conduct further capacity building activities or provide direct support to organizations in ACP countries.More could be done to systematically collect, share and discuss success and failure stories of social media adoption in ACP countries, to foster peer learning.Additionally, there is a need to work more with managers in partners organizations to sensitize them to the advantages of social media in the workplace. The pilot with the East Africa Farmers Federation to support them in the creation of a social media strategy should be assessed.Finally, in terms of monitoring and evaluation, there is a need for more qualitative data and stories of change. Focus groups with former trainees could be organized for this.