Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Social Impact Evaluation Report

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

Social Impact Evaluation Report

  1. 1. 1 By One Global Economy Moustafa Mourad Alyssa Perez Colin Richardson and Martine Holston Final Report | October 2014 Digital Inclusion Social Impact Evaluation
  2. 2. Acknowledgements One Global Economy would like to express its gratitude to the Symantec Corporation for its continued support for OGE and the grant that has allowed us to undertake this evaluation in particular. For a number of years Symantec has supported OGE’s mission of serving low income communities in many countries including India and Mexico, and most recently, featured One Global Economy in its online brand campaign.
  3. 3. Table of Contents Acknowledgements............................................................... i Executive Summary...............................................................1 1. Background..................................................................... 3 1.1 Goal and Objectives of the Impact Evaluation............................ 3 1.2 The Information-Based Ecosystem Strategy.............................. 4 1.3 Internet Access Points................................................... 5 1.4 Training................................................................. 6 Digital Literacy................................................................. 6 Entrepreneurship and Technology....................................... 6 Community Connectors..................................................7 Internet Safety...........................................................7 1.5 Local Content.................................................................. 8 1.6 Country Programs............................................................. 9 South Africa................................................................... 9 Egypt.........................................................................10 India.......................................................................... 11 Mexico........................................................................ 12 2. Evaluation Design and Methodology.......................................... 13 2.1 Goal and Objectives of the Impact Evaluation........................... 13 2.2 Developing the Evaluation Design and Framework......................... 14 Theory of Change............................................................ 15 Results Chain Model......................................................... 17 Success Formula.............................................................. 18 Content for Impact: Skill Categories......................................... 19 Indicators.....................................................................20 2.3 Evaluation Design and Framework.......................................... 21 Program Monitoring......................................................... 21 Outcome Evaluation......................................................... 22 Data Collection Tools........................................................ 22 2.4 The Evaluation Toolkit........................................................23 Digital Inclusion Social Impact Evaluation
  4. 4. 3. Implementing the Evaluation....................................................25 3.1 Results Chain Model and Success Formula..................................26 3.2 End-User Survey.............................................................. 27 Sampling Strategy and Location............................................. 27 Data Collection............................................................... 27 Data Entry, Management and Analysis.....................................28 3.3 Routine Reporting............................................................29 4. Initial Evaluation Results.......................................................... 31 4.1 Google Analytics............................................................. 31 Total and Unique Views...................................................... 31 Average Number of Page Views and Average Time Spent on The Platform.................................. 31 Top Content..................................................................33 4.2 Survey Participants...........................................................35 4.3 Background Characteristics of Respondents................................35 4.4 Training and Access..........................................................38 4.5 Access Local Content........................................................ 41 4.6 Take Informed Action........................................................43 4.7 Increased Opportunities In Income.........................................46 Use Identity to Participate in the Local and Global Economy..............46 Start a Business...............................................................46 Career Advancement and Employment..................................... 47 4.8 Increased Opportunities in Health, Education, and Citizenship............50 Increased Opportunities in Health.......................................... 51 Increased Opportunities in Education....................................... 52 Increased Opportunities in Social Development............................53 4.9 Transform Communities.....................................................55 5. Conclusions and Lessons Learned............................................... 57 6. The Road Forward................................................................... 61 Appendix A: Data Collection Methods Along the Results Chain...................63 Appendix B: Impact Evaluation Toolkit................................................64 Appendix C: Top Five Articles by Country by Year...................................65 Evaluation Dictionary..................................................................... 67 Mission Definitions.................................................................68 References................................................................................. 71
  5. 5. 1 One Global Economy began its Impact Evaluation program a year ago after receiving a grant from the Symantec Corporation to evaluate existing programs in Egypt, India, Mexico, and South Africa, and to build its internal capacity to continue to monitor and evaluate future programs. Over the course of the program, One Global Economy (OGE) has learned a great deal about the processes of monitoring and evaluation, and about the people and the communities that OGE serves. The program began with OGE developing its theory of change, or how its programs improve the lives of the people it aims to serve. This helped clarify and determine the specific social impact metrics to be measured through the evaluation. The next step in the process was the design of a community survey to be administered in all four countries in Arabic, Hindi, Spanish, and English. OGE received 4,364 completed surveys, which provided a trove of data about people who had either attended training delivered by OGE or had accessed its online content on one of its digital platforms. Consequently, OGE was able to learn more about the people it serves and how they use and view the content and training. OGE focuses on employment as a solid indicator of joining the economic mainstream. Therefore, all of OGE’s trainings start with teaching basic digital literacy so that participants will be able to apply those skills to both the job search as well as increasing their employability. OGE’s findings about employment were very encouraging. 86% of survey respondents did not have a job before they participated in a training, but of those, 35% were employed by the time the evaluation process had begun. Half of the respondents, who reported they were employed, had found their job online. Google Analytics has allowed OGE to see which pages of its content are the most trafficked, how many pages people visit, how long users stay on a page, and other hard data. Through combining the Google analytics data with the data gathered from the community survey, OGE gained a much more detailed and nuanced understanding of the impact of its online content and how users adopt global best practices to improve their health, education, and social development. 35% of previously unemployed survey respondents were employed by the time the evaluation process had begun EXCUTIVE SUMMARY
  6. 6. 2 Digital Inclusion Social Impact Evaluation The survey data showed that 99% of respondents, who accessed content on OGE’s digital platforms, took some kind of action. The most popular content dovetailed with the feedback from the stakeholders’ engagement process that guided the content creation to develop the platform in each country. Employment and job-seeking skills were most popular in Egypt, health and sanitary practices in India, citizenship in South Africa, and job skills in Mexico. Another significant finding related to how people use OGE’s content and training revealed that 86% of respondents from Egypt and 60% of respondents from South Africa reported teaching other people in their communities about what they learned from the digital platform or in an OGE training. These results indicate that OGE’s content and training have a multiplicative effect on the communities it serves. The Impact Evaluation Program has bolstered OGE’s capacity to continually monitor its programs. OGE has worked with its partner organizations to help them develop performance indicators for programs delivered together. Additionally, OGE has created a monthly reporting tool and a training reporting tool that its partner organizations and local managers will use to provide regular feedback to OGE about the progress of its programs, express what needs improvement, and to share lessons they have learned. In developing an evaluation methodology and a host of tools, OGE has gained both a deeper understanding about the social impact its programs generate as well as developed a greater capacity in monitoring them. OGE has already begun applying lessons learned from this program. Firstly, OGE is implementing the regular reporting. Secondly, OGE will continue to deliver programs that proved successful. Lastly, OGE has identified content areas that it would like to develop and improve. OGE looks forward to implementing the evaluation methodology to measure the impact of future programs as well as refining programs based on the results of the evaluation. 99% of survey respondents who accessed content on OGE’s digital platforms, took some kind of action 86% of survey respondents from Egypt reported teaching other people in their communities about what they learned from the digital platform or in an OGE training
  7. 7. 3 1.1 Goal and Objectives of the Impact Evaluation One Global Economy (OGE) works with under-served communities in over 15 countries across five continents to create and implement information-based development strategies to help households and individuals raise their standard of living, join the economic mainstream, and innovate local solutions. OGE has a diverse portfolio of programs, each suited to the unique community it works in through a process of local stakeholder meetings and collaboration with the community and local non-governmental organizations (NGOs). These programs focus on access to broadband Internet connections, creating relevant digital content in local languages, and training people in areas of digital literacy, entrepreneurship, and techniques to gain the skills necessary to gain employment and upward mobility. Since its founding in 2005, OGE has launched 17 international community development websites. These digital platforms (Note: Words in bold orange font are defined in the Evaluation Dictionary) contain vital community development content on topics such as education, employment, entrepreneurship, and health written in local languages for readers at a 4th- grade reading level. The content housed on the digital platforms is composed of both global best practices and locally generated articles that focus on the needs of the community for which it is written. OGE’s digital platforms now reach more than 2.5 million unique visitors per year and since its founding in 2005, 8 million people have benefited from One Global Economy’s empowering community development information. In addition, OGE has trained thousands of people on digital literacy and assisted in the opening of Internet access points to enable people to have the ability to access the local content on the digital platforms. The purpose of this Impact Evaluation Program was to assess the long-term impact of OGE’s existing programs on mission-driven outcomes and to build OGE’s capacity to continually monitor and evaluate its programs going forward. This is particularly important because OGE’s programs serve very different communities, address a wide variety of issues, and do so across diverse levels of development and urbanization. BACKGROUND1 DEFINITION Digital Platform International community development websites that contain vital community development content on topics such as education, employment, entrepreneurship, and health written in local languages for readers at a 4th-grade reading level. The content housed on the digital platforms is composed of both global best practices and locally-generated articles that focus on the needs of the community for which it is written. DEFINITION Digital Literacy The ability to find, evaluate, utilize, share, and create content using information technologies and the Internet (Cornell University); the ability to perform tasks effectively in a digital environment. Literacy includes the ability to read and interpret media (text, sound, images), to reproduce data and images through digital manipulation, and to evaluate and apply new knowledge gained from digital environments (Jones-Kavalier and Flannigan).
  8. 8. 4 Digital Inclusion Social Impact Evaluation This impact evaluation focused on OGE’s programs in South Africa, Mexico, Egypt, and India to design and implement its methodology. There are common aspects of OGE’s work in all of those countries, but each country had different programs, content, and training curricula. The framework and tools created as part of this evaluation are meant to provide other organizations working in digital literacy and inclusion with tools to adapt to measure their impact and improve programming. 1.2 The Information-based Ecosystem Strategy OGE’s digital inclusion model addresses three barriers to participation in the information economy: lack of access to internet services, lack of access to actionable local content in local languages, and lack of digital literacy skills through what it calls the Information-based Ecosystem Strategy (see Figure 1). LACK OF ACCESS TO THE INTERNET In the information age, access to the Internet is vital to participate in the global economy. Without access to the Internet, the information available online will not reach underserved communities. OGE partners with community technology centers around the world that serve rural and low-income communities, and offer access to the Internet for free, or at a low cost that is considered affordable by the local residents. Through its partnerships with these local organizations, OGE creates a network of access points, where the digital platform is hosted in each country. LACK OF ACTIONABLE CONTENT IN LOCAL LANGUAGES There is a disparity between global language populations and content available online in those language. For example, in 2012, “Arabic content on the web represented just 3 percent of the total digital content online, yet Arabic speakers make up more than 5 percent of the global population” (Franceschi Bicchierai 2012). Online content in local languages that are not as widely spoken such as Kinyarwanda and Swahili account for even less, at below 1 percent. Consequently, digital content focused on developmental themes would account for even less, making it challenging for individuals in developing countries to access information in their language that is aimed at helping them to improve their standard of living. With more people accessing the Internet in developing countries through their mobile devices and local telecenters, there is increased potential for online information to contribute to the improvement of development outcomes in the areas of income, health, and social development.
  9. 9. 5 Beehive content on the digital platform is designed to address community development priorities identified by local stakeholders through a highly participatory engagement and consensus building approach. The content combines global best practices with highly localized information, written in the local language for readers with low literacy levels. LACK OF DIGITAL LITERACY SKILLS Without the skills necessary to access information on the Internet, people will not utilize access points and cannot find the actionable content available in their language. OGE provides training to partner computer center managers on ways they can support digital literacy in their community, launch demand- driven programs and services that address local needs, and teach users how to navigate the Beehive as well as access other existing online resources that provide global best practice information. OGE’s three-pronged strategy has successfully helped to create an ecosystem that has enabled millions in underserved communities to access high quality, actionable content on developmental themes. 1.3 Internet Access Points OGE works with in-country partner organizations to utilize their existing Internet access points as places to deliver trainings, hold stakeholder engagement meetings, and teach the community about its digital platforms. OGE helps to build the capacity of the local access point managers by drawing on its experience and best practices to help the centers to be more organized and to run more smoothly. Figure 1. Information-based Ecosystem for Digital Inclusion
  10. 10. 6 Digital Inclusion Social Impact Evaluation 1.4 Training DIGITAL LITERACY Digital literacy is critical to each person’s success in the 21st century and to maximizing the benefit of OGE’s programs, which is why it is the bedrock of OGE’s training programs. “Digital literacy is the ability to find, evaluate, utilize, share, and create content using information technologies and the Internet” (Cornell University). It is the ability to perform tasks effectively in a digital environment. “Literacy includes the ability to read and interpret media (text, sound, images), to reproduce data and images through digital manipulation, and to evaluate and apply new knowledge gained from digital environments” (Jones-Kavalier and Flannigan). Email, social networking, word processing, and using the Internet are necessary skills for anyone seeking gainful employment, a better job, or to expand his or her business. OGE teaches these and more advanced skills in many workshops. Using email and being able to navigate the internet open up a huge amount of opportunities to save money, take advantage of government programs, educate oneself, and make the kind of informed decisions that allow people to improve their own lives. OGE firmly believes that access, content and training are complementary components of digital literacy, in that they work together to improve and expand livelihood opportunities. Without training on how to use technology properly, and training on how it can be incorporated into daily life, and applied to finding information and solutions for development challenges, access would not play such a transformative role. Similarly, access does not benefit underserved populations if there does not exist content that meets them where they are in terms of their language, education level, and local context. This is why all of OGE’s training programs include digital literacy. ENTREPRENEURSHIP AND TECHNOLOGY Entrepreneurship is fundamental to growing today’s developing economies and technology and globalization allow for unprecedented entrepreneurial opportunities. OGE focuses on equipping emerging entrepreneurs with the digital and mobile skills necessary to enter the formal business world and confront everyday business challenges. Empowering entrepreneurs with core soft and hard skills builds their potential to start and grow their enterprises, and contributes to small business creation at the community-level. OGE’s Information-based Ecosystem Strategy supports and strengthens its entrepreneurship-focused programming by leveraging access, content, and training to build the capacity of aspiring entrepreneurs. The entrepreneurial training teaches basic technology skills to encourage technology skills application as a valuable tool for small business growth. OGE guides the development of essential business planning materials through providing templates and instructions for creating a business plan, budget, and marketing plan amongst others. OGE also provides online tools, localized information on entrepreneurship,
  11. 11. 7 and mentoring support though mobile technology to entrepreneurs to help them grow their existing businesses. Further, OGE helps small businesses access and use banking services by teaching basic financial literacy to help small business owners take out loans and to use government programs and financial institutions that will help keep their businesses stable and profitable. One such training program is OGE’s Mobile Entrepreneur’s Program (MEP), which emphasizes the use of digital and mobile tools to grow and improve small businesses. Workshops teach and monitor the achievement of a series of online skills such as establishing an online presence for a business, incorporating the use of social media into marketing plans, creating a LinkedIn profile for online professional networking, monitoring business bank accounts online and through SMS alerts, and conducting online research to increase business knowledge and to access government or private funding, or additional support through local organizations and services. Consists of modules focused on similar career development themes and skills, but with a focus on marketing individuals’ entrepreneurial skill sets to obtain jobs in a related field. COMMUNITY CONNECTORS The Community Connectors curriculum in digital literacy and leadership is focused on teaching young people how to apply digital and mobile tools to become actively involved in community development initiatives, leaders in their communities and technology ambassadors for their peers and families. The program teaches a combination of information and communications technology (ICT) skills and soft skills (planning and management, time management, project management, public speaking, self-confidence, etc). The program varies in each country and consists of additional modules addressing entrepreneurship, career development, Internet safety etc., and can focus on helping youth improve academic performance or find jobs through technology skills application depending on the target population (See Table A, OGE Training Programs). Community Connectors graduates have applied skills learned to lead workshops at their community centers to teach community members technology skills, conduct research on community and economic development topics to map out projects their community may implement, develop local businesses, and identify income-generating opportunities. The program fosters youth empowerment and skills-building for personal, professional, and social development. INTERNET SAFETY Internet safety has been incorporated as a focus in training in Mexico and India, where OGE has a partnership with Symantec, and has carried out projects promoting the involvement of locally-based Symantec employee volunteers. Local employees and engineers have worked with OGE and its partners to develop curricula for workshops and Internet safety campaigns focused on teaching first-time Internet users how to protect their identities online, how to protect their computers from viruses, and how to identify common threats such as information theft and fraud. Symantec has also contributed content on online safety, which is published on each digital platform. DEFINITION Mobile Entrepreneur’s Program (MEP) The Mobile Entrepreneurs Program (MEP) emphasizes the use of digital and mobile tools to grow and improve small businesses. Workshops teach and monitor the achievement of a series of online skills such as establishing an online presence for a business, incorporating the use of social media into marketing plans, creating a LinkedIn profile for online professional networking, monitoring business bank accounts online and through SMS alerts, and conducting online research to increase business knowledge and to access government or private funding, or additional support through local organizations and services. Consists of modules focused on similar career development themes and skills, but with a focus on marketing individuals’ entrepreneurial skill sets to obtain jobs in a related field.
  12. 12. 8 Digital Inclusion Social Impact Evaluation Community Connectors Community Technology Center Management Mobile Entrepreneurs Program Internet Safety South Africa Digital Literacy, Leadership, Community Development Digital Literacy & Business Management for Community Computer Centers Digital & Mobile Literacy for Entrepreneurship, Business Planning, Marketing N/A Mexico Digital Literacy, Leadership, Entrepreneurship, Internet Safety, Community Development Digital Literacy & Business Management for Community Computer Centers N/A Internet & Personal ID Safety India Digital Literacy, Leadership, Internet Safety, Job-Searching & Career Development/ Professional Skills Digital Literacy & Business Management for Community Computer Centers N/A Internet & Personal ID Safety Egypt Digital Literacy, Leadership, Entrepreneurship, Business, Job-Searching & Career Development/Professional Skills N/A N/A N/A 1.5 Local Content One Global Economy develops digital platforms written in local languages that are accessible to people with a 4th grade reading level. OGE’s community development-focused content is a combination of global best practices and highly localized information. Best practice information covers topics that are relevant all over the world, such as education, employment, health, and money, and focus on teaching people specific best practices such as how to write a business plan, how to select a school for their child, and basic prenatal care. Highly localized content complements these global best practices by providing information on how and where to access local resources, services, and government programs that benefit underserved populations and help them to take action to improve their lives in these areas. Content is produced by local editors, NGOs , and student journalists. The process begins with a stakeholders engagement workshop, where a roadmap is created to guide the development of the digital platform, and then the content is created over a period of 4-6 months. Each digital platform is launched as a public website once there is content about all community development topics identified in the workshop. Digital platforms are launched publicly with at least 200 highly localized articles. New content is added on specific topics as OGE develops more partnerships with local NGOs. OGE expands upon content on specific focus areas to complement its training programs. The digital platform is hosted, maintained and checked for quality of content by OGE. Table A. OGE Training Programs
  13. 13. 9 1.6 Country Programs SOUTH AFRICA OGE has worked in South Africa since 2007 when it established the Durban Centre, a community technology center hosted at the Austerville Public Library. The Durban Centre serves residents of South Durban and the Austerville community, and offers free access to the Internet and the South Africa digital platform, known as a “Beehive”, as well as hosts workshops in digital literacy for youth and adults. In (2008, OGE expanded its work in South Africa through a 5-year commitment to the Clinton Global Initiative (CGI) with partners, Cisco, Siyafunda, and Appleseeds Academy. Through the CGI, the partners established a network of Community Knowledge Centers (CKCs) across Sub-Saharan Africa (in Kenya, South Africa, Rwanda, Ethiopia, Nigeria, and Uganda). In South Africa, the partners trained Siyafunda community telecentre managers in digital literacy and business management skills essential to running sustainable computer centers that effectively serve their surrounding communities by building upon their basic offerings to offer demand-driven programs. The partners also developed and launched Community Connectors, a youth technology, leadership, and entrepreneurship training program and trained partner centers in the implementation and delivery of the program to local youth. In addition to the South Africa digital platform, OGE launched portals localized to Durban, Johannesburg, and Cape Town. Over the last five years, OGE has continued to work in partnership with Siyafunda to deliver its Mobile Entrepreneurs Program, which has supported over 1,500 South African street vendors, aspiring entrepreneurs, and small business owners through training and skills building in technology and entrepreneurship skills. The training is complemented with text messages to reinforce learning and disseminate helpful information, tips, and advice for growing and improving enterprises. PROGRAM SNAPSHOT South Africa Year Started: 2007 Key Activities: Establishing a network of Community Knowledge Centers (CKCs), Community Connectors Program, Mobile Entrepreneurs Program, and Beehives for South Africa, Johannesburg, Cape Town, and Durban People Trained: Over 2,500 Access Points Opened: Over 80 Focus Areas: Entrepreneurship and youth leadership DEFINITION Beehive The name of OGE’s digital platform in South Africa, Egypt, and India.
  14. 14. 10 Digital Inclusion Social Impact Evaluation EGYPT OGE’s work in Egypt has a particular emphasis on employment and entrepreneurship, where it has been working with local partner, CEOSS (Coptic Evangelical Organization for Social Services) and HANDS (Hands Along the Nile), based in Alexandria, Virginia, since 2011 to connect young people to livelihood opportunities through Asset Mapping, access to global best practices and highly localized content on the Egypt digital platform (known as the Beehive), and training in digital literacy, entrepreneurship, and career development skills. Through its Employment Through Information Technology program, OGE has mapped the skills of approximately 23,000 young men and women along with the outsourcing and employment opportunities of over 5,000 employers to support opportunity identification and job matching through online and mobile platforms. OGE and CEOSS have trained over 1,200 young people in marginalized communities throughout Cairo, Minya, and Beni-Suef at 10 local computer centers established through the program. OGE launched the Egypt Beehive in addition to two localized portals created for Cairo and Minya, which feature global best practice information and community development-focused content in Arabic. The Beehives have served over a million individuals since their launch in early 2013. PROGRAM SNAPSHOT Egypt Year Started: 2011 Key Activities: Asset Mapping, Online & Mobile Job-Matching System, Egypt, Cairo & Minya Beehives, Community Connectors People Trained: Over 1,200 Access Points Opened: 10 (in Cairo, Minya & Beni-Suef) Focus Areas: Employment and Entrepreneurship
  15. 15. 11 PROGRAM SNAPSHOT India Year Started: 2010 Key Activities: Beehives for India, Mumbai, and Pune in English, Hindi, and Marathi; working with Pratham Infotech Foundation; training programs; education programs People Trained: 765 Access Points Opened: 10 Focus Areas: Jobs, health, digital literacy INDIA OGE’s India Digital Inclusion Program focused on integrating the core components of local internet access, localized online content, and digital literacy and professional skills training into new programming and services at 10 computer centers run by OGE’s local partner, Pratham InfoTech Foundation (PIF) in Maharashtra. OGE and (PIF) trained over 4,000 people in digital literacy, technology, leadership, entrepreneurship, and community service, as well as working with local health professionals to educate 488 people—most of whom are young women—about health and sanitation. The India Digital Inclusion Program has a particular emphasis on employment. The program offers coaching services to help participants begin their employment search, practice interviewing, job placement, and inclusion in an online employment database. Many participants have learned how to create and/or access email accounts and to use them for job-seeking and professional purposes. The third aspect of this program is OGE’s digital platform known as the Beehive ( Viewed by over 200,000 people in the last year, the Indian Beehive is composed of articles about financial literacy, employment, and health written in English, Hindi, and Marathi for an audience with a 4th- grade reading level.
  16. 16. 12 Digital Inclusion Social Impact Evaluation MEXICO OGE launched the Mexico digital platform, InfoFá, in 2010 and the Toluca digital platform (Toluca.InfoFá in 2013. OGE has been working with University partner, Universidad Autónoma del Estado de México (UAEM) in Toluca and its journalism department to engage students in the development of highly localized content for the two platforms by providing them with small stipends, community service and internship credit in addition to incorporating content creation into curricula, and offering the Mexico and Toluca digital platforms as an online publishing outlet. Through engaging student journalists, over 500 articles were produced for the Mexico platforms over a period of three years/ to date. Since their launch, more than 3.3 million unique visitors are accessing the digital platforms in Mexico, with over 300,000 accessing it via their mobile phones. OGE also launched its Community Connectors program in three rural villages in Guanajuato (El Garbanzo, San Agustin, and Victoria), where it works with local partner, Choice Humanitarian, and University partner, Universidad Technológica del Norte del Estado de Guanajuato, to support the development of sustainable community technology centers. With the past year, OGE also implemented the youth program at three semi-urban high schools in Toluca, in partnership with UAEM student trainers, training a total of approximately 150 youth in Mexico. In 2013, OGE delivered educational workshops in Internet safety for first-time Internet users to over 500 Toluca residents together with student trainers and locally based volunteer Symantec engineers. PROGRAM SNAPSHOT Mexico Year Started: 2010 Key Activities: Beehives for Mexico & Toluca, Student Journalist Beehive Editors Program, Community Technology Center Management Training, Community Connectors, Internet Security Campaign People Trained: Over 200 Access Points Opened: 4 Focus Areas: Health, Employment, Education, Internet Safety, Youth Leadership
  17. 17. 13 There is a heightened awareness among the science and technology community regarding the social impact potential of connectivity and economic inclusion through digital and mobile tools. Academics and think tanks report on the positive effects of access to the Internet, yet there is a need for real-life analysis of the effects digital inclusion has on improving the lives of those in poverty. OGE developed this Impact Evaluation to address this need and to quantify its social impact potential and results, gather data that indicates lasting social value of its digital inclusion programs, and track progress towards its defined social impact targets. In this Impact Evaluation, OGE sought to provide a guide for a rigorous analysis of the “lift,” or margin of improvement, of its digital inclusion services upon low-income populations in developing countries who are new to the Internet and lack skills necessary to utilize the Internet to improve their lives. This evaluation examines the lift in terms of the economic and human development effects on individuals who participate in the digital online platform as users, in the Community Connectors digital literacy training programs, and the patrons of OGE’s partner Community Knowledge Centers. 2.1 Goal and Objectives of the Impact Evaluation Through this Impact Evaluation, OGE sought to track the impact of the digital inclusion program on participants’ economic opportunities and quality of life, as expressed in terms of human development indicators: access to opportunities that increase their income, education, and improve their health. In developing a methodology to assess the impact of OGE’s training and programs, this Impact Evaluation aimed to create a standardized way of evaluating impact in digital literacy programs for the benefit of other organizations doing similar work. Digital inclusion programs pose unique challenges in designing and implementing an impact evaluation. By nature, these programs are not place-based interventions; while the topics are localized geographically the content can be accessed globally. Unlike traditional development work, OGE’s programs are not defined in geographic terms and the organization is not focused on working in specific communities for extended periods of time. Therefore, this evaluation required the development EVALUATION DESIGN & METHODOLOGY2 OGE’s Mission One Global Economy works with under-served communities to create and implement information-based development strategies to help households and individuals raise their standard of living, join the economic mainstream, and innovate local solutions
  18. 18. 14 Digital Inclusion Social Impact Evaluation of new methods and tools to assess the impact of OGE’s programs and of digital inclusion programs in general. Thus, one of the main goals of this evaluation was to create a toolkit of adaptable tools and frameworks for other organizations to implement evaluations in this expanding niche. This organizational capacity-building project also served to strengthen focus and codify an outcomes-based approach for implementation across all OGE country programs in the future. THE OBJECTIVES OF THE IMPACT EVALUATION WERE TO: • Articulate a theory of change of how OGE’s training and programs lead to mission-driven outcomes • Develop key qualitative and quantitative performance and impact indicators • Create a comprehensive strategy to gather data for the key indicators, including routine programmatic reporting and follow-up with participants – Implement a follow-up survey in partner computer centers and online in four actively-funded countries (Egypt, India, Mexico and South Africa) – Develop internal tools for more efficient communications regarding progress toward impact metrics • Create a toolkit with adaptable templates that provides a comprehensive guide to implementing an impact evaluation for digital inclusion • Analyze the process and outcome of this initial impact evaluation with attention to improving programming, outcomes, and refining the evaluation methodology • Build internal capacity to monitor and evaluate the impact of Digital Inclusion programs for future projects • Share the findings, results, and conclusions publicly 2.2 Developing the Evaluation Design & Framework The process of developing the evaluation design and framework included: • Creating a theory of change to articulate how OGE achieves its impact. This causal logic of how OGE’s programs deliver results is the underpinning of the impact evaluation and was used to create its framework, indicators, and data collection methods.
  19. 19. 15 • Adapting two program planning tools, the Results Chain Model and the Success Formula, to further map out the inputs, strategies, and outputs that lead to OGE’s hypothesized impact. These two exercises identified the process that leads to impact as well as what the impact looks like. • Creating a comprehensive list of indicators using these two tools to ensure all aspects of programs were measured as part of the evaluation. Each step of the Results Chain was included, as well as a clear definition of all the skill and content categories identified through the Success Formula process. The comprehensive list of indicators was distilled into a list of key indicators, tested in this evaluation and to be used as a menu of programmatic indicators for future planning. • Using the list of indicators to develop data collection tools and systems to measure each of the key indicators identified in this process. THEORY OF CHANGE OGE began the impact evaluation with the question: What effect does digital literacy and access to the Internet have on social and economic development of individuals and communities? To answer this question, the first step of the evaluation was to hypothesize how OGE achieves its impact – the organization’s theory of change. The evaluation team deconstructed the organization’s mission by creating internal definitions of each piece of the mission (see Evaluation Dictionary: Mission Definitions) to articulate the organization’s theory of change: Strengthening access to digital and mobile tools by partnering with local computer centers, training computer center managers and patrons in digital literacy and business skills and providing access to locally-written, actionable content on developmental themes in their own languages will enable households and individuals to take informed action that will transform their lives and their communities. Figure 2. Evaluation Methodology Development
  20. 20. 16 Digital Inclusion Social Impact Evaluation More specifically, the OGE training curricula combined with both locally generated content and global best practices in key issue areas provides users with the skills and information they need to take significant steps towards participating in both local and global economies. Through digital inclusion, users and their communities can raise their standard of living and join the economic mainstream by increasing their economic, health, and educational opportunities, such as starting a small business, improving their livelihood prospects, improving their ability to make informed health, education, and business decisions; and increasing their participation in the local community. A key element of OGE’s Theory of Change is that the individuals in the communities OGE serves must take action to complete the cycle of change. While OGE fosters the information ecosystem with local partners and creates and shares opportunities for people to take action to improve their lives, OGE believes firmly that the sustainable solution is one in which local participants are the key actors, leading local progress. Figure 3. One Global Economy’s Theory of Change
  21. 21. 17 RESULTS CHAIN MODEL The evaluation team then applied the theory of change to two program planning tools to further articulate how the organization achieves its impact. Through clearly defining and articulating the mechanisms and processes that lead to mission-driven outcomes, the team sought to create a comprehensive evaluation model that encompasses all elements of OGE’s programming to best assess and estimate the “lift” and a set of program planning tools adapted to OGE’s theory of change that could be used for future programs. Thus, these tools were used both to help guide the evaluation by looking retrospectively at OGE’s program models and as an exercise to develop planning tools that would be used in all future programs and adapted for other organizations and included in the Evaluation Toolkit. First the team used the World Bank’s results chain model found in the Impact Evaluation in Practice. A results chain shows cause and effect: how the sequence of program inputs (e.g. staff and other resources) and activities lead to specific program outputs, and how these outputs are directly related to accomplishment of program goals (i.e. impact). This model shows how specific program activities directly lead to the desired program goal and what must be done in order to achieve programmatic targets and results. It also documents what goes into a program (the inputs), which is integral for replicability and scalability of programs. The evaluation team created an overall results chain model for the organization as well as adaptations for country-level programs. We used this structure to succinctly visualize and describe each of OGE’s programmatic activities, including their anticipated proximate and distal impact. The results chain demonstrates the cause and effect relationship of OGE’s programming. Moving forward, OGE will require program staff at the country level to create a Results Chain Model for their program so that they can understand how programmatic activity results in the desired program goal and what must be done in order to achieve programmatic targets and results. By creating a model that showcases how the program’s theory of change is directly tied to the program’s specific focus and goals, stakeholders create a common vision for the program, which serves as a roadmap for both implementing and evaluating the program. DEFINITION Results Chain Model Template form of the results chain for country teams to fill in at the beginning of a program, a practice for them to see how programmatic activities are directly tied to longer- term, mission-driven outcomes.
  22. 22. 18 Digital Inclusion Social Impact Evaluation Note: The full version of the Digital Inclusion Results Chain includes a section for listing indicators along each step of the chain. Country teams list the inputs, activities, outputs, outcomes, and impact for their programs and then create a list of indicators to measure each. SUCCESS FORMULA OGE also adapted the Success Formula framework from the firm Mission Measurement and its CEO Jason Saul’s book Benchmarking for Nonprofits as a second program planning tool for this evaluation. The Digital Inclusion Success Formula is a universal tool that enables One Global Economy and other digital inclusion nonprofits to consider its social impact as a simple equation. At the end of the formula is the ultimate impact the organization aims to achieve through programming, created through the sum of the intermediate steps at a strategic, organizational level. The Success Formula makes OGE’s Theory of Change actionable and measurable. At the top of the Success Formula is the underlying principle that acquiring digital literacy and key skills combined with access to both locally generated content and global best practices in key issue areas provides users with the skills and information they need to take significant steps towards participating in both local and global economies. The Success Formula is used to outline the strategy and activities that will be used in the key issue areas (income, health, education, and social development) of the program, and the performance indicators that will be used to measure progress and success. Figure 4. OGE Digital Inclusion Results Chain Adapted from Gertler, Martinez et al. Impact Evaluation in Practice. The World Bank 2011.
  23. 23. 19 CONTENT FOR IMPACT: SKILL CATEGORIES To further refine the model, we brainstormed and defined the main content areas of OGE’s programming, which together we believe are critical to achieving OGE’s desired impact. We called these groupings of content “Skill Categories” because the goal of the content in digital inclusion programming is building capacity and informed action. Part of OGE’s theory of change is that access to and taking informed action on these specific content areas are what create the desired impact as described in OGE’s mission statement. The four primary Skill Categories we identified are: digital literacy, business and entrepreneurship, livelihood and employment, and personal/social/ community development. Of note, Digital Literacy is a fundamental skill, because it creates the ability to access content in the other Skill Categories. The Skill Categories were further broken down into sub-skills, again using the criteria that these skills are imperative to achieving the desired results and are a fundamental part of OGE’s training curricula and programmatic activities. The skills and sub-skills identified in this exercise were fundamental to creating indicators and creating training assessments as part of the routine reporting component of the Impact Evaluation. Figure 5. OGE Digital Inclusion Success Formula Note: The full version of the Digital Inclusion Success Formula includes sections for listing program strategies and indicators for income, health, education, and social development. Country teams list the strategies they will use in their programs to create change in these categories, and then create a list of indicators to measure each. DEFINITION Digital Literacy The ability to find, evaluate, utilize, share, and create content using information technologies and the Internet (Cornell University); the ability to perform tasks effectively in a digital environment. Literacy includes the ability to read and interpret media (text, sound, images), to reproduce data and images through digital manipulation, and to evaluate and apply new knowledge gained from digital environments (Jones-Kavalier and Flannigan).
  24. 24. 20 Digital Inclusion Social Impact Evaluation INDICATORS The indicator table serves as the backbone of the evaluation. The evaluation team developed indicators, or means to measure the comprehensive model of the theory of change. Indicators were created for each step in the Results Chain, and within each step for the skill categories. Creating the indicators in this way ensured that we measured all aspects of the process of delivering programs and the components that are imperative to creating impact. The indicators for inputs, activities, and outputs are process indicators, which measure what is put into a program and the deliverables of programmatic activities. The indicators for outcomes and impact are outcome indicators, which measure the lasting impact of the programs on the participants. To create outcome indicators tied to OGE’s mission of raising the standard of living, join the economic mainstream, and innovate local solutions, we used the Human Development Index, published by the United Nations Development Program. This Index uses composite indices to measure health (life expectancy), education (mean years of education), and income (gross national income, GNI, per capita) to rank countries in terms of development. While the indicators themselves cannot be used at the individual and household level, we used these broad categories to define standard of living for the purposes of this impact evaluation. Figure 6. Skill categories and sub-skills DEFINITION Process Indicator Indicators to measure whether planned activities took place, add more details in relation to the product (“output”) of the activity, and/or monitor the quality of the activities conducted, based on a number of established quality criteria or standards. (WHO) DEFINITION Outcome indicator Indicators that measure the objectives of an intervention, that is its ‘results’, its outcome. They are the result of both the “quantity” (“how many”) and quality (“how well”) of the activities implemented. See full definition in the Evaluation Dictionary
  25. 25. 21 OGE therefore defines standard of living as the income, education, and health opportunities accessed by a given individual or household. It includes the following: • INCOME: Changes in annual income and savings, employment and career growth, entrepreneurship. • EDUCATION: Pursue or attain GED, applied to or enrolled in university. • HEALTH: Accessing quality health care systems, disease management, access preventative medicine (prenatal care, sanitation and water access, family planning, immunizations, infant care, nutrition) By using OGE’s mission statement to identify and define the final outcomes in the results chain, we aimed to ensure that this impact evaluation measures OGE’s results against its mission, reinforcing the mission at all levels and integrating the mission in its evaluation of itself. All indicators were assessed to be specific, measurable, attributable, realistic, and targeted. 2.3 Evaluation Design and Framework The impact evaluation applied a conventional social science approach using a non-experimental design. We collected information on the impact of OGE’s programs and activities on the desired outcomes through standardized routine program monitoring and a follow-up survey with training participants and digital platform users. The nature of OGE’s programs renders an experimental (randomized) or quasi-experimental (comparison group) approach infeasible. Therefore, the evaluation compared participants’ utilization of information technology and key socio-economic indicators before and after training. The evaluation used several methods to measure changes in the skill acquisition, utilization, and impact of information technology and resources from OGE’s programs. The evaluation methods bolstered routine reporting to monitor programs and assess input, process, and immediate outcomes of programs and initiated a survey to evaluate outcomes. PROGRAM MONITORING By measuring the resources, activities, and program achievements, organizations can track the causal logic of program outcomes, determine if interventions were carried out as planned, if the interventions reached the targeted beneficiaries, and if the interventions were performed on schedule, and what the results from the interventions were for the beneficiaries.
  26. 26. 22 Digital Inclusion Social Impact Evaluation As part of the evaluation, OGE developed three components to program monitoring: • A STANDARDIZED MONTHLY REPORTING FORM and system to collect program data regularly, including data on the resources used (inputs), activities completed, and direct program outputs. Field staff and partners complete this form on a monthly basis to record progress towards key program objectives, which will directly feed into the impact evaluation. • A STANDARDIZED TRAINING REPORT FORM, which includes the content of the training, what curricula were used and how they were adapted, and the number of people trained. In addition, post-training assessments were created to ascertain the level of proficiency in the key skills taught during the training. • A TRAFFIC MONITORING TOOL to track OGE’s international digital platforms using Google Analytics. This monitoring tool allows OGE to gather significant data about the people who are using the digital platforms. OUTCOME EVALUATION To measure the impact of digital inclusion programs, it is necessary to survey participants and measure behavior and socio-economic changes of those who participated in trainings and/or accessed information via the digital platform. • IMPACT SURVEY: A standardized survey given both online and through active follow-up with program participants after they have been trained in one or more of OGE’s programs (e.g. Digital Inclusion, Community Connectors, Entrepreneurs, etc). The purpose of this survey is to find out the longer-term impact of the training on participants’ lives and socio- economic condition. The survey was conducted in four countries where OGE has conducted trainings—specifically, in India, Mexico, Egypt, and South Africa—for this initial evaluation. DATA COLLECTION TOOLS As part of creating the indicator table, we identified when and how the data would be collected and from whom. This process identified the different data collection tools that were necessary and when they should be implemented. To develop the data collection tools, we used the indicators to create questions. We systematically went through the indicator table to ensure that all data required to measure the indicators will be collected. A summary of data collection along the results chain is presented in Figure 7. By measuring indicators along the results chain, the impact evaluation methodology prevents producing a ‘black box’ that only identifies whether or not the predicted results occurred without explaining why or how the results were achieved. See Appendix A for a detailed description of data collection methods across the results chain.
  27. 27. 23 As we refined the data collection tools, we refined and updated the indicator table. The survey questionnaire went through several rounds of reviews with the team to streamline, simplify and ensure that high-quality data would be collected that would measure the desired outcomes of the evaluation and then was sent to local partners for review and feedback. 2.4 The Evaluation Toolkit One of the goals of this Impact Evaluation was to create a methodology to enable the organization and its partners to better measure the social and economic impact of its Digital Inclusion Programs so that the lessons can be used to refine and scale future programs. Therefore, as part of the Impact Evaluation, OGE developed an Evaluation Toolkit to codify the process, tools, and implementation of a digital inclusion impact evaluation to be used in the future for all of OGE’s digital inclusion programming and for other organizations doing similar programming to adapt and use it to measure their own impact. This toolkit is a resource to help the management team, programmatic staff, and partners succeed at enabling digital inclusion toward developing their own communities. The purpose of this toolkit is to enable better digital inclusion practitioners to understand, define, measure and improve their impact. In the Impact Toolkit, OGE offers several guides for localizing the methodology, including a Success Indicators Planner that breaks down OGE’s theory of change and definitions of digital inclusion success. This will help to cultivate local critical thinking, ownership, and leadership of digital inclusion progress and measurement on the ground. Throughout the process of implementing this initial impact evaluation, we incorporated lessons learned into the toolkit and refined the framework, data collection tools, and systems. Figure 7. Data collection summary along the results chain
  28. 28. 24 Digital Inclusion Social Impact Evaluation This Digital Inclusion Impact Evaluation Toolkit features capacity building guides, templates, and exercises for One Global Economy (OGE), its international partners, and other Digital Inclusion organizations to better define, plan and measure against its proposed outcomes (see Figure 8). OGE is sharing this universal toolkit with templates designed to be customized for each country/program with the collaboration of local partners, and enables partners to adapt and help execute the evaluation in their country. For a detailed list of the contents of the Impact Evaluation Toolkit, see Appendix B. Figure 8. The Digital Inclusion Impact Evaluation Toolkit
  29. 29. 25 The evaluation team included OGE programmatic and leadership staff in collaboration with an evaluation consultant who provided technical guidance and assistance. The team began planning for the evaluation in the fall of 2013. The Impact Evaluation methodology, including the Theory of Change, Results Chain Model, Success Framework, indicators, and data collection methods were designed and created over the course of five months, from October 2013 to February 2014. Once the indicators were created and the methodology outlined, the team began designing an end-user survey and internal monitoring tools in early 2014 and implementing them in the summer. The analysis of all data and writing of the final report took place between August and September 2014 (see Figure 9). The purpose of this impact evaluation was to be formative for continued evaluation of impact. As such, this evaluation was carried out in only four countries to test and improve the methodology. Moving forward, OGE will continue to implement the routine reporting created as part of this evaluation (monthly, training, and traffic reports) in all of its current and future country programs and will implement subsequent surveys to continue to assess the impact of its programs. OGE chose India, South Africa, Mexico, and Egypt as the countries in which to conduct this formative Impact Evaluation for a number of reasons. Firstly, OGE had ongoing programs in each of those countries, which meant it had on- the-ground partners who could help contact people who had taken trainings and accessed content. Secondly, these countries represent a diverse array of communities OGE serves, ranging from the densely urban areas of Mumbai to very rural villages in the Mexican countryside with many gradations in between. Thirdly, these four countries covered a wide array of different programs that OGE employs. IMPLEMENTING THE EVALUATION3
  30. 30. 26 Digital Inclusion Social Impact Evaluation 3.1 Results Chain Model and Success Formula The Results Chain Model and Success Formula are program planning tools, and served two functions in this evaluation. First, we used these tools to retrospectively outline the theory of change and strategies for achieving the desired outcomes of OGE’s programming overall. This was an essential step to help develop a comprehensive set of indicators that would drive the data collection methodology of the evaluation. Second, we field tested these tools with country teams to gain consensus and improve the tools so that they could be included in the Evaluation Toolkit as program planning tools – used in all future OGE programs and adapted by other organizations when planning similar programs. The feedback from country teams in South Africa and Mexico was that these were useful tools in looking at the program as a whole and better understanding the relationship between program outputs and impact. Managers in these countries were able to participate in a step-by-step process of diagramming how these programs are designed to help their beneficiaries. This clarified the strategies through which real change is affected and highlighted the importance of their programmatic activities. Of note the team in South Africa used an innovative approach to completing the Success Formula: they completed the formula from the perspective of one person who couldn’t find a job to figure out the best way to work with them. This approach was incorporated into the directions for completing Figure 9. Impact Evaluation Timeline 2013 2014
  31. 31. 27 the Success Formula and results chain, but from a more holistic perspective: completing the exercises several times over with different client avatars, each with different problems and needs, as a needs assessment, program planning, and evaluation tool. For example in Mexico, the team of student journalists reflected on how articles on various locally relevant topics contributed to community, personal, and social development, and how different formats for disseminating global best practice information provided access to more services and resources locally. Overall, a challenge with these tools was getting program teams to be as specific as possible about programmatic activity. Without guidance, local teams often wrote blanket statements that were not particularly meaningful or measurable. To help guide teams in the future, the evaluation team created a list of program activities and indicators for them to include, adapt, or expand upon. 3.2 End-User Survey SAMPLING STRATEGY AND LOCATION The end-user survey is the most significant surveying OGE has ever undertaken of the communities it serves. OGE posted links to the survey on the local online portals in each country to try to get online users to participate and actively followed-up with past program participants in communities to conduct the survey in-person. The survey was implemented in Mexico, Egypt, South Africa, and India. The target population was anyone who had either visited a local online portal or had attended a training course with an OGE curriculum. This is a very demographically unspecific set of people, but that was necessary given the diverse population that OGE serves. DATA COLLECTION A standardized survey questionnaire was uploaded to SurveyGizmo, a web- based survey program that allows for skip logic, multiple choice, and write-in responses. A link to the survey was posted to the digital platform in each of the four countries, with translations in Arabic, Hindi, Spanish, and English available. In addition, staff of OGE’s partners administered the questionnaire in-person with hard copies to members of their community. Data collection began on March 15 and continued through July 22, 2014. It is impossible to overstate how critical OGE’s in-country partners were to the success of this survey. OGE worked with Pratham InfoTech Foundation in India; the Coptic Evangelical Organization of Social Services (CEOSS) in Egypt;
  32. 32. 28 Digital Inclusion Social Impact Evaluation Siyafunda in South Africa; and Universidad Autónoma del Estado de México in Mexico. OGE works with these partners to implement its programs in each country. Not only do these NGOs have a deep reach into the communities where OGE works, but in many cases they also run computer or community centers where OGE could get people from the community who had participated in its programs to take the survey. The partners met with people in their communities, conducted outreach, and encouraged people to take the survey. DATA ENTRY, MANAGEMENT AND ANALYSIS Most individual respondents completed the survey on SurveyGizmo. The hard copies of the survey were directly entered into SurveyGizmo by members of OGE’s partner NGOs to ensure standardized data between online and in- person respondents. The analysis plan used the Success Formula and Results Chain as a guide to analyze the key performance indicators identified during the process of creating those tools. For demographic and direct program impact variables, data was compared between countries to see how the different programs affected outcomes. For longer-term impact variables, data was compared between those who did and did not participate in a training to see the long- term effects of the training on key impact variables, particularly the impact of training on taking informed action in key development areas. Furthermore, we analyzed the data by time since training to uncover the longer-term impact of the program and training. Overall, this analysis was conducted with an eye towards understanding the effect of content and training and how we can improve our programs to achieve greater impact. A criterion of p<0.05 was used to assess the statistical significance of differences between groups. Differences with p<0.01 were noted as highly significant.
  33. 33. 29 3.3 Routine Reporting A major goal of the Impact Evaluation was to build OGE’s capacity to monitor and evaluate its programs. Therefore, as part of the evaluation, a comprehensive routine reporting system was developed and implemented (see Section 3.2 above). OGE collaborated with its partners in each of the countries to create the routine reporting templates, including monthly reporting forms, a standardized training report form, and a traffic monitoring tool. The Monthly Routine Reporting Form and the Training Report Form together create a comprehensive internal monitoring system for OGE. The purpose of this system is to: • STANDARDIZE INFORMATION ACROSS COUNTRY PROGRAMS. The same information will be collected from all of OGE’s programs using this form. With standardized information, we can track programmatic activities and progress organization-wide. Regular updates on program activities ensure the history of a program is documented thoroughly. • ENABLE “REAL-TIME” REPORTING. Activities are updated monthly as they happen, rather than waiting until a report is due. If updates program milestones are given on an on-going basis, the burden of giving information for reports at the end of the period is reduced. Rather than going back and asking country teams what happened during the reporting period, staff can generate a report using the data that has been entered already. • CONSOLIDATED SOURCE OF PROGRAM INFORMATION. The Monthly Routine Reporting Form collects information on major activities (e.g. trainings or workshops, computer centers opened, outreach activities, and community activities), content developed for the digital platform, computer center activity (e.g. users and center development activities), follow-up with training participants, and stories of people innovating local solutions, quotes and testimonials, and other notes. The Training Report Form is part of a larger set of tools for designing and evaluating trainings, which include choosing curricula and teaching objectives and post-training evaluations of skills acquired. The Training Report Form reports on the content and objectives of the training, the attendees, and the results of post-training assessments. The reporting system utilizes a web-based survey design, Survey Gizmo, with skip logic: answers to initial questions direct the respondent to subsequent questions to complete information specific to that type of activity and allows them to skip questions that are irrelevant. DEFINITION Routine Programmatic Reporting Standardized monthly report form submitted by the country team to headquarters, including directions and a description of how this reporting fits into the larger impact evaluation and OGE’s mission. DEFINITION Training Report Forms Standardized training report form submitted by the country team within one month of training to headquarters, including directions and a description of how this reporting fits into the larger impact evaluation and OGE’s mission.
  34. 34. 30 Digital Inclusion Social Impact Evaluation By including the local partners in the development of the reporting system, OGE was able to integrate their feedback to make the system more accurate, relevant, and feasible. For example, the South Africa Manager provided helpful feedback for localizing the Monthly Reporting Form, to the Mobile Entrepreneurs Program specific to South Africa, by suggesting that the form integrate more skills in the checklists and that the form include a space for local managers to provide testimonials from participants, which are gathered regularly. Walking in-country managers and partners through their own success formula and results chain often helps them gain a better perspective on the work that they are doing. Google Analytics allows OGE to track the information of digital platform users. By tracking information using the Traffic Monitoring Tool, OGE can learn which pages are viewed the most and for the longest amount of time in different areas, where users come to the digital platforms from, and which topics resonate the most. This allows OGE to give partner organizations accurate feedback about articles written by community members, as well as pointing OGE towards the issues that are most pertinent to different communities. DEFINITION Traffic Monitoring Tool A form to collect and consolidate information from Google Analytics.
  35. 35. 31 4.1 Google Analytics TOTAL AND UNIQUE VIEWS At the time of this report, OGE has had 10,756,356 sessions and 9,386,745 unique views on its digital platforms since 2005. As seen in Figure 10, unique views have been steadily increasing yearly for OGE digital platforms overall. Unique views on the Mexico digital platform have remained consistent since July 2011 at an average of 450,000 unique views per year. The Egypt digital platform, which started in 2013, had over 570,000 unique views that year. AVERAGE NUMBER OF PAGE VIEWS AND AVERAGE TIME SPENT ON THE PLATFORM Two ways to measure online engagement are the number of pages viewed in a session and the amount of time spent on the website. OGE monitors these two metrics on all of its web properties using Google Analytics (Figure 11). INITIAL EVALUATION RESULTS Figure 10. Unique views per year by country and OGE overall 4
  36. 36. 32 Digital Inclusion Social Impact Evaluation In South Africa the number of pages visited per session has dropped a small amount from just over two pages per session to about 1.8 pages per session while the time spent on the website has increased from about 1:05 to 1:18. There has been a similar trend in Mexico where page views decreased from 1.8 per session to 1.4 while the time on the sites has risen dramatically from 1:25 to about 2 minutes. This signals a deeper engagement with the material. India has trended oppositely, where page views have gone up from an average of 2.85 to 3 and time spent on the web property has fallen from about 3:30 to about 2:55. Though the time on the site has decreased, almost three minutes spent on the site remains higher than the industry average of 10 to 20 seconds (Nielsen 2011) and the “average of a little over one minute on a newspaper website” (Grabowicz 2014). Figure 11. Average number of pages viewed and time spent per session on the digital platform
  37. 37. 33 TOP CONTENT As seen in Figure 12, health, education, and employment are consistently among most accessed content. Because each portal is localized to its respective region, programmatic efforts focus on relevant topics, and their significance to the target population is often highlighted by trends in users’ activity as seen in the diversity between countries in these results. For example, South Africa shows the most views for citizenship-related content compared to Mexico. In South Africa, the stakeholders’ engagement workshop revealedthatmanySouthAfricanresidentswerenotawareoftheirrightsascitizens under the then new post-Apartheid democracy. Through developing content with localNGOs,OGEdefinedcitizenshipasakeycommunitydevelopmentpriority, and featureditasthefifthmaintopic.ForEgyptandIndiathisfifthtopicisagriculture. SouthAfrica’scitizenship-focusedcontentoutlinescitizens’ basicrightsandexplains SouthAfrica’ssystemofgovernmentinalanguageuserscanunderstandin orderto access local services, initiate processes for obtaining licenses and certificates, and become informed and active citizens through participating in elections. In other cases, top content indicates hot issues and can inform article creation for the development of more specific content addressing those topics. For example, soon after the launch of the Mexico digital platform, a review of Google Analytics and comments from users on the platform showed that there was a demand for more employment-focused articles. OGE, its local team, and student editors at UAEM (partner University in Mexico) focused on developing articles aimed at young people and job seekers, increasing the number of online resources that help people understand how to apply for jobs online, learn interview skills, and link them directly to Mexico-based online job-search engines. The demand for livelihood-focused content addressing employment and education issues is also illustrated by the high percentage of content viewed on women and teenagers, which remains a top topic due to additional content OGE developed to guide high school dropouts, single mothers, and adults through choosing the best way to complete their education and earn degrees and certificates. The structure of the program and its components in each individual country can also explain why certain topics are more frequently trafficked over others. For example, in Egypt, the goal of the project is to link young people to employment opportunities. Content on the Egypt digital platform therefore largely focuses on connecting individuals to resources addressing issues of financial literacy, income generation, entrepreneurship, agribusiness, and best practices for microenterprise development. In India, a recent program focused on implementing health camps to teach basic sanitation practices, preventative healthcare, and adolescent health at the partner centers. Many of the health camps have focused on sexual health and serve young women, who lack access to such information. Top content viewed on health in general, and articles on safe sex, and date rape drug awareness are reflective of a need for adolescent health information. See Appendix C for the top five articles by year for each country and OGE overall.
  38. 38. 34 Digital Inclusion Social Impact Evaluation MOBILE VS. DESKTOP USERS The percentage of people accessing OGE’s digital platforms from mobile devices steadily increased since 2011. In South Africa between 2013 and 2014, 53% of people accessing the digital platform did so via mobile devices. Though the percent of mobile users are still low in Egypt (9%), India (22%), and Mexico (24%), they are not insubstantial. All four countries are showing substantial upward trends, with the percent of mobile users doubling in all four countries in the last year. This trend has serious consideration for future programs, which will be discussed later in this report. Figure 12. Top content topics by country by year Figure 13. Percent of all users accessing the digital platform with a mobile device by country by year
  39. 39. 35 4.2 Survey Participants OGE collected a total of 4,676 surveys between March 15 and July 22, 2014. Incomplete surveys (n=308) and surveys from respondents living in countries not included in the evaluation (n=8) were removed from the analysis. A total of 4,364 surveys were included in the analysis: 2,407 from Egypt; 1,292 from India; 491 from South Africa; and 174 from Mexico (see Figure 14). The number of respondents in total and in each country is sufficient for OGE’s purposes because the total is large enough to test for statistical significance and each country’s total reflects the number of participants in OGE programs. For instance, OGE works with CEOSS, one of the largest social service NGOs in Egypt. The programs reach a lot of people and therefore the most surveys were collected in that country. In contrast, OGE’s work in Mexico is in a few very rural villages so the 174 completed surveys represent a very large number of the participants in OGE’s programs there. 4.3 Background Characteristics of Respondents Table B presents the demographics of survey respondents. This is a very detailed and representative snapshot of OGE’s constituency, and it reflects OGE’s focus on populations underrepresented in the economic mainstream, particularly women, young people, and the unemployed. Overall, 59% of the respondents were women; while the gender of respondents varied significantly between countries the range was not large (56% in Mexico to 65% in South Africa). See Spotlight: Gender for a detailed analysis of the impact of OGE’s programs on women. Figure 14. Data for analysis by country
  40. 40. 36 Digital Inclusion Social Impact Evaluation Of note, most respondents were between the ages of 18 and 34 at the time of the survey (90%). This is partially because of the large amount of respondents in Egypt (55% of the total survey) who were of that age (2,400 of the 2,407). However, that trend held across the other countries as well, with 78% of Indian, 91% of Mexican, and 85% of South African respondents below the age of 30. This reflects the emphasis OGE’s programs have on training young people because youth around the world often lack job prospects and job skills, and employment as a young adult is one of the best routes out of poverty. Young people, who are more likely to have free time and less likely to have families, are particularly able to take advantage of the job skills and entrepreneurial programs that OGE offers. Almost half of all Egyptians surveyed (48%) are unemployed even though 69% have some college experience or have graduated from college. This speaks to the difficult economic situation that many Egyptians face, but also to the benefits of becoming as job-ready and as good at the soft skills of interviewing and networking as possible. In India, despite a low unemployment rate, many of the respondents were students (27%) or homemakers (25%). These are two populations that can greatly benefit from digital literacy and job skills as well as entrepreneurial training because of the opportunities they provide for small business creation and joining the job force.
  41. 41. 37 Egypt (N=2,407) India (N=1,292) Mexico (N=174) South Africa (N=491) Total (N=4,364) Gender* Male 1,010 (42.2%) 526 (40.8%) 76 (43.7%) 168 (35.3%) 1,780 (41.1%) Female 1,382 (57.8%) 763 (59.2%) 98 (56.3%) 308 (64.7%) 2,551 (58.9%) Age** 12-18 3 (0.1%) 2 (0.2%) 69 (39.7%) 2 (0.4%) 76 (1.7%) 18-24 1,376 (57.2%) 598 (46.4%) 66 (37.9%) 255 (52.3%) 2,295 (52.7%) 25-34 1,022 (42.5%) 406 (31.5%) 25 (14.4%) 157 (32.2%) 1,610 (37.0%) 35-44 4 (0.2%) 197 (15.3%) 9 (5.2%) 53 (10.9%) 263 (6.0%) 45-54 0 74 (5.7%) 4 (2.3%) 15 (3.1%) 93 (2.1%) 55-64 0 9 (0.7%) 0 6 (1.2%) 15 (0.3%) 65+ 0 3 (0.2%) 1 (0.6%) 0 4 (0.1%) Marital status** Married 594 (24.9%) 642 (50.3%) 25 (14.5%) 49 (10.2%) 1,310 (30.3%) Separated/Divorced/ Widowed 23 (1.0%) 23 (1.8%) 7 (4.0%) 19 (4.0%) 72 (1.7%) Single 1,771 (74.1%) 612 (47.9%) 141 (81.5%) 413 (85.8%) 2,937 (68.0%) Highest education** Elementary 16 (0.7%) 246 (19.0%) 15 (8.6%) 2 (0.4%) 279 (6.4%) Middle school 10 (0.4%) 323 (25.0%) 53 (30.5%) 9 (1.9%) 395 (9.1%) Some high school 43 (1.8%) 214 (16.6%) 20 (11.5%) 54 (11.0%) 331 (7.6%) High school grad 416 (17.3%) 232 (18.0%) 32 (18.4%) 305 (62.1%) 985 (22.6%) Some college or technical school 258 (10.7%) 173 (13.4%) 16 (9.2%) 60 (12.2%) 507 (11.6%) College or technical school graduate 1,553 (64.5%) 97 (7.5%) 32 (18.4%) 61 (12.4%) 1,743 (39.9%) Other 111 (4.6%) 7 (.5%) 6 (3.4%) 0 124 (2.8%) Employment** Homemaker 118 (5.0%) 320 (24.8%) 14 (8.2%) 9 (1.9%) 461 (10.7%) Retired 1 (0.1%) 5 (0.4%) 2 (1.2%) 0 8 (0.1%) Self-employed 246 (10.4%) 88 (6.8%) 8 (4.7%) 44 (9.2%) 386 (9.0%) Student 46 (2.0%) 349 (27.1%) 95 (55.6%) 145 (30.2%) 635 (14.8%) Unemployed 1,139 (48.4%) 62 (4.8%) 9 (5.2%) 197 (41.0%) 1,407 (32.8%) Working for an employer 801 (34.1%) 465 (36.1%) 43 (25.1%) 85 (17.7%) 1,394 (32.5%) * p<0.05 ** p<0.01 Table B. Background characteristics of respondents
  42. 42. 38 Digital Inclusion Social Impact Evaluation 4.4 Training and Access As seen in Figure 15, 3,562 people who took the survey (82%) participated in one or more trainings by OGE or one of its partners between 2009 and 2014. Most respondents reported participating in a training between 2012 and 2014, largely from respondents in Egypt and India. The large numbers of people participating in trainings in Egypt and India is mainly due to three factors: 1) Egypt and India were the largest number of respondents to the survey, 2) the partners’ large reach in these countries, and 3) the nature and scale of OGE’s programs in these countries. In both India and Egypt, OGE works with larger, local NGOs, Pratham and CEOSS, respectively, both of which have a greater reach than its partners in Mexico. OGE implemented a large-scale, multi-year project in Egypt, designed to train at least 1,200 youth. In India, OGE launched its training curriculum at Pratham’s community technology centers in Pune, training over 2,900 individuals in 2012, and built the capacity of 10 centers reaching 15 communities in Maharashtra State in 2013-2014 to implement a number of training programs that served over 4,000 people. Siyafunda, OGE’s partner in South Africa, also has a large network of community technology centers located in mostly rural townships in Johannesburg. However, OGE only works with about 5-8 of their centers per year, as the project is smaller in scale. Finally, in Mexico, OGE works with three centers in very small, remote villages, as well as small classes at local high schools in Toluca, in partnership with UAEM. Missing training date from 204 respondents. Figure 15. Training by country by year
  43. 43. 39 In Egypt and India, most respondents (92% and 93% respectively) had participated in any training by OGE or one of its partners compared to half the respondents in South Africa (62%) and a quarter in Mexico (29%). The most common trainings were Career guidance and job hunting (73% of those who participated in a training) and Community Connectors (66%). It is important to note that the training programs in each country all focus on digital literacy, technology adoption, and leadership skills. The leadership focus varies in each country depending on the target population. In Mexico, leadership skills taught are core soft and hard skills that can be exercised in an academic or professional setting, or used to become actively involved in community development initiatives. In Egypt and South Africa, there is an emphasis on entrepreneurship and management, and in South Africa, India and Egypt, on employment. Therefore, there exists great overlap in training content from country to country, largely due to the fact that digital literacy is very much connected to and has the impact to improve a number of other, more specific skill sets pertaining to peoples’ livelihoods. It can be inferred from these results and based on each country’s unique program offerings that not every respondent indicated the training they necessarily participated in. For example, the Mobile Entrepreneurs Program, only offered in South Africa, consists of modules focused on entrepreneurship, but also contains modules that teach career development skills so that entrepreneurs also learn how to get jobs, if the choose, that allow them to utilize their entrepreneurial skillset. Therefore, some of the respondents may have said they had received training in career guidance and other job-related skills if they participated in this training. In Mexico, respondents who indicated they received career guidance and job training could represent adults who participated in Community Connector-led technology workshops at their local centers, which are aimed at parents, many of which are first-time Internet users, who are interested in learning how to look and apply for jobs online.
  44. 44. 40 Digital Inclusion Social Impact Evaluation Table C. Background characteristics of respondents Egypt (N=2,407) India (N=1,292) Mexico (N=174) South Africa (N=491) Total (N=4,364) Any training** 2,204 (91.6%) 1,205 (93.3%) 51 (29.3%) 306 (62.3%) 3,766 (86.3%) Career guidance and job hunting** 2,142 (97.2%) 483 (40.1%) 12 (23.5%) 104 (34.0%) 2,741 (72.8%) Community Connectors** 1,822 (82.7%) 559 (46.4%) 32 (62.8%) 72 (25.5%) 2,485 (66.0%) Business** 97 (4.4%) 29 (2.4%) 7 (13.7%) 112 (36.6%) 245 (6.5%) Health camp** 0 224 (18.6%) 0 0 224 (6.0%) Digital literacy** 0 166 (13.8%) 1 (2.0%) 7 (2.3%) 174 (4.6%) Internet security and IT adoption** 1 (0.1%) 86 (7.1%) 9 (17.7%) 7 (2.3%) 103 (2.7%) Mobile Entrepreneurship** 10 (0.5%) 0 0 84 (27.5%) 94 (2.5%) Professional and soft skills** 0 8 (0.6%) 4 (7.8%) 15 (4.9%) 27 (0.7%) Managers** 4 (0.2%) 0 6 (11.8%) 6 (2.0%) 16 (0.4%) Other (not specified) 5 (0.2%) 116 (9.6%) 1 (2.0%) 5 (1.7%) 127 (3.4%) None 220 (9.1%) 95 (7.4%) 123 (70.7%) 191 (38.9%) 629 (14.4%) * p<0.05 ** p<0.01 The majority (71%) of respondents in Egypt reported having Internet access at home; in the other countries Internet access in the home varied from 14% in India to 41% in Mexico (Table D). In India, respondents most commonly reported that they accessed the Internet at a CKC (84%) while in Mexico almost half of the respondents reported accessing the Internet at home (40%), school (43%), and at a CKC (37%) in addition to many using telecenters (29%) and work (19%). In South Africa, the majority said they accessed the Internet at a CKC (67%); while some reported accessing the Internet at home (20%), school (15%), or at a telecenter (14%).
  45. 45. 41 Egypt (N=2,407) India (N=1,292) Mexico (N=174) South Africa (N=491) Total (N=4,364) Where do you access the Internet? At home** 1,692 (70.3%) 173 (13.4%) 70 (40.2%) 101 (20.6%) 2,036 (46.7%) At school** 14 (0.6%) 14 (1.1%) 75 (43.1%) 77 (15.7%) 180 (4.1%) At work** 171 (7.1%) 112 (8.7%) 33 (19.0%) 36 (7.3%) 352 (8.1%) At a telecenter** 120 (5.0%) 8 (0.6%) 50 (28.7%) 67 (13.7%) 245 (5.6%) At a CKC** 457 (19.0%) 1,086 (84.1%) 65 (37.4%) 329 (67.0%) 1,937 (44.4%) On a mobile phone** 58 (2.4%) 155 (12.0%) 3 (1.7%) 12 (2.4%) 228 (5.2%) Using a mobile broadband USB stick** 20 (0.8%) 0 0 0 20 (0.5%) At a library 0 0 0 1 (0.2%) 1 (0.1%) Have internet access at home** 1,714 (71.2%) 183 (14.2%) 72 (41.4%) 138 (28.1%) 2,107 (48.3%) * p<0.05 ** p<0.01 Table D. Background characteristics of respondents 4.5 Access Local Content The high percentage of respondents who have accessed the digital platform in Egypt and India is reflective of the reach of OGE’s partners in those countries, as well as the high number of access points. Only 25% of respondents in South Africa reported ever accessing the digital platform. This number is somewhat surprising because OGE’s partner in South Africa also has a significant reach and number of access points, however this could be due to the small sample size taken from South Africa. If OGE had surveyed a larger number of respondents from a greater variety of townships where its access points are located, results might have shown a higher percentage of people who have accessed the platform. The small percentage of people who have ever accessed the digital platform in Mexico is representative of the scale of OGE’s work and the number of access points (three), which are located in small and rural towns.
  46. 46. 42 Digital Inclusion Social Impact Evaluation Table E. Read about topics on the digital platform Egypt (N=2,407) India (N=1,292) Mexico (N=174) South Africa (N=491) Total (N=4,364) Ever accessed the digital platform** 1,758 (73.0%) 1,114 (86.2%) 44 (25.3%) 124 (25.3%) 3,040 (69.7%) Topics accessed: Agriculture** 317 (13.2%) 400 (31.0%) 14 (8.1%) 23 (4.7%) 754 (17.3%) Citizenship** 109 (4.5%) 165 (12.7%) 10 (5.8%) 26 (5.3%) 310 (7.1%) Health** 1,046 (43.5%) 998 (77.2%) 26 (14.9%) 47 (9.6%) 2,117 (48.5%) Education** 1,004 (41.7%) 906 (70.1%) 21 (12.1%) 58 (11.8%) 1,989 (45.6%) Recommended articles to other people** 1,495 (85.0%) 325 (29.2%) 9 (17.7%) 74 (59.7%) 1,917 (63.1%) * p<0.05 ** p<0.01 Figure 16, which shows the top three content areas by country, demonstrates that content on the digital platform is tied to programmatic themes, but also how content that the popularity of content is not always directly related to training topics and may serve as key resources in supporting peoples’ livelihood development. For example, while the programmatic focus in OGE’s programs in Egypt was employment and entrepreneurship, the program served some rural communities where agribusinesses account for many of the small businesses. Therefore it is unsurprising that agriculture was a popular topic among people accessing the Egypt digital platform content. All three of India’s top three content areas are reflective of training topics and services offered at partner centers. In this case, the India results correlate to program objectives focused on supporting people’s improvement in health, education, and employment. In Mexico, there is no training program focused solely on health, so the large percentage of people viewing health content could be indicative of both the demand for and the value of localized information on the topic. A key section of the Mexico digital platform is the Homework Help section, which was developed to include links to online educational resources such as tutorials and activities for over 100 concepts in math, science, reading and other core subjects for primary, secondary, and high schools students, aimed at guiding parents in helping their children improve their academic achievement. According to Google Analytics, more than a million users have accessed the Homework Help section, which could contribute to education being among a top content area for Mexico. South Africa’s program has a strong emphasis on employment and entrepreneurship, which is also captured by the top content results.
  47. 47. 43 All programs emphasize using and recommending others to the digital platform for the purpose of accessing global best practice information and community development-focused content on locally relevant issues and topics. The digital inclusion ecosystem OGE has created with its partners— access, content, and training—is designed to support word of mouth and actions taken to increase technology adoption and usage of the digital platform in the communities it serves. Respondents were asked if they recommended articles on the digital platform to others, and if so what topics they recommended. There was wide variation between countries in both the frequency of recommending articles and they type of content. Respondents from Egypt were most likely to report recommending articles on the digital platform (85%), followed by South Africa (60%) and Mexico (52%). Respondents from India were least likely to report recommending articles on the digital platform to others. The topics that were recommended mirrored the top content in each country. 4.6 Take Informed Action The core of all of OGE’s programs is to help people to be better able to help themselves by acquiring skills and gaining access to better information via information technology to make informed decisions about their lives. We measured this by asking not only what actions people took after they attended a training or after they accessed content on the digital platform, but if those actions improved their lives by gaining employment, starting a business, earning more money, increasing their savings, and improving their health and/or education. Figure 16. Top three content areas by country Figure 17. Ever recommended articles and top three topics recommended on the digital platform
  48. 48. 44 Digital Inclusion Social Impact Evaluation Figure 18 shows the how often respondents in different countries report they go online. It is very encouraging that more than half of respondents reported that they go online at least once each day (orange bars in Figure 19). However, respondents in India reported going online with much less frequency: only 29% reported going online daily and 43% reported going online only once every few months (vs. less than 2% of respondents in the other countries). This speaks to the need for computer centers and other points of access in India and the benefit of online training and content in the other countries. Figure 19 shows that only 29% of all respondents use email for personal communication and 10% use it for business communication. This is a critically important skill, particularly in the 21st century economy. It is possible that so few people use email for business because many of the respondents are either unemployed or have a job that does not require sending email. That said, the low numbers of people using email for personal use are indicative of low levels of digital literacy given that so many of the respondents reported going online regularly. It is very important to have an online identity for finding and getting a job as well as making the types of connections that can lead to professional advancement. Email is one way to create an online identity and social Figure 18. How often do you go online?
  49. 49. 45 networking is another, which is why it is encouraging that almost two thirds of respondents reported using online social networks (61%). This represents a great opportunity to teach people how to use social networking to form ties to their communities and to advance their professional lives, particularly in Egypt where unemployment is high, Internet use is very regular, and many respondents (43%) reported using the Internet to search and apply for jobs. One reason for this high number is the OGE’s job-matching program where Egyptians post their skills on an online database and employers post job openings. Lastly, the data makes it clear that many people use the internet to try to improve their lives by studying, researching, applying for jobs, and social networking rather than just for frivolous purposes. Less than 1% of respondents said they used the internet for downloading music and watching movies online [data not shown]. Figure 19. How often do you go online? ** p <0.01
  50. 50. 46 Digital Inclusion Social Impact Evaluation 4.7 Increased Opportunities in Income USE IDENTITY TO PARTICIPATE IN THE LOCAL AND GLOBAL ECONOMY Over a quarter of the respondents reported having bank accounts and using online banking, as seen in Table E (27%). However, there was wide variation across countries: only 2% of respondents in Egypt reported having a bank account compared to 70% in South Africa. However, of those who had bank accounts, those from Egypt were much more likely to access their bank account online (69% of those with a bank account, compared to 16% overall). Very few respondents reported paying their bills online (5%). Egypt (N=2,407) India (N=1,292) Mexico (N=174) South Africa (N=491) Total (N=4,364) Have a bank account** 55 (2.3%) 751 (58.1%) 39 (22.4%) 344 (70.1%) 1,189 (27.3%) Access bank account online** 38 (69.1%) 56 (7.5%) 21 (53.9%) 71 (20.6%) 186 (15.6%) Pay bills online** 74 (3.1%) 68 (5.3%) 25 (14.4%) 31 (6.3%) 198 (4.5%) * p<0.05 ** p<0.01 START A BUSINESS OGE administers business trainings that aim to help people start businesses and employ best practices to maximize the chances that these businesses will be successful. In these trainings people learn how to create an operational budget, apply for loans and grants, and create marketing and business plans amongst other activities. Of the 245 respondents who said they had participated in one of OGE’s business trainings, 46 reported that they started a business after training (19%). Those who started a business were much more likely to have implemented business best practices than those that did not, such as creating operational budgets (57% vs. 19%), a business plan (70% vs. 44%), and a marketing plan (52% vs. 19%). However, many respondents implemented these best practices without starting a business. For example, 44% of respondents who didn’t start a business after the business training stated that they created a business plan. Since the majority of respondents attended a training within the last year, the survey may have captured people in the process of starting a business. With increased time since the training, future results may show a higher proportion of trainees starting a business. [data not shown] Table E. Read about topics on the digital platform