If you don’t know where you’re going, how are you gonna’ know when you get there?–Yogi BerraSpecific – state in concrete, detailed and well‐defined terms – What exactly are we going to do for whom? Measurable – should be quantifiable and the source of measurement has been identified. Attainable/Achievable – can the objective be achieved in the proposed time frame with the resources available? Relevant/Realistic – is the objective directly related to the overarching communication goal from your communication plan? Time‐bound – have deadlines been set?M&E is a continuous process that occurs throughout the life of a program. To be most effective, M&E should be planned at the design stage of a program, with the time, money, and personnel that will be required calculated and allocated in advance. Monitoring should be conducted at every stage of the program, with data collected, analyzed, and used on a continuous basis. Monitoring should be conducted at every stage of the program, with data collected, analyzed, and used on a continuous basis. Every project or intervention should have a monitoring and evaluation (M&E) plan. This is the fundamental document that details a program’s objectives, the interventions developed to achieve these objectives, and describes the procedures that will be implemented to determine whether or not the objectives are met. It shows how the expected results of a program relate to its goals and objectives, describes the data needed and how these data will be collected and analyzed, how this information will be used, the resources that will be needed, and how the program will be accountable to stakeholders. M&E plans should be created during the design phase of a program and can be organized in a variety of ways. Typically, they include: The underlying assumptions on which the achievement of program goals depend; The anticipated relationships between activities, outputs, and outcomes; Well-defined conceptual measures and definitions, along with baseline values; The monitoring schedule; A list of data sources to be used; Cost estimates for the M&E activities; A list of the partnerships and collaborations that will help achieve the desired results; A plan for the dissemination and utilization of the information gainedFrameworks are key elements of M&E plans that depict the components of a project and the sequence of steps needed to achieve the desired outcomes. They help increase understanding of the program's goals and objectives, define the relationships between factors key to implementation, and delineate the internal and external elements that could affect its success. They are crucial for understanding and analyzing how a program is supposed to work.
M&E Plan Components: Indicators Indicators are clues, signs or markers that measure one aspect of a program and show how close a program is to its desired path and outcomes. They are used to provide benchmarks for demonstrating the achievements of a program.One of the most critical steps in designing an M&E system is selecting appropriate indicators. The M&E plan should include descriptions of the indicators that will be used to monitor program implementation and achievement of the goals and objectives.An indicator is a variable that measures one aspect of a program or project that is directly related to the program’s objectives. Monitoring should be conducted at every stage of the program, with data collected, analyzed, and used on a continuous basis. Throughout the data collection process it is essential that data quality be monitored and maintained. Data quality is important to consider when determining the usefulness of various data sources; the data collected are most useful when they are of the highest quality. It is important to use the highest quality data that are obtainable, but this often requires a trade off with what it is feasible to obtain. The highest quality data are usually obtained through the triangulation of data from several sources. It is also important to remember that behavioral and motivational factors on the part of the people collecting and analyzing the data can also affect its quality.How the information gathered will be stored, disseminated, and used should be defined at the planning stage of the project and described in the M&E plan. This will help ensure that findings from M&E efforts are not wasted because they are not shared. Here are some data quality issues to consider:Coverage: Will the data cover all of the elements of interest?Completeness: Is there a complete set of data for each element of interest?Accuracy: Have the instruments been tested to ensure validity and reliability of the data?Frequency: Are the data collected as frequently as needed?Reporting Schedule: Do the available data reflect the time periods of interest?Accessibility: Are the data needed collectable/retrievable?Power: Is the sample size big enough to provide a stable estimate or detect change?
Step 1: The first step in social media measurement is to determine what you’re trying to accomplish and how you will approach it.Step 2: Determine how you will measure success. Metrics development should follow the same process. First, determine how you will measure success from an organizational perspective – whether it is to drive brand/product awareness, source competitive insights, improve search engine placement, or learn from audiences – before you approach it from a social perspective. Step 3: Evaluate your organization’s readiness to measure social media This in one of the most critical elements of social media measurement strategy. Assess your resources, the level of domain, analytical and tool expertise needed, and the current state of internal collaboration. Many organizations lack sufficiently trained staff for social media measurement and delegate it to overcommitted and under-prepared employees. Step 4: Choose tools in light of strategy, metrics, and organization. Once you know what you’re trying to accomplish, how you’ll measure success, and what resources you have available, you’re ready for tool selection. This is still a very new industry, so be aware that tools are as yet immature and change quickly. There is no single best tool for every objective or every organization.
Blog:Improve global access to and knowledge of FP/RH services throghout developing countries. We highlight KM products and also share the latest news and insights currenty impacting the global health community. K4Health productsGlobal and public health news and eventsK4Health events, conferences, campaignPartner events and campaignsWays to take action and promote global health worldwideK4Health blogs 2-3 times per week; will amp up as blog redesignedBlogs are the center of our social media strategy. Everything blogged pushed out to all other media RSS feeds to facebook, tweeted, twitter RSS feed to linkIn. Pitched out for cross promotion to other blogsFacebookFacebook most effective opportunity for running campaign– update, revamp, launchDialogue and receive interesting articles from organizations and follows FB Ad on FB: K4Health, photoshare, Drviestraffice to FB page, and photoshareTwitter Use most often when at conferences (e.g. 30-40 tweets and retweets)Target audience for our tweets and followers: Partner organizations, NOT primary organizations; jounalists, communications professionalsThese feed into our linkin accountWe tweet everydayLinkedIn100 followers (closed group)Plan to open group and expand dialogue beyond tweetsRSS feed– automated content for social media tools
Blog:Improve global access to and knowledge of FP/RH services throghout developing countries. We highlight KM products and also share the latest news and insights currenty impacting the global health community. K4Health productsGlobal and public health news and eventsK4Health events, conferences, campaignPartner events and campaignsWays to take action and promote global health worldwideK4Health blogs 2-3 times per week; will amp up as blog redesignedBlogs are the center of our social media strategy. Everything blogged pushed out to all other media RSS feeds to facebook, tweeted, twitter RSS feed to linkIn. Pitched out for cross promotion to other blogsFacebookFacebook most effective opportunity for running campaign– update, revamp, launchDialogue and receive interesting articles from organizations and follows FB Ad on FB: K4Health, photoshare, Drvies traffice to FB page, and photoshareTwitter Use most often when at conferences (e.g. 30-40 tweets and retweets)Target audience for our tweets and followers: Partner organizations, NOT primary organizations; jounalists, communications professionalsThese feed into our linkin accountWe tweet everydayLinkedIn100 followers (closed group)Plan to open group and expand dialogue beyond tweetsRSS feed– automated content for social media tools
We use these various forms of social media. However, one our biggest promotion tools is cross-promotion. Above is an example of how we cross-posted a blog on the Interagency Youth Working Group site and both K4Health and IYWG tweeted about it.
We use a few things to measure our social media. Facebook Insights: Are extremely useful and free measure of your facebook pages. When you are an admin on a facebook page you can see the insights link appear in the top right corner. You can sort by data ranges and it keeps all the historical data as well. Google Analytics: the basic analytics of your sites compared with specific social media campaigns can help see if a particular campaign has led to a spike in your analytics. This is a great measure. For example, our Project Director tweets one a month our top three visited toolkits and we usually see a spike in visits to our site based on that. In order to track these things, it is important to annotate this on your analytics (see next slide). Another great use of Google Analytics is to note the referring sites to see if you are actually getting clicks from one of your social media sites.
In order to follow specific events, we place annotations within our google analytics, this allows us to compare specific events in time to spikes or lulls in our analytics. We don’t have to use our memories to remember what was happening on that particular day or that month.
This graphic is from google analytics and it shows that we are now back on a upswing in our metrics. We have focused more on our blog writing lately and cross-promotion. We are also able to recognize that our decline was based on a lack of attention to the blog as well as a transition in leadership.
Here are some examples of ways to measure your reach with twitter or other social media. They do not need to be expensive, but some measure more impact than just analytics. The challenge is to gauge your impact over time using these different measures. Depending on budget or what your metrics are depends on which service bets fit your situation. The important thing is to have a strategy before you begin doing social media. Twoolr: Just for Tweets, does not do retrospective. So if you do not sign up for it when you start your twitter account you will only know the information from when you start. ($9.90/month or $49/year)Hootsuite: Can be used for any social media: twitter, facebook, linkedIn, Ping.fm, Wordpress, MySpace, Foursquare, and mixi. Free version allows you to organize your posts and schedule future posts. And if you upgrade to pro for $5.99/month you can create custom analytics for all your different social media networks. Tweetreach: shows your reach in terms of the past 50 tweets you have done. So it shows the type of tweet (retweet, @replies, or regular tweets) and how much exposure and impressions your tweets have had. For a full report it costs about $20. Adobe Social Analytics powered by Omniture part of the Adobe Online Marketing Suite:
Are Friends a Dime a Dozen? Establishing Social Media M&E
Are Friends a Dime a Dozen? Establishing Social Media M&E<br />USAID Global Health Mini-University<br />Chris Rottler, Forum One Communications<br />Tara Sullivan, JHUCCP (K4Health Project)<br />Vince Blaser, IAVI<br />September 30, 2011<br />
Agenda<br />New Media 101<br />Monitoring & Evaluation 101<br />Key Social Media Metrics<br />Case Studies (K4Health & IAVI)<br />Discussion<br />
Key Communication Objectives for Using New Media<br />Create awareness of an issue, project, etc.<br />Launching a project or major resource<br />Establish a need among the right audience<br />Comparing projects/services<br />Develop or change opinion<br />Influence the influencers<br />Drive action and traffic among the right audience<br />Establish credibility and trust<br />
Facilitate information sharing among audience networks<br />Expand reach to include more diverse audiences<br />Facilitate interactive communication, connection, and public engagement <br />Increase the timely dissemination and potential impact of health and safety information<br />Promote an organization’s work and results<br />Develop strategic communication partnerships<br />Raise awareness and advocate for global health issues<br />Key (Global Health) Communication Objectives for Using New Media<br />
New Media 101<br />Note: Presence management framework derived from original work by Chris Brogan<br />
Online Presence Framework<br />Home Base: Priority 1 (50% of your time budget)<br />“Social” portion of your hosted presence<br />Outposts: Priority 2 (40% of your time budget)<br />Key social sites that you actively participate in<br />Passports: Priority 3 (10% of your time budget)<br />Profiles on lower priority social sites<br />Mostly to listen, occasionally participate<br />Note: Presence management framework derived from original work by Chris Brogan<br />
Key Social Media Metrics<br />Step 1 (Strategy): Align your strategy with organization’s objectives<br />Step 2 (Metrics): Determine how you will measure success<br />Step 3 (Organization): Evaluate your organization’s readiness to measure social media<br />Step 4(Technology): Choose tools in light of strategy, metrics, and organizaiton (Etlinger, 2011) <br />
Establishing a systematic process <br />to collect, analyze, and use social media metrics<br />Tara Sullivan, Knowledge Management Director<br />Rebecca Shore, Communications Specialist<br />Kate Stence, Communications Manager<br />Saori Ohkubo, M&E Advisor<br />
K4Health Mission<br />To increase the use and dissemination of evidence-based, accurate and up-to-date information to improve health service delivery and health outcomes worldwide.<br />Make quality health information easy-to-find and easy-to-use<br />
Quality Over Quantity<br />M&E lessons from the International AIDS Vaccine Initiative’s new media efforts <br />Vince Blaser<br />New Media Specialist<br />Washington, DC 11th Annual Global Health Mini-University <br />9/30/11<br />
What is the International AIDS Vaccine Initiative (IAVI)?<br /><ul><li> IAVI is a global, non-profit, public-private partnership with a mission to ensure the development of safe, effective, accessible, preventive HIV vaccines for use worldwide
We work with partners around the world, with HQ in New York and field offices in Africa, Europe and India
The bulk IAVI’s funding is spent on R&D for vaccine candidates. We also conduct policy analyses, advocate for the field and engage communities in the trial process and AIDS vaccine education</li></li></ul><li>IAVI’s Communication Objectives<br />Main Objective: Inform relevant audiences that an AIDS vaccine is possible and vital, and that IAVI’s programs in particular are valuable <br />Secondary Objectives:<br /><ul><li> Maintain and increase support for AIDS vaccine development
Reinforce IAVI as a “go-to” organization and partner on AIDS vaccine development
Raise appropriate awareness of IAVI’s activities and innovations</li></li></ul><li>IAVI’s New Media Audit – What We Hoped to Learn<br /><ul><li> What are the major conversations?
Average of 400-500 posts on FB and Twitter daily related to our issues, 1,000+ daily during the 2010 International AIDS Conference
Key stakeholders were reading global health, AIDS, development and political blogs in addition to traditional news sources
Lessons for engagement</li></li></ul><li>Findings – What Are the Major Conversations?<br />News, news and more news<br />@SciDevNet: India, South Africa to team up on HIV vaccine research http://bit.ly/d6sPEx<br />Support for HIV vaccine R&D<br />@OnlyInterject: Computational Biology @ work! via @AIDSvaccine Determining shape of viral protein could spur #HIVvaccinedev http://bit.ly/bmSgZK#AIDSvax10<br />The conspiracies, the misinformed and the sarcastic<br />@marshallbock: @naporeon Vaccines cause autism! And AIDS! AIDSTISM!<br />@JoziStylista: RT @BeigeTheColour: One would imagine that a small dose of HIV blood is one of the ingredients in the AIDS vaccine...<br />
Findings – What is Our Audience Looking For?<br /><ul><li> They’re looking for news
They’re willing to help spread the word</li></li></ul><li>Creating and Evaluating IAVI’s New Media Strategy<br />Opportunities<br /><ul><li>To reflect changes in media landscape by placing IAVI spokespeople and partners in key blogs for stakeholders
To post the latest on AIDS vaccines and IAVI in social media outlets that are most used by stakeholders and best fit IAVI’s communications objectives
To keep a pulse on latest discussions about our field from our major stakeholders as well as general audiences</li></ul>Challenges<br /><ul><li>To maintain the organization’s expertise and credibility in the new media space
To avoid wasting limited staff time on relatively ineffective new media outlets and conversations not productive toward our mission
To focus evaluation of new media strategy on key stakeholder engagement</li></li></ul><li>Where We Focus (and Where We Don’t) – Blogs<br />Blog placement<br /><ul><li>Scientific blogs (Science Insider, Science Speaks: HIV & TB News)
AIDS, global health policy blogs (ONE Campaign Blog, Global Health Magazine Blog)
Policy/politics blogs (Huffington Post, MFAN blog, The Hill Congress Blog)
Major media blogs (CNN, Guardian Global Health Policy Blog)
Development and donor blogs (USAID Impact Blog, Scidev.net)
NOTour own blog</li></li></ul><li>Where We Focus (and Where We Don’t) – Social Media Outlets<br />Twitter (@AIDSvaccine)<br /><ul><li>High emphasis, most staff time and most posts</li></ul>Facebook (facebook.com/AIDSvaccine)<br /><ul><li> Second highest emphasis and staff time – most posts and multimedia content related directly to IAVI</li></ul>YouTube (IAVIvideos)<br /><ul><li>Time varies by project and goal of videos</li></ul>LinkedIN<br /><ul><li> Discussion board and group maintained for purposes of recruiting and informing field of major breakthroughs</li></ul>Other outlets<br /><ul><li>Little time spent on MySpace, Flickr. Wait-and-see on new outlets like Google+ </li></li></ul><li>What We Post About<br />IAVI news, announcements and reports<br /> Developing an #HIV#AIDS#vaccine isn't just science. Check out our brief on IAVI's vaccine preparedness approach http://bit.ly/c5xv7Q<br />AIDS vaccine, global health/development news, trends<br /> Great new resource on CAPRISA #HIV#microbicide trial results & what's next from AVAC (@HIVpxresearch). Check it out! http://bit.ly/aIuepe<br />Inspiring quotations and key messages<br /> New U.S. #MDG plan: "We will pursue breakthroughs in public health across the developing world," incl#HIV#vaccineshttp://bit.ly/bh5TCh<br />Major events<br />#AIDS2010, what do you know future #HIV prevention choices? Earn your prize at the Global Village Women’s Networking Zone Tue 20th 5:15pm!<br />Responses, messages to other posters<br /> @Ballona: #HIV antibodies discovery is step fwd in developing an effective #AIDS vaccine, but much work remains.<br />
How We Evaluate<br />Quantitative Methods<br /><ul><li> Website traffic numbers, followers, YouTube views, bit.ly clickthroughs, etc.
Volume of discussion on our topic areas that utilizes our information and outreach</li></ul>Qualitative methods<br /><ul><li>Feedback from key stakeholders