By Rachel Weidinger, Rachel Dearborn, Matt Fitzgerald,Saray Dugas, Kieran Mulvaney and Britt Bravohttp:/     /upwell.usTwi...
Table of ContentsI: Executive Summary                              Page 3II: Introduction                                 ...
Executive SummaryThe ocean is in crisis, plagued by overfishing, habitat loss, and acidification, among other issues.While...
In our work to date, the team at Upwell has come to believe that there are three measurablecharacteristics of the online o...
the quickest, dirtiest thing we can get out the door that we think will have a measurable effecton a conversation.By apply...
• Personal: We build relationships with humans, not organizations. The liveliest online      conversations happen between ...
Over time, we’ve seen the number of social mentions generated from each attention campaigngrow, concurrent with the growth...
Overfishing 14000                                                                                                         ...
starved ocean activists. According to our February 2013 survey, through these tools andopportunities, Upwell has helped th...
about five times higher than them.Collectively, overfishing represents a grab bag of ocean brands. The Overfishing convers...
are usually based in one of three elements: well-known brands promoting their sustainableofferings (Safeway, McDonalds), f...
Emergent best practices for online campaigning from the Upwell pilot include:   • You can’t predict what will go viral.   ...
• Encourage a flat structure.   • Keep the campaigning team small, but not too small.   • Keep time for developing creativ...
Marine Protected Areas:   • The MPA conversation is tiny in comparison to other ocean conversations.   • Our MPA vocabular...
IntroductionOcean Conservancy and the Waitt Foundation collaboratively developed the Upwell project in2010. The project’s ...
16
Theory of Change andContext for Our WorkThe ocean is in crisis, plagued by overfishing, habitat loss, and acidification, a...
Testing the WatersWith the loose ideas of direction (detailed above) in mind, we reached out to members of thecommunity to...
A Social FocusWe decided early on during this scoping process that social media was going to be our playingfield. Campaign...
mission when they fully appreciated that we would not be a part of that competition and wouldbe trying to find ways to mak...
organizations and academics could provide. Some organizations outside the ocean sector arebeginning to experiment with thi...
We Shall Have Bigger Ears and EyesInto the InternetIf Upwell was going to operate on behalf of Team Ocean, we were going t...
Step change vs. Incremental Change [source]Upwell entered this challenge looking for step change—a massive, discontinuous ...
Metrics: Social MentionsOur primary metric for understanding the conversations analyzed in this report is what we referto ...
Methods: Big ListeningIntroductionUpwell employs Big Listening in order to understand the volume and character of onlineco...
Sharks, Tuna, Gulf of Mexico and Ocean Acidification. Our method of conversational analysis hasbeen called Big Listening, ...
Our war room is a little different. Instead of monitoring a corporate brand or a product, wemonitor the brand of the ocean...
each day by the topic’s diehard conversationalists. Ifeveryone else left the party, the Baseline would still bethere, danc...
We selected the mean to establish a specific value for each day of the week for three reasons.First, the mean is the start...
Social mentions for Upwell’s Sustainable Seafood keyword group vs. Upwell’s Sustainable SeafoodBaseline, June 1, 2012 - Au...
Since keyword groups have keywords added and subtracted on an ongoing basis, there are someinherent challenges. How should...
Upwell mind map for Oceans keywords, January 2012For marine debris the concept map would include items such as marine tras...
computational power, billions of dollars in said company’s algorithmic investments, and theconcentrated smarts of your own...
SCOPEInitial investigation   • Outline the conceptual and temporal boundaries of analysis for the topic   • In consultatio...
ADAPTRefining keyword groups   • Share keyword lists with key informants (subject experts, foundation staff, campaigners  ...
as it existed at a particular time (of export), from a particular tool (or combination of tools), theresulting values cann...
Social mentions for Upwell’s Sustainable Seafood keyword group vs. Upwell’s Sustainable SeafoodBaseline, June 1, 2012 - Au...
attention as a hump or a mesa, rather than the taller, more angular spike). Spikes look good oncharts, and they help push ...
Day-of-the-week values for the Sustainable Seafood Baseline, along with the Sustainable Seafoodmean, and mean +1x, +1.5x a...
Social mentions for Upwell’s Sustainable Seafood keyword group vs. Upwell’s Sustainable SeafoodBaseline vs. ‘Mean + 1 Stan...
What Does Spike Quantification Tell Us?Upwell’s spike quantification methodology is in alpha, so to speak, and going forwa...
Sustainable Seafood: Winter 2011Social mentions by day for Upwell’s Sustainable Seafood keyword group, as compared to theS...
Sustainable Seafood: Winter 2012Social mentions by day for Upwell’s Sustainable Seafood keyword group, as compared to theS...
Overfishing: Winter 2011Social mentions by day for Upwell’s Overfishing keyword group, as compared to the OverfishingBasel...
Overfishing: Winter 2012Social mentions by day for Upwell’s Overfishing keyword group, as compared to the Overfishingbasel...
Keyword SetsThe following search terms are Upwell Radian6 keyword sets for Upwell’s primary campaigntopics—Sustainable Sea...
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upwell Pilot Report 2013
Upcoming SlideShare
Loading in...5
×

Upwell Pilot Report 2013

5,104

Published on

During its first year of incubation, Upwell successfully pioneered the development of new methodologies in social monitoring, demonstrated success in elevating the ocean conversation above the baseline, earned praise for its non-branded approach to campaigning from social media thought leaders and attracted additional philanthropic interest in expanding the project beyond the intent of the pilot phase across a range of environmental issues. We are grateful for the Waitt Foundation’s significant initial investment, which provided the vision and commitment to launch this entrepreneurial initiative and are appreciative of other funding we have received for the project.

This is the final report of Upwell’s pilot phase, completed in February 2013.

Published in: News & Politics
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,104
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
8
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Upwell Pilot Report 2013

  1. 1. By Rachel Weidinger, Rachel Dearborn, Matt Fitzgerald,Saray Dugas, Kieran Mulvaney and Britt Bravohttp:/ /upwell.usTwitter: @upwell_usUpwell Pilot Reporthttp:/ /creativecommons.org/licenses/by-sa/3.0/
  2. 2. Table of ContentsI: Executive Summary Page 3II: Introduction Page 15III: Theory of Change and Context for Our Work Page 16IV: Process and Methodology Methods: Big Listening Page 25 Methods: Campaigning Page 50V: Metrics of Impact Attention Impacts and Graphs Page 75 Ocean Evangelist Capacity Impacts Page 96VI: Insights Comparative Ocean Conversation Page 114 Analytics Insights: Big Listening Page 138 Campaigning, Collaboration and Powerful Page 145 Amplifiers Network Map: Ocean Evangelists and Ocean Page 161 Voices Online 2
  3. 3. Executive SummaryThe ocean is in crisis, plagued by overfishing, habitat loss, and acidification, among other issues.While the ocean serves as the engine for our climate and plays a central role in the global foodsystem, it still fails to register for many as a relevant and primary issue. It is, quite literally, out ofsight and out of mind. The virtual invisibility of the ocean in public discourse is a major obstaclefor the ocean conservation community to adopt and implement solution-based policies.The key to Upwell’s success—and thus, the success of the ocean conservation community—isnot to blast new, shiny information into the interwebs, but rather to nurture and bridge virtualand real-life distributed, diverse networks, and to leverage the combined reach and power ofthose networks of communicators to participate in and amplify the best content and campaigns.In inventing a new kind of collaboration, we’ve provided the tools and the space, and relied onthe ever-growing community of ocean communicators to work together to make change.Upwell’s array of goals—to utilize the immediacy of online communications, experiment withways to increase the reach of valuable content, empower and foster a broader network of oceancommunicators, and enrich our understanding of the conversational ecosystem surroundingocean topics—coalesced our broader vision of “conditioning the climate for change.” Webelieve that by getting more people talking about ocean issues and raising the baseline ofconversation, broader audiences will be more likely to take action, change behavior, and pushfor policy change that will have positive effects for our oceans.Our primary metric for understanding the conversations analyzed in this report is what we referto as a “social mention” (or “social item”). Upwell defines a social mention as the text inclusionof a monitored keyword in a post on social media platforms like Twitter, Facebook, blogs,mainstream news with RSS feeds, forums/boards, YouTube or Pinterest. Social mentions areonline acts of self-expression in which individuals, organizations and other entities invest (atleast) a small amount of social capital.Upwell employs Big Listening in order to understand the volume and character of onlineconversations about ocean issues. Big Listening is the art and practice of tracking topical onlineconversations over time—listening to what “the internet,” writ large, is talking about. Whencombined with data-informed campaigning, Big Listening provides a methodology forincreasing both the frequency and volume of online conversation around a particular issue. Thebasic idea is to identify pockets of real-time or historical conversation, wherever they may be,and then to use that information to make the conversation bigger. Big Listening is distinguishedfrom traditional social media monitoring by its scale, fluidity, focus on issue or cause monitoring,and expanded access to historical data. 3
  4. 4. In our work to date, the team at Upwell has come to believe that there are three measurablecharacteristics of the online ocean conversation. We are increasingly attentive to: 1. Constant level of conversational volume (Baseline), 2. Notable outliers in increased volume (spikes), and 3. Density of conversational hotspots (spike frequency).Upwell practices Big Listening on English-language conversations in the following eight topicareas: Overfishing, Sustainable Seafood, Marine Protected Areas (MPAs), Oceans, Cetaceans(whales and dolphins), Sharks, Tuna, Gulf of Mexico and Ocean Acidification. For each topic,both real-time and historical data provide essential context for understanding the volume,evolution and characteristics of the overall conversation.Each topic we monitor is characterized and defined by a set of search terms (includingexclusions) that we refine on an ongoing basis. While we recognize the limitations of “keywordgroups,” such as their reliance on text-based results and the absence of contextual awareness,they do provide a powerful tool for analyzing online attention. The development and activerefinement of keyword groups is at the heart of Big Listening methodology.At the time of writing, our current Baseline (v3.1) is the average of the lowest 20% of socialmention values for a topic on a given day of the week. In addition to the Baseline, we tracksignificant increases in online attention for a particular topic, or spikes. When you graph thosesocial mentions, you can actually see that burst of attention ‘spike’ the graph—hence the name.Upwell defines a spike as occurring when the social mention volume for a given day meets orexceeds one standard deviation from the mean of all recorded values for that same day of theweek.Upwell’s campaigning model is informed by Big Listening data and combines a few additionalkey elements. Our campaigns are attention campaigns, focused on raising attention to oceanissues. They are minimum viable campaigns, operating on short time-frames and focused onrapid delivery of content, continuous learning and iteration. They are run and amplified across adistributed network, rather than being housed on and amplified by way of our own platforms.What we do with attention campaigns is drive more attention to existing content and actionsthat are not on our properties. They’re not associated with our brand. We use this loosely heldconnection, tying into the momentum of the news cycle and being strategically opportunistic inthe pursuit of creating spikes in attention.Through our minimum viable campaigns, we employ ongoing, iterative, continuous delivery ofcontent, resisting our urges toward perfection and providing irreverent, timely, contextualcontent to audiences immediately instead of strategizing for six months or a year. We focus on 4
  5. 5. the quickest, dirtiest thing we can get out the door that we think will have a measurable effecton a conversation.By applying both these models, Upwell has crafted a new way of campaigning that is easilydelivered, measured, and adapts to the ever-changing sea of conversation. In summary, throughour campaigns, Upwell: • Surfs existing conversations in order to increase and expand attention. • Measures social mentions (rather than policy outcomes, petition signatures, or public opinion) to evaluate the success of our campaign efforts. • Delivers, measures, and learns from campaigns on a short time cycle, embedding lessons and insights immediately. We sacrifice perfection. • Collaborates with a network of ocean stakeholders and curating a diverse set of existing ocean content, rather than building on our own brand and creating our own content. Our campaigns are not aligned with Upwell program priorities or policy goals, but instead amplify attention to the priorities and goals of those in our network. • Running our campaigns across a distributed network of ocean communicators, rather than relying on our own platforms as information hubs.Rather than collect a large set of official MOU’s and partner logos to put up on our website, webuilt a loosely held, distributed network. We’ve reached out to nodes of people who control thecommunications channels that reach lots of supporters and followers who are interested inocean issues. We’ve been scrappy and ruthless about who we put into that distributed network,trying to make it diverse and ensure the reach is big.These are the values that guide Upwell in building and strengthening our distributed network: • Trust: we share only science-based content, ensuring that other science-based institutions know that the content we share is trustworthy. • Transparency: we share our campaign and big listening data with our network, so they can apply our lessons in their own work. • Brand-agnostic: we work as willingly with Greenpeace as we do with Deep Sea News, as we do with the Facebook page “I Fucking Love Science.” We will share an organization or individual’s content or campaign, as long as it promotes ocean conservation goals and fits our curation criteria (detailed below). Often, promoting content from an array of brands means releasing control of the message. • Issue-agnostic: We aren’t only focusing on overfishing, through GMO salmon or catch shares, to cultivate the network. We amplify any ocean campaign or content as long as it fits our curation criteria, raising attention for the crisis the ocean faces. 5
  6. 6. • Personal: We build relationships with humans, not organizations. The liveliest online conversations happen between people, not institutions. We model the authentic behavior of the internet. • Generous: We provide small bits of advice and feedback to help our network do better. If their work will get more people talking about the ocean online, it fits with our mission.Our Big Listening practice helps us understand the volume and character of oceanconversations, individually and in relation to one another. It also helps us to strategically choosewhere to invest attention. Knowing the scale of conversations—for instance, that the sharksconversation regularly spikes to over 40,000 social mentions in a day (and often much higher),whereas the marine protected areas/marine reserves conversation sits at about 50 per day—helps us right-size our expectations for attention, identify pockets of audiences ripe forengagement, and time our campaigning efforts to capitalize on the regular ebb and flow ofconversation.We curate things to amplify that meet these criteria: • Good science • Socially shareable • Conservation impact • Building social capital • New influencers • Topical • Spikeability • Under amplifiedOnce we’ve identified an opportunity, choosing a tool for dissemination is only part of the battle.We often research, curate, and create in order to provide the most shareable content. There’s noexact science to what we do—our methods are mostly informed by years of experiencecampaigning in social media. However, a few scenarios, outlined below, highlight the mostcommon ways we approach attention campaigning. • Scenario 1: The science and the message is good, but the content isn’t shareable. • Scenario 2: There’s conversation beyond the ocean community. Can we tap into it? • Scenario 3: Team Ocean isn’t coordinated. Can we create more message redundancy? • Scenario 4: The Upwell network doesn’t have direct access to Big Listening data. Can we provide insights to make their campaigns more effective? 6
  7. 7. Over time, we’ve seen the number of social mentions generated from each attention campaigngrow, concurrent with the growth of our distributed network. This is the proof in the pudding.As we continue to expand Team Ocean and encourage networked sharing, the number of socialmentions about the ocean will increase, and ultimately increase the baselines of oceanconversations.Both the Sustainable Seafood and Overfishing conversations have substantially changedsince the founding of Upwell. Both distinct conversations have seen significant increases inspike volume, spike frequency, and ratio of average daily social mentions to the averagebaseline.Sustainable Seafood 1400 1400 1200 1200 1000 1000 800 800 600 600 400 400 200 200 0 0 Oct-11 Nov-11 Dec-11 Jan-12 Oct-12 Nov-12 Dec-12 Jan-13 Baseline Spike Threshold High Spike Threshold Sustainable Seafood Baseline Spike Threshold High Spike Threshold Sustainable SeafoodSide-by-side comparison for Winter 2011 (left) and Winter 2012 (right) showing social mentions byday for Upwell’s Sustainable Seafood keyword group, as compared to the baseline, spike thresholdand high spike threshold (Winter 2011: 10/17/2011 - 1/31/12; Winter 2012: 10/1/2012 - 1/29/13)In Winter 2011 (above left), when Upwell began Big Listening in Sustainable Seafood, socialmention volume was an average of 423 mentions per day. By Winter 2012 (above right),Sustainable Seafood social mention volume is up 29.9%. Spike frequency in the SustainableSeafood conversation increased 265%. Those spikes were not just occurring more often, theywere also getting bigger. Large volume spikes, those meeting Upwell’s high spike threshold, sawa 475% increase. 7
  8. 8. Overfishing 14000 14000 12000 12000 10000 10000 8000 8000 6000 6000 4000 4000 2000 2000 0 0 Oct. 17, 2011 Nov. 17, 2011 Dec. 17, 2011 Jan. 17, 2012 Oct. 1, 2012 Nov. 1, 2012 Dec. 1, 2012 Jan. 1, 2013 Baseline Spike Threshold High Spike Threshold Overfishing Baseline Spike Threshold High Spike Threshold OverfishingSide-by-side comparison for Winter 2011 (left) and Winter 2012 (right) showing social mentions byday for Upwell’s Overfishing keyword group, as compared to the baseline, spike threshold and highspike threshold (Winter 2011: 10/17/2011 - 1/31/12; Winter 2012: 10/1/2012 - 1/29/13)In Winter 2011 (above left), when Upwell began Big Listening in Overfishing, social mentionvolume was an average of 423 mentions per day. By Winter 2012 (above right), Overfishingsocial mention volume is up 71%. Overfishing spike frequency increased 784%. Those spikeswere not just occurring more often, they were also getting bigger. Large volume spikes, thosemeeting Upwell’s high spike threshold, also saw a similar 475% increase.Annotated campaign graphs are included in this report, and illustrate more specifically whereand how Upwell intervened in these two conversations. The Overfishing Conversation The Sustainable Seafood Conversation Winter 2012 Winter 2012 Upwell Campaign and Social Mention Spikes Oct 2012- Jan 2013 ! Upwell Campaign and Social Mention Spikes Oct 2012- Jan 2013 ! Gangnam Style, CA MPAs, Fish Tornado 1400 14000 NU-20 NU-24 Vote4 1200 Ocean Video 12000 Antarctic Ocean (day 10) & I Oyster NY NU-22 Cuomo Pacific Bluefin Oysters NY the 96.4% 1000 Big Blue 10000 NMS 40th & NYT Trawling FAD Safeway NU-21 Blogs NU-24 NU-5 NU-19 800 NU-23 8000 Vote4the Ocean How to Kill a Great White Seamounts Cuomo JAWS vs & Rooftops Oysters NY Sinatra Costa Rica Big Blue 600 6000 Fin Ban Blogs 4000 400 2000 200 Antarctic Antartic (day 1 of 15) (day 15) 0 0 Oct-12 Nov-12 Dec-12 Jan-13 Oct-12 Nov-12 Dec-12 Jan-13 Baseline Spike Threshold Mean +1.0 STDEV Sustainable Seafood SS Baseline Spike Threshold Mean +1 STDEV Overfishing OFThe Tide Report, Upwell’s blog and social media channels, topic-specific webinars, plus staffspeaking engagements, guest blog posts and project consulting have provided channels fordelivering shareable content, and practical training and tools to a diverse audience of time- 8
  9. 9. starved ocean activists. According to our February 2013 survey, through these tools andopportunities, Upwell has helped the community: • Receive content that they wouldn’t come across through their usual channels • Stay up-to-date on the hottest ocean news • Save time by providing content that they could amplify to their community • Made them feel like they’re part of a community • Helped them balance humor with serious issues in their communicationsWhich ocean topics have the most Baselinevolume? 3000 90000 2500 80000 70000 2000 60000 50000 1500 40000 30000 1000 20000 10000 500 0 Sun Mon Tue Wed Thu Fri Sat 0 Sun Mon Tue Wed Thu Fri Sat MPAs Ocean Acidification Sustainable Seafood Sun Mon Tue Wed Thu Fri Sat Overfishing Gulf of Mexico Tuna Sharks Cetaceans OceanSocial mention Baselines for Upwell’s primary ocean topicsPerhaps not surprisingly, when we look at overall levels of conversational Baselines, the generic“oceans” conversation is orders of magnitude larger than the conversations for its constituentcomponents. While to some extent this is the result of so many conversations being conductedunder the “oceans” banner, the word “ocean” is itself so widely used that, without properfiltering, those other uses can distort the apparent size of the discussion. The next two largest ofour topics, Cetaceans and Sharks, also demonstrate comparatively high Baselines when assessedagainst the others. We can see substantial differences among our lowest-volume topics. MPAshas the lowest Baseline, Ocean Acidification and Sustainable Seafood are basically tied forsecond-lowest (each exceeds the for certain days of the week), and Overfishing comes in at 9
  10. 10. about five times higher than them.Collectively, overfishing represents a grab bag of ocean brands. The Overfishing conversationbrings together species such as sharks, tuna, salmon and lesser known but equally importantfish, with wonkish report subjects such as fisheries management and lackluster internationalconferences. The topic encompasses a relatively broad conversational area, and one that hashistorically churned out quarterly bursts of dire news.Overfishing has about five times the Baseline volume of Sustainable Seafood, and roughly two-thirds that of our next biggest topic, the Gulf of Mexico. The comparison with SustainableSeafood is particularly interesting because the two topics are obviously intricately connected—the difference is how people talk about them. Whereas sustainable seafood suffers from afragmented and cloudy brand identity (what is sustainable seafood, anyway?), overfishing hascharismatic ocean species such as sharks and bluefin who are in clear and present danger.Danger is catnip to the internet. The Overfishing conversation actually benefits, from anattention point of view, from the ongoing damage that we are doing to our oceans and fisheries.Bad news spikes high and fast online and then it goes away. Intriguingly, the spikes withinOverfishing have been occurring more frequently as Upwell has been monitoring (andcampaigning on) the topic. Overfishing is becoming more spikey and the spikes are increasing involume.The Sustainable Seafood conversation is low-volume with low-level spikes, even while theconcept is becoming increasingly well-established in consumer minds. For comparison, MarineProtected Areas has a lower baseline than Sustainable Seafood but occasionally spikes higherthan the Sustainable Seafood max. Ocean Acidification displays the same characteristic. Anddespite their obvious connections, the volume of the Sustainable Seafood conversation is onlyone fifth of that of the Overfishing conversation. Good news for fisheries and consumers, it turnsout, is not as attention-generating as bad news.The overall brand of Sustainable Seafood is fragmented, awkward and wonky. People simply donot talk about the sustainable seafood that they ate last night, or, crucially, not in those terms.The food service industry has recognized this: one trade publication forecasts growing demandfor sustainable seafood even as it pointed out that consumers prefer the term “wild”—whichobviously means something very different. Furthermore, “sustainable seafood” itself is not aterm well-suited for short-form platforms like Twitter—it takes too many characters and is hardto use in a sentence that doesn’t read as dry. Taken as a whole, the fragmentation of theSustainable Seafood conversation means that it is more difficult to accurately capture itaccurately with keywords, and that a low volume doesn’t necessarily mean people aren’t talking.Unlike Overfishing, which has regular media hooks through connections to Shark Week, direreport releases and celebrity activists, the Sustainable Seafood conversation doesn’t generallytranslate into spikes from live events and or big news stories. Where we do see spikes occur they 10
  11. 11. are usually based in one of three elements: well-known brands promoting their sustainableofferings (Safeway, McDonalds), fraud, or a bridge campaign (many of them attributable toUpwell). One other notable burst of attention can be expected from the Sustainable SeafoodSummit—although the resulting content hasn’t been particularly shareable.A comfort with complexity is necessary to forecast weather. Big Listening, similarly, requiressignificant human skill and intuition to, first, develop robust conversational descriptors(keywords) and then, second, to use the resulting information to identify opportunities for acampaign to spike a given conversation. Upwell has intentionally cross-trained campaign andlistening roles so that this integration between listening and intervention is as efficient aspossible. This comes not from any computer readout but from regular, hands-on practice.‘Weather’ forecasting of the social web is a nascent practice. Regular Big Listening to a givenconversation is essential for building an analyst’s awareness of the conversational dynamics atplay. It is most efficient to listen on an ongoing basis. Presence in the conversation is thedifference between watching a baseball game and reconstructing it through the box score.The structure of Upwell intentionally underpins the process for doing Big Listening. Eachmember of Upwell draws on a variety of tools and practices—some shared, some personalized—to generate immediately actionable insight into each day’s online events. We supplement ourpersonal suite of tools and practices with shared Upwell systems (such as Radian6).    Personal Listening Systems [human and machine-assisted] + Shared Listening Systems [machine-assisted and human-network-assisted] + Morning Meeting [humans in conversation] = Big ListeningWhile the context for Big Listening is constantly shifting, we believe that current trends point tosome likely future developments. These include: • New firehoses • Divergent functions • Smarter robots • Privacy fights • Buyer beware • Social science catches up to social media • More visuals • Spike marketplaces • More upwellings 11
  12. 12. Emergent best practices for online campaigning from the Upwell pilot include: • You can’t predict what will go viral. • Timeliness and a hook are still really important, but the half-life of news online is shorter than it used to be. Pay attention to ROI on campaigns. • Bridge conversations, movements and communities to make your message go farther. • Identify opportunities based on Big Listening. • Use simple messaging. • Think about the whole viewing and sharing experience. • Narrow in. • Be poised for rapid response. • Pair content with asks, but balance asks across a spectrum of engagement. • Celebrate victories. • Normalize obscure issues or complex ideas with iconic imagery, cultural anchors, or tribe signifiers. • Define your goals and metrics based on what is actually measurable. • Revive old stuff. • Videos: shorter, prettier, more pithy. • Memes: don’t try to make them from scratch. • Celebrity promotion: not a silver bullet.Collaboration in communications is hard, and can be expensive. Emergent best practices forCollaboration, the Distributed Network Way from the Upwell pilot include: • Provide brand neutral content. • Embrace the larger ecosystem of communicators. • Be open to ad hoc partnerships. • Share other organizations’ and people’s content. • Find unique high-touch activities to cultivate personal relationships. • In difficult times, be human.In running rapid attention campaigns, and focusing primarily on social platforms as the mediumfor our ocean famous-making, Upwell has developed a few best practices that can be applied toother small, nimble online teams. • Develop systems to capture insights. 12
  13. 13. • Encourage a flat structure. • Keep the campaigning team small, but not too small. • Keep time for developing creative assets to a minimum. • Run lots of little campaigns, and extend the ones that work. • Lean on the personal interests, strengths and networks of your team members. • Recognize and admit your weaknesses.Top insights and best practices for amplifying attention to ocean issues in general, as well assome that are specific to those communicating about overfishing, sustainable seafood, andmarine protected areas from the Upwell pilot include:For ocean communications: • The ocean is out of sight and out of mind. • We assumed there would be a lot of great ocean content. We were wrong about the ‘great’ part. • Plan social media outreach in advance of scientific report releases. • Lower your science hackles. • Cross-promote social content via collaborative outlets. • Anthropomorphize ocean creatures. • Don’t let beautiful ocean pictures do all the talking.Sustainable Seafood: • Scary stories get attention. • “It’s complicated” is a bad relationship status and a bad brand. • The actual practice of eating sustainable seafood continues to be challenging, and news coverage is not making it appear easier. • Focus on specific products, brands and species rather than the overall sustainable seafood issue. • Recipes and fluff pieces don’t generate social mentions.Overfishing: • Focus on actions that are doable and close to home. • Sensational stories make headlines. • Sharks are the quarterback of overfishing, and Shark Week is the Super Bowl of online ocean conversations. Don’t sleep on Shark Week. 13
  14. 14. Marine Protected Areas: • The MPA conversation is tiny in comparison to other ocean conversations. • Our MPA vocabulary is fragmented, awkward and wonky. • Share successes. • Emphasize individual connection to MPAs as public commons to create support.This is the final report of Upwell’s pilot phase, completed in February 2012. In it, the foundingteam of Upwell documents new methodologies for conversation analysis, the shape of keyocean conversations, the impacts of our campaign efforts, and emerging best practices for a newera of online communications. We do so in service of the larger marine conservation sector, andwith the hope that what we have learned in our short effort will speed all our collective efforts.The ocean is our client. 14
  15. 15. IntroductionOcean Conservancy and the Waitt Foundation collaboratively developed the Upwell project in2010. The project’s goal was to increase the volume of the conversation about the ocean toenhance awareness and support for ocean issues among mass audiences. Ocean Conservancyinitially envisioned an 18-24 month pilot phase for the project, with the incubation stageconcluding in the summer of 2013. We conducted a national search for the project’s leadership,hiring Rachel Weidinger, and launched the fully staffed program in early 2012. During our pilotphase, the Upwell team has enjoyed the contributions of a great number of excellent crewmembers, including Kieran Mulvaney, Ray Dearborn, Matt Fitzgerald, Saray Dugas, Britt Bravo,Lara Franklin, Aaron Muszalski and Kevin Zelnio, and interns Christine Danner, Paulina Dao,Liana Wong, and Kaori Ogawa.  We’re grateful for the support of the Waitt Foundation, ananonymous donor, the Gordon & Betty Moore Foundation, and our colleagues at the OceanConservancy including Janis Jones, Melissa Ehrenreich, Shannon Crownover, Amelia Montjoy,Julia Roberson and George Leonard. Vicki Spruill has been an important mentor for the project.This experimental pilot project charted new territory to engage a larger and more diverseaudience in the ocean conversation and to elevate the ocean while not elevating any particularorganization or perspective. We have done this by quantifying the level of the oceanconversation across a range of topics and measuring the impact of engagement on the issue, afirst for the strategic ocean communications initiatives. The Waitt Foundation served as OceanConservancy’s lead partner to help shape the direction, finance the use of new cutting-edgetechnological tools to actively monitor online conversations, and develop aggressive rapidresponse campaigns to reach and mobilize new audiences to care about ocean content. At thebehest of the Foundation, Upwell focused primarily on elevating the online conversations aboutoverfishing and sustainable seafood during this incubation period to test the efficacy of thisinnovative approach. We have had other forward-thinking funders join us in support of thisproject over the past two years.During its first year of incubation, Upwell successfully pioneered the development of newmethodologies in social monitoring, demonstrated success in elevating the ocean conversationabove the baseline, earned praise for its non-branded approach to campaigning from socialmedia thought leaders and attracted additional philanthropic interest in expanding the projectbeyond the intent of the pilot phase across a range of environmental issues. We are grateful forthe Waitt Foundation’s significant initial investment, which provided the vision andcommitment to launch this entrepreneurial initiative and are appreciative of other funding wehave received for the project.This is the final report of Upwell’s pilot phase, completed in February 2012. 15
  16. 16. 16
  17. 17. Theory of Change andContext for Our WorkThe ocean is in crisis, plagued by overfishing, habitat loss, and acidification, among other issues.While the ocean serves as the engine for our climate and plays a central role in the global foodsystem, it still fails to register for many as a relevant and primary issue. It is, quite literally, out ofsight and out of mind. The virtual invisibility of the ocean in public discourse is a major obstaclefor the ocean conservation community to adopting and implementing solution-based policies.The genesis of Upwell was rooted in a need, identified and articulated by Ted Waitt jointly withVikki Spruill of Ocean Conservancy in late 2010, to build an “ocean war room” whose core focuswas to increase attention to ocean issues among new and mainstream audiences.  The effortwould be a non-branded communications effort that would utilize new and traditional media tobuild a fast, aggressive and agile strategic communications platform to increase attention toocean conservation issues in real-time.Although the specific look, feel and direction had yet to be determined, some key elementsarticulated in the earliest stages remain true, over two years later: • It would be an informational effort, one that would simultaneously seek to raise the volume of key ocean issues while elevating them above the growing cacophony of background noise on the Internet and elsewhere. • The effort would be, in a sense, ‘unbranded.’ It would not act as a competing entity in ocean conservation, but would instead highlight the work in conservation and science already being done by others. As was expressed often in internal deliberative conversations in the project’s earliest stages, “people within the community should ideally be fully aware we exist, and that we are a resource to be utilized; but ideally, people in the street will never know of our existence, but simply be more aware of the ocean than they were before.” For that reason, the project was initially referred to internally as “Ocean Underground.” • It needed to be fast, ready to respond to and amplify developments and news at a moment’s notice. 17
  18. 18. Testing the WatersWith the loose ideas of direction (detailed above) in mind, we reached out to members of thecommunity to gauge their levels of interest, and understand where we could be the mosteffective. We spoke with many different players in the diverse ecosystem of oceancommunications: researcher/bloggers (e.g., John Bruno of the University of North Carolina),social media experts at NGOs with a particular focus on online mobilization (e.g., Ben Kroetz atGreenpeace), scientist-communicators (e.g., Nancy Knowlton, Jeremy Jackson and StevePalumbi) and many more.We shared with them these still-nascent goals and philosophy: We aim to raise the volume of ocean messaging, by utilizing the huge variety of outlets now available to maintain a constant drumbeat of news and information. We will be doing more than tweeting, blogging, and linking to every ocean story we come across. We will also be providing context, emphasizing issues and topics of particular import and helping ensure an understanding of the way they link to each other. In this way, we hope to raise the volume without merely contributing to the overall background noise. We will not be competing for the limelight. We will not be competing for funding. We do not intend to be another “brand” in the public eye. We shall we be a resource, a means of highlighting, synthesizing and contextualizing ocean issues in a way that brings further attention to those issues and to those who are researching and campaigning on them. We will seek to operate on a multitude of levels. We will work with researchers, organizations and others to produce original content for publication online or in print. We will aim to work with existing outlets and to create our own. We will offer to work with, and provide content for, outlets that need it and also to disseminate content for those who lack such networks. We will be content aggregators, synthesizers, distributors, creators, and magnifiers. [...] We hope that once we become established, we’ll swiftly settle into a pattern, in which it will become routine for members of the community to provide us with content that we can aggregate, magnify and synthesize, and that as a result ocean issues will have no problem being heard in the cacophony of the Internet.Initial feedback was incredibly positive. Members of the community agreed that there was aneed for deeper understanding of the currents of ocean conversation, and a need for a new wayto harness the energy and content and funnel it into more effective campaigning. 18
  19. 19. A Social FocusWe decided early on during this scoping process that social media was going to be our playingfield. Campaigns in the digital age—at all levels—require the rapid communication and personalconnections that social media cultivates. We would focus on unique metrics to evaluate oursuccess in penetrating and motivating social networks to spread ocean content within their peernetworks to both boost the volume of online conversations focused on the ocean and tobroaden the conversation beyond the choir. We would also respect and leverage the power oftraditional media by helping to connect social content in ways that would create mainstreammedia attention or extend shelf life. We would strive to make scientific research accessible topopular audiences online and identify relevance to new social and mainstream audiences. As wearticulated in early documents: “As an overarching goal, we have been tasked to increase the volume of ocean news, information and issues. ...Where Ocean Underground can carve out a unique and invaluable role is, as noted at the top of this section, cutting through that noise, highlighting the news that is important, discarding that which is less so, and providing a greater context for that news.”The articulation of this goal responded to what Eli Pariser—formerly of MoveOn.org and nowfounder and CEO of Upworthy—calls the “Filter Bubble”: an increased personalization of theinternet. Search engines, social media platforms and online news outlets apply highlysophisticated algorithms to analyze our internet behaviors and customize our experience,necessarily limiting our exposure to new ideas that might be critical to progressive socialdiscourse. We wanted to explore ways to circumvent and manipulate filter bubbles to broadenthe ocean conversation beyond the choir.We also wanted to experiment with ways to modernize conservation communications, aligningthem with the language and speed of the internet. By monitoring spiking attention and onlineconversation in real-time, we would quickly intervene in a conversation and inject oceancontent into popular conversations. By using humor, wit, hard-hitting facts and education andrecontextualization, we would create highly shareable content tailored to spread through socialspaces.The Need for a Big Team OceanIn our initial conversations, we learned quickly that many people in the ocean world felt theyoperated in silos, focused on one topic, region, or issue. Competition in the marine conservationspace is real—“blue” organizations get only a small fraction of environmental funding. Indeed,several of those with whom we spoke showed particular interest in and enthusiasm for our 19
  20. 20. mission when they fully appreciated that we would not be a part of that competition and wouldbe trying to find ways to make their work more effective. Our philosophy at Upwell is that we’repart of a big Team Ocean that includes marine conservation organizations, marine scientists andocean activists. Our competition is Justin Bieber, not each other. While Team Ocean is anythingbut small, most activists, researchers and free agents fly the ocean flag far below their own,specialized flag on the pole. It was clear that we had to invent a new way to collaborate.A primary way that we’d cut through the noise and make valuable content reach broaderaudiences was to foster a bigger and more diverse network of ocean communicators. In ourearly analyses of ocean conversations, we saw what nonprofit social media expert and trainerBeth Kanter calls the rise of “free agents,” and a shift away from traditional nonprofit “fortress”communications. Fortress institutions, Kanter asserts, “work hard to keep their communitiesand constituents at a distance, pushing out messages and dictating strategy rather than listeningor building relationships.”Welcome to the Fortress. Now please go away. Photo by Stuck in CustomsIncreasingly, environmental NGOs—the fortresses—were not driving conversation. Free agents—bloggers at Deep Sea News, managers of Facebook pages like I Fucking Love Science, and socialmedia savvy public figures like George Takei, to name a few—were generating conversation bysharing irreverent content and engaging their followers in a more personal way. Citizen-ledefforts utilizing social media, like the Arab Spring and the Occupy movement, are examples ofthe power of decentralized communication.We wanted to bridge and engage both these communities, helping organizations to chip away attheir fortress walls, and connecting free agents with the deep, science-based content that 20
  21. 21. organizations and academics could provide. Some organizations outside the ocean sector arebeginning to experiment with this. Greenpeace elevates examples of people-poweredorganizing through its Digital Mobilisation Lab, and in December 2012, MoveOn.org embarked onwhat they call a “bottom-up revolution,” empowering its supporters to determine what issuesand campaigns MoveOn throws its weight behind. Aside from polling members on issues towork on, MoveOn also has a platform that allows members to upload or share content that canbubble up into campaigns.The key to our success—and thus, the success of the ocean conservation community—wouldnot be to blast new, shiny information into the interwebs, but rather to nurture and bridgevirtual and real-life distributed, diverse networks, and to leverage the combined reach andpower of those networks of communicators to participate in and amplify the best content andcampaigns. In inventing a new kind of collaboration, we’d provide the tools and the space, andrely on the ever-growing community of ocean communicators to work together to makechange.The Very Large Array in New Mexico harnesses a network of radio telescopes to increase itslistening power. Upwell does the same. But with social media networks rather than radiotelescopes. 21
  22. 22. We Shall Have Bigger Ears and EyesInto the InternetIf Upwell was going to operate on behalf of Team Ocean, we were going to need a way toidentify its members and assess our collective efforts. We needed a big picture perspective onthe ocean online. How many people were on our team? What were they talking about? Were wegetting our butt kicked, attention-wise, by I Can Haz Cheezburger? We needed to understand thevolume and character of online conversations about the ocean.LOLrus: playing for both Team Ocean and Team Cheezburger [source]Upwell recognized that the broadcast model of communications was insufficient for anetworked world in which attention and engagement are the primary currency. But we alsorecognized that if online attention was a currency, the ocean was basically broke. Despitewidespread love for the actual thing, the ocean as represented online was a shadow of itself. Goto the beach and the ocean was captivating. Go on Facebook and it was hard to find at all.We began to develop what would become Big Listening, a methodology and philosophy, oflistening to dynamically evolving online conversations writ large. Our primary lens for assessingsuccess would be whether or not our shared purpose succeeded, not whether our organizationdid. We would eschew the brand constraints that had crippled the ocean’s institutional voicesonline and we would speak fluent internet. We would do everything we could to make theocean more famous on the internet, and we would use Big Listening to measure our progress. 22
  23. 23. Step change vs. Incremental Change [source]Upwell entered this challenge looking for step change—a massive, discontinuous leap forward—because the ocean needs a win that really matters. Since those initial developments, BigListening has gone from an abstract concept to a replicable, demonstrated methodology. Our bigwindow on Team Ocean has also had the fortuitous effect of developing new campaigningtechniques for which the ocean sector now has a competitive advantage. That advantage won’tlast forever.Conditioning the Climate forChangeAs Kari Marie Norgaard notes in her book Living in Denial1 : Before an issue can make it into a council meeting, onto picket signs, into the framing of a local news story, or into a newspaper editorial, somebody has to start talking about it. ... Conversation is the site for exchange of information and ideas, for human contact, and for the building of community.Upwell’s array of goals—to utilize the immediacy of online communications, experiment withways to increase the reach of valuable content, empower and foster a broader network of oceancommunicators, and enrich our understanding of the conversational ecosystem surroundingocean topics coalesced into our broader vision of “conditioning the climate for change.” Webelieve that by getting more people talking about ocean issues and raising the baseline ofconversation, broader audiences will be more likely to take action, change behavior, and pushfor policy change that will have positive effects for our oceans.1 Norgaard, K. M. Living in Denial. 2011, Kindle Edition, location 809. 23
  24. 24. Metrics: Social MentionsOur primary metric for understanding the conversations analyzed in this report is what we referto as a “social mention” (or “social item”). Upwell defines a social mention as the text inclusionof a monitored keyword in a post on a social media platform like Twitter, Facebook, a blog,mainstream news with an RSS feed, a forum/board, YouTube or Pinterest. Social mentions areonline acts of self-expression in which individuals, organizations and other entities invest (atleast) a small amount of social capital. Social mentions have more in common with the Other Metrics Social Mentions metric of media hits than they do with the more (not social mentions) common, older PR and marketing metric of Tweets and retweets Impressions impressions. Upwell focuses on counting and analyzing social mentions (rather than Mainstream news articles with RSS Views impressions or online mentions) because we feeds and comments believe that the number of people who choose to take an action to create or share content is a Posts, shares and better indicator of engagement than the number comments on Clicks of people who have simply seen (or could have Facebook seen) that content. Likes / Loves / Favs Blog posts and [Facebook, Tumblr, It is worth noting that, while it is theoretically comments Twitter] possible to accurately count every single social Re-blogs on tumblr mention on a topic, Upwell’s Big Listening methodology focuses on characterizing Forum or board posts conversations just thoroughly enough to campaign successfully within them.Furthermore, Upwell believes that social mentions are abetter leading indicator of willingness to take action for the What About “Likes”?oceans than other communications metrics. This is because Likes, loves, and faves (differentsocial mentions represent actions, the choice of an terminology for different socialindividual to risk a small amount of social capital by media platforms) are in a middleassociating their online identity with a piece of online ground. While they are not socialcontent. In aggregate, the volume of social mentions not mentions (as people are notonly represents the amount of attention being paid to a creating new content), they aretopic, but a forecast of potential campaign success. also not as passive as views or impressions. While likes, loves,The strength of a community, by our standards, is measured and faves are not counted bynot by its size, but rather by its engagement level. For Radian6, Upwell does measureexample, if one tweet has 12,000 impressions (the number them, when possible. However,of people who follow the account that posted the tweet), for the purposes of this report wewe count the tweet the same way that we would count a have omitted these metrics sincetweet with 200 impressions. If a person or organization is they constitute only minimalnetwork-oriented, it would follow that their content would public engagement and can require laborious, resource-lead to more retweets, replies and/or mentions. If a tweet intensive manual calculation.goes out to 12,000 followers but gets zero retweets, it is lessof an indicator of willingness to take action than a tweetthat goes out to 200 followers and gets 10 retweets. 24
  25. 25. Methods: Big ListeningIntroductionUpwell employs Big Listening in order to understand the volume and character of onlineconversations about ocean issues.In our work to date, the team at Upwell has come to believe that there are three measurablecharacteristics of the online ocean conversation. We are increasingly attentive to: 1. Constant level of conversational volume (Baseline), 2. Notable outliers in increased volume (spikes), and 3. Density of conversational hotspots (spike frequency).Over time, we want to come to understand the role of all three as they contribute toconditioning the climate for change. This section details the current state and maturity ofUpwell’s Big Listening practices, including our Baseline methodology and spike quantificationmethodology.What is Big Listening?“Take things as they are. Punch when you have to punch. Kick when you have to kick.”- Bruce Lee As more and more of our conversation moves online, the potential of big data to help advocacy organizations understand the environment for their works also increases. To seize this opportunity fully requires setting aside preconceptions and engaging with the world as it is, right now, not as it was assumed to be nine months prior in a grant proposal. In the words of Bruce Lee, you have to “take things as they are,” and use immediate insights to inform your actions.Bruce Lee, strategic opportunist Since November 2011, Upwell has been monitoringthe online ocean conversation on a daily basis to identify opportunities to use our distributednetwork for online campaigning. We listen to eight primary ocean topics: Overfishing,Sustainable Seafood, Marine Protected Areas (MPAs), Oceans, Cetaceans (whales and dolphins), 25
  26. 26. Sharks, Tuna, Gulf of Mexico and Ocean Acidification. Our method of conversational analysis hasbeen called Big Listening, first by Micah Sifry of Personal Democracy Forum, and later bynonprofit social media expert Beth Kanter.Typically, Upwell has used Big Listening to inform campaigns that are then implemented acrossour distributed network of evangelists, influencers and social media managers in order to spreadthe marine conservation conversation beyond the "ocean sector" (beyond ocean conservationorganizations and marine scientists). During and after our campaigns, we use the same BigListening methodology to measure how many social mentions (e.g., tweets, Facebook posts,blog posts) happened in real time.Big Listening is the art and practice of tracking topical online conversations over time—listening to what “the internet,” writ large, is talking about. When combined with data-informedcampaigning, Big Listening provides a methodology for increasing both the frequency andvolume of online conversation around a particular issue. The basic idea is to identify pockets ofreal-time or historical conversation, wherever they may be, and then to use that information tomake the conversation grow bigger. Big Listening is distinguished from traditional social mediamonitoring by its scale, fluidity, focus on issue or cause monitoring, and expanded access tohistorical data.We are not alone in innovating in online conversation. Big corporations, their brands and themilitary are all attempting to make sense of the new networked landscape, whether we realizeit’s there or not. From Target predicting a teenage girl was pregnant through her purchasepattern, or the Obama campaign stitching together millions of voter records with proprietaryconsumer datasets, this cloud-based, networked, indexable world is here to stay. At Nestle, the largest food company in the world, they have gotten the message. After reaching the peak (or perhaps the trough) of social media mismanagement during a Greenpeace campaign that targeted the use of palm oil in Kit Kats, the company dramatically ramped up their online listening through their Digital Acceleration Team. As profiled in a recent Reuters story, the team operates out of a social media war room. Radian6 widgets gleam on wall-mounted flat screen monitors as employees fight for the reputation of,A view inside Nestle’s Digital Acceleration Team among other Nestle products, the plasticheadquarters [original source] water bottle industry.22 http:/ /uk.reuters.com/article/2012/10/26/uk-nestle-online-water-idUKBRE89P07Q20121026 26
  27. 27. Our war room is a little different. Instead of monitoring a corporate brand or a product, wemonitor the brand of the ocean, focusing on sustainable seafood and overfishing. WhereasNestle’s listening topics are comparatively static and focused on their company properties, oursflow and evolve with the dynamic cause or movement-based conversations that we monitor.This distinction in listening, between the static product and brand conversations typified byNestle, and the shape-shifting, dynamic ocean conversations that Upwell follows, is significant,and a key distinction of Big Listening as we define and practice it. We should note that whenNestle D.A.T. members monitor the plastic water bottle conversation as a whole, rather thantheir company’s share of it, they are practicing something more similar to what we do at Upwell.Baseline MethodologyWhat is a Baseline?Upwell practices Big Listening on English-language conversations in the following eight topicareas: Overfishing, Sustainable Seafood, Marine Protected Areas (MPAs), Oceans, Cetaceans(whales and dolphins), Sharks, Tuna, Gulf of Mexico and Ocean Acidification. For each topic,both real-time and historical data provide essential context for understanding the volume,evolution and characteristics of the overall conversation.Each topic we monitor is characterized and defined by a set of search terms (includingexclusions) that we refine on an ongoing basis. While we recognize the limitations of “keywordgroups,” such as their reliance on text-based results and the absence of contextual awareness,they do provide a powerful tool for analyzing online attention. The development and activerefinement of keyword groups is at the heart of Big Listening methodology.We use Big Listening in order to: • identify and target high-value items for campaign purposes, • compare the relative size of different ocean sub issues (e.g. sharks vs. whales), and • measure the impact of our campaigns.Since Upwell is a campaign agency (among other things), we needed a way to characterize theseconversations as they exist, absent our interventions. Enter the Baseline. Baselines help us toanchor campaign performance targets in measures of past conversational volume. We set goalsinformed by the Baseline (as well as by spikes), and then campaign to meet and exceed thosetargets.Upwell informally defines a conversation’s Baseline at the point below which the daily volumedoesn’t drop. It can be thought of as a floor (although it is often quite high—in the tens ofthousands for a conversation like Cetaceans) or as the number of social mentions performed 27
  28. 28. each day by the topic’s diehard conversationalists. Ifeveryone else left the party, the Baseline would still bethere, dancing by itself.The Baseline: Up Close andPersonalUpwell’s Baseline methodology has evolved to capturethe highly dynamic conversations we watch, especiallycyclical variations by day of the week. These cyclicalvariations often result from usage and posting patterns.For example, people tend to talk on the internet when Robyn: euro popstar, solitary dancer,they’re at work. Over the course of our pilot phase, human Baseline metaphorUpwell has used three different version of Baselinemethodology to better measure the dynamic onlineconversation space: • Baseline v1.0: The lowest level of daily social mentions for a given conversation, for a given period (implemented using Upwell topical keyword groups) [in use through late August 2012] • Baseline v2.0: The median daily social mentions for a given conversation/keyword group for a given period [in use through mid-November 2012] • Baseline v3.0: The average of the lowest 10% of social mention values for a topic on a given day of the week [in use through early January 2013] • Baseline v3.1: The average of the lowest 20% of social mention values for a topic on a given day of the week [currently in use]Our Baseline quantification methodology was created with input from leaders in the fieldincluding: K. D. Payne, Chairman & founder of Salience/ KDPaine & Partners, and co-author ofthe recently released Measuring The Networked Nonprofit; leading nonprofit technologist andPackard Fellow, Beth Kanter; and a senior educational policy analyst for a leading nationalmeasurement/ social statistics firm contracted by the Gates Foundation, the Broad Foundationand the U.S. Department of Education.To calculate the Baseline for a particular topic we begin by compiling all available socialmention data for the period since we started monitoring it (mid-October 2011 or later, dependingon the topic). We then disaggregate the data by the day of the week in order to deal with cyclicalvariations in post volume and compare Mondays to Mondays, and Sundays to Sundays. Oncethat’s done we calculate the average (mean) of the lowest 20% of values for each of the sevendays. Taken together, those day-of-the-week values are what we refer to as the Baseline. 28
  29. 29. We selected the mean to establish a specific value for each day of the week for three reasons.First, the mean is the starting point for calculating standard deviations used in our spikequantification methodology. Second, given the small size of most ocean conversations, themean is the most typically consistent and available measurement when analyzing theconversation on a by day-of-the-week basis.Baseline Social Mentions by Day-of-Week for Upwell’s Sustainable Seafood keyword group(10/17/11 - 1/29/13).These daily Baseline values are then graphed against social mention data over time. The graphbelow shows the result. 29
  30. 30. Social mentions for Upwell’s Sustainable Seafood keyword group vs. Upwell’s Sustainable SeafoodBaseline, June 1, 2012 - August 1, 2012.Context and Challenges for BaselineQuantificationAs seen in the overfishing and sustainable seafood conversations, day-of-the-week periodicity ishighly evident in social mention volume, largely driven by mention increases during U.S.working hours across Eastern to Pacific time zones, and with substantial drop offs on weekends.Using the current Baseline as a reference for setting campaign goals removes the disincentivespresent in previous Baseline quantifications to campaign on lower volume days, and gives amore accurate picture of success on all campaign days.Our campaign efforts, along with unexpected conversational developments (e.g., the release of anew report or a natural disaster) require us to add new terms to the relevant keyword group sothat our work, or online mentions of a relevant, unanticipated event, is captured in our searchterms. In the inevitable cases where we find noise in the results returned by particular terms (oldand new), we respond by tightening or removing those terms from the keyword group. 30
  31. 31. Since keyword groups have keywords added and subtracted on an ongoing basis, there are someinherent challenges. How should Baseline calculations account for these changes? How andwhen should Baselines be refreshed? Should campaign targets be refreshed retrospectively?How should we treat a spike that loses or gains volume with a refreshed keyword group?Moving forward, Upwell will continue to drive innovations in our Baseline methodology and inthe integration of that methodology into our campaign process.Finding the BaselineDeveloping a Baseline for a topic or conversation requires an iterative process of definition,testing and measurement. And once you’ve got it, you have to refresh it to account forconversational evolutions. It’s like a marriage in that way, or so we’re told.Measuring something requires you to define it. The trick in measuring a conversation is thatconversations change over time as participants engage in dialogue. As conversationsdynamically evolve over time, so too do the methods of expression (i.e. terminology, imagery,metaphors, etc.), the composition of participants, and the accompanying platforms. Much likespecies evolution, these changes are not always in a direction that we perceive to be fruitful.Evolution can lead to progress just as it can lead to dead-ends or fragmentation. Because of this,no conversational “listening” can ever be exhaustive (as some elements of the conversation willalways be overlooked), nor can it be perfectly accurate (as noise will always creep in). This iswhy we continue to add and subtract terms to our Baseline keyword groups. Ongoingmonitoring and modification of a Baseline keyword group is the most effective way to keep thatkeyword group refreshed and accurate. Since designing a keyword is as much about selectiveaddition as selective omission, our ability to effectively evolve keywords and keyword groups isinformed just as much by our listening practice with Radian6 as it is by our personal listening,professional network and subject matter expertise. No robot can do this, although some aretrying, and interns probably can’t do it either.In “baselining” a conversation, Upwell begins by developing a conceptual framework for thetopic in question. For the purposes of explanation, let’s imagine our topic is marine debris. Wewould begin our Baseline development process by outlining the conceptual and temporalboundaries of analysis for marine debris. The temporal aspect is important because a keywordgroup developed for one time period may lose significant accuracy (and utility) when applied toanother period. For the conceptual outlines we often make use of a mind map such as the onefor Oceans shown below. 31
  32. 32. Upwell mind map for Oceans keywords, January 2012For marine debris the concept map would include items such as marine trash, the pacific gyre,marine plastics, great pacific garbage patch, seaplex (minus exclusions for the botanicalshampoo of the same name), albatross AND plastic. The concept map would also include people,campaigns, expeditions and organizations such as Miriam Goldstein (a marine debris expert),The Trash Free Seas Alliance, the Plastiki, and Seaplex.The concept map becomes a design artifact for further conversations about the conversation.Although we sometimes shortcut this process in the interests of time, we refine the mapthrough a series of discussions and email exchanges with subject matter experts andknowledgeable people in the industry or industries at play.Once we have a solid map of the conversation, we turn the map into a series of keywords.Keywords are textual search terms, much like something you might google. A keyword forUpwell, for example, would be “Upwell.” It might also be a distinctive phrase (or fragmentthereof) such as our tagline, “the ocean is our client.” A keyword, in this way, can actually be anumber of elements (such as multiple words in a phrase) despite its singular form. This fact willbecome more important as we discuss more of those elements. A collection of keywords iscalled a keyword group, and search results for keyword groups are the foundational output ofBig Listening.A couple wrinkles make the construction of keywords and keyword groups significantly morechallenging than one might expect. The first is noise. When you type something into a Googlesearch bar and click on a result, you are deploying a potent combination of Google’s massive 32
  33. 33. computational power, billions of dollars in said company’s algorithmic investments, and theconcentrated smarts of your own interpretive brainpower. The last element is particularlyimportant. Whereas Google displays dozens, if not thousands of search results for you to choosefrom—and then asks you to filter those results—the keyword queries we construct for BigListening must be built so as to filter out as much noise as possible. To return to friend-of-Upwell Miriam Goldstein, mentioned above for her marine debris expertise, a well-constructedkeyword group for that subject would probably not include her name as a standalone keyword—the reason being that she talks about other, non-marine-debris subjects as well. Entering“Miriam Goldstein” as a keyword nets you any mention of her full name, whether that takes theshape of a blog post about ocean trash or a friend’s tweet referencing her attendance at aparticularly yummy brunch meeting. What you leave out of a keyword or keyword group is asimportant as what you put in.Pruning out extraneous results through proper keyword construction brings us to the secondwrinkle: exclusions. Exclusions are also textual search terms, but their purpose is to filter outresults that match their terms. Exclusions can be tied to specific keywords or to entire keywordgroups. A well-constructed (or scoped) exclusion can be the difference between finding onlinementions of sharks, the creatures, or finding online mentions of the San Jose Sharks, the hockeyteam (creatures of a different sort). Exclusions can filter out things beyond keywords (such asentire categories of website domains, particular geographies of origin, or what a computerdetermines to be a particular language) using a variety of tools used in Big Listening—sourcefilters, to give one example.Another wrinkle in keyword construction is proximity. Proximity is not available in every toolthat might be applied in a Big Listening process but it is present in Radian6, our tool of choice atthe moment. Proximity is a modifier that can be applied to two or more words in a keyword, say“marine” and “debris.” Proximity, denoted by “~”, tells the tool/service how close a set of wordsmust be in order to return a match. Closeness basically means: how many other words come inbetween? If we were to set proximity to zero for “marine debris,” Radian6 would return onlyitems that include that exact phrase. If we set proximity to three, for example, we might getresults such as “marine layer clotted with debris.” That distinction becomes increasinglyimportant when Radian6 scrapes long forum discussions, news articles or blog posts in which atopic might be mentioned in an extremely peripheral manner. Proximity provides another toolto scope a given keyword and focus the results in a particular way.Keyword development feeds into an ongoing measurement process of Scope / Test / Adapt /Share. The cycle is presented below. 33
  34. 34. SCOPEInitial investigation • Outline the conceptual and temporal boundaries of analysis for the topic • In consultation with subject-matter experts and other stakeholders, create a seed list of topics, subtopics, potential online influencers, and known online sources, events, and campaignsTESTGenerating preliminary keywords • Use the seed list to develop initial keyword inputs for online search and social media monitoring services • Develop a more detailed set of keywords • Verify keyword accuracy and relevance using Radian6 to graph and spot-check search results, adding exclusion terms to filter extraneous results/noise, or various degrees of proximity to widen the net 34
  35. 35. ADAPTRefining keyword groups • Share keyword lists with key informants (subject experts, foundation staff, campaigners etc.) and incorporate feedback (e.g., additional terms, scope adjustments) • Repeat steps 3, 4 and 5 with updated keywordsCultivating and maintaining keywords • Campaign or otherwise monitor keyword group results on an ongoing basis • Update Baseline keyword groups with new inclusions and exclusions based on current events, campaigns and other developments (while always testing for the introduction of noise, per step 5) • On an as-needed basis, generally after at least three months of listening, share Baseline keyword groups with subject matter experts and other groups to gather feedback and potential improvementsSHAREExporting and preparing data • On a monthly, quarterly or to-order basis, export Big Listening data based on the most current keyword groups • Recalculate Baseline values • Graph and annotate charts with spike identificationsPackaging and distributing insights • Create reports, blog posts and other types of synthesis for external audiencesImproving the methodology • Gather feedback and process what we’ve discovered • Iterate our overall set of proceduresA crucial detail of the final stage of this process is the fact that exporting the data freezes it intime. To offer a contemporary example, all of the data in this report was current as of the end ofJanuary 2013, and then it was frozen in a spreadsheet. It’s important to remember that theconversations we monitor continue to change, even as we’re measuring and reporting on them.Because the exported data is a snapshot of results for a particular conversation’s keyword group, 35
  36. 36. as it existed at a particular time (of export), from a particular tool (or combination of tools), theresulting values cannot and should not be separated from the keyword group that producedthem. Furthermore, due to the item volume returned by some of the larger keyword groups,exporting data will sometimes produce variations in measurements for the same hour, day ortime period. This variation is due to the tools we use and is generally extremely small given thescale of the topics we’re monitoring. These two factors combine to reinforce our belief that BigListening data can only be fully interpreted if the underlying keywords are available—anythingless is a black box.Spike QuantificationWhat is a Spike?A spike is a significant increase in online attention for a particular topic. When you graph thosesocial mentions, you can actually see that burst of attention ‘spike’ the graph—hence the name.We have been observing spikes in the wild, so to speak, since the beginning of Upwell. It’s aconcept that is at least somewhat familiar to anyone who has ever described a video as “viral,”or checked out the list of the most shared articles on the New York Times website. A lot ofpeople sharing one thing over a short time creates a spike. In the world of Big Listening, that onething they share can actually be a large number of different things on the same topic, but thegeneral point remains the same. Surges in attention create spikes. So how do you measure one?Let’s revisit that graph of the Sustainable Seafood keyword group that we looked at earlier. 36
  37. 37. Social mentions for Upwell’s Sustainable Seafood keyword group vs. Upwell’s Sustainable SeafoodBaseline, June 1, 2012 - August 1, 2012.It seems pretty clear that there are two spikes in this time period. One appears on June 8, theother on June 16. But what about the other days? How far above the Baseline does socialmention volume have to be in order to qualify as a spike? We set out to find a way to comparespikes that would answer the question.Before we dive in, it’s important to note that social mention volume for a given day is aconstruct. We decided to use a day as the operating unit of time both because the tools we haveavailable to us use that temporal distinction, and because a day as a unit of measurement iswidely understood. That is not to say that one couldn’t decide to measure spikes by the hour, bythe minute, or by some other amount of time. We made a conscious decision to build our initialdefinition of a spike around the day, but infinite other options exist as well.A second caveat is that focusing on spikes may obscure what is actually making up the long tailof post volume. Upwell talks about, and quantifies, much of this activity as the Baseline, butthere may be other small-to-medium bursts of attention that last more than a day andconsequently don’t visually ‘spike’ a graph in the same way (think of a multi-day increase in 37
  38. 38. attention as a hump or a mesa, rather than the taller, more angular spike). Spikes look good oncharts, and they help push conversations into the wider internet, but they are not the wholestory of an online topic. We long for a day when tools for Big Listening allow us to view topicvolume graphs like geologists look at cross-sections of rocks—that day is not here yet.With those caveats out of the way we can return to our earlier question: what is a spike?Remember from our discussion of Baseline quantification that Upwell’s analysis is designed toinform a set of interventionist activities. We: • identify and target high-value items to campaign on; • compare the relative size of different ocean sub issues (e.g. sharks vs. whales); and • measure the impact of our campaigns.Spike quantification informs our campaigning and provides one measure of results. We’re notinterested in just contributing to the noise around a given ocean topic, we actually want to helpa signal to emerge. Spikes are those signals. Evaluating opportunities to campaign becomes amuch more concrete activity when you know exactly how many social mentions are needed tobreak through the regular volume of conversation.After examining historical social mention volume for our Sustainable Seafood and Overfishingkeyword groups, we calculated a variety of statistical thresholds for the exported data andcompared the results to our measured campaign and spike data. As discussed earlier, Upwell’sBaseline calculations are derived from the insight that our primary ocean topics eachdemonstrate a weekly periodicity. Similarly, in calculating potential thresholds for whatconstitutes a spike, we started with that same insight and then calculated various multiples ofstandard deviation above the average (mean) value for that day of the week. Because standarddeviation measures how spread out the values within a data set are, using it to measure aparticular value’s variation from the “normal” value of that data set is a good way to test for aspike. Spikes are visible because they’re outliers, and that’s what the standard deviationthreshold(s) tests. 38
  39. 39. Day-of-the-week values for the Sustainable Seafood Baseline, along with the Sustainable Seafoodmean, and mean +1x, +1.5x and +2x standard deviations (10/17/11 - 1/29/13). [Source]As seen above, the standard deviation thresholds are higher than both the Baseline and themean. Graphing those thresholds against our campaign and event records revealed that the onestandard deviation threshold was the most accurate representation of what we were observingon a day-to-day basis. 39
  40. 40. Social mentions for Upwell’s Sustainable Seafood keyword group vs. Upwell’s Sustainable SeafoodBaseline vs. ‘Mean + 1 Standard Deviation’ Spike Threshold (June 1, 2012 - August 1, 2012)Upwell defines a spike as occurring when the social mention volume for a given day meetsor exceeds one standard deviation from the mean of all recorded values for that same day ofthe week.While a critic might accuse us of working backwards to find the threshold that gives the best fit,we would actually agree. Sustainable Seafood and Overfishing are the topics that we know thebest—because we’ve monitored them and campaigned on them with the most focus—and wewere looking for a metric that would have practical implications for attention campaigns. Asmentioned before, we remain open to other spike quantification approaches but this one is ourpreferred option, given what we know right now. 40
  41. 41. What Does Spike Quantification Tell Us?Upwell’s spike quantification methodology is in alpha, so to speak, and going forward we willlook to improve it. The possibilities for more comparative measures of success are numerous.One thing is certain, however: applying a spike quantification lens to our work is illuminating.Spike comparison beta methodology?The graphs on the following pages show our first Winter in 2011 and most recent Winter in 2012working in the Overfishing and Sustainable Seafood conversations. Both one standard deviationand two standard deviation threshold lines are included for reference.The comparison in time periods for both conversations is dramatic. There is a noticeableincrease in spike frequency (the number of spikes), spike volume or  “spikiness” (see: the numberof spikes exceeding two standard deviations), and in the overall volume of conversation in thetime period as measured against the Baseline. To be blunt: this is what success looks like. 41
  42. 42. Sustainable Seafood: Winter 2011Social mentions by day for Upwell’s Sustainable Seafood keyword group, as compared to theSustainable Seafood Baseline, as well as to spike thresholds of one standard deviation and twostandard deviations above the day-of-the-week mean (10/17/2011 - 1/31/12). Total post volume:45,255 social mentions over 107 days. Average volume / day: 423 social mentions. 42
  43. 43. Sustainable Seafood: Winter 2012Social mentions by day for Upwell’s Sustainable Seafood keyword group, as compared to theSustainable Seafood Baseline, as well as to spike thresholds of one standard deviation and twostandard deviations above the day-of-the-week mean (10/1/2012 - 1/29/2013). Total post volume:66,456 social mentions over 121 days. Average volume / day: 549 social mentions. 43
  44. 44. Overfishing: Winter 2011Social mentions by day for Upwell’s Overfishing keyword group, as compared to the OverfishingBaseline, as well as to spike thresholds of one standard deviation and two standard deviationsabove the day-of-the-week mean (10/17/2011 - 1/31/12). Total post volume: 211,799 social mentionsover 107 days. Average volume / day: 1,979 social mentions. 44
  45. 45. Overfishing: Winter 2012Social mentions by day for Upwell’s Overfishing keyword group, as compared to the Overfishingbaseline, as well as spike thresholds of one standard deviation and two standard deviations abovethe day-of-the-week mean (10/1/2012 - 1/29/13). Total post volume: 409,692 social mentions over121 days. Average volume / day: 3,386 social mentions. 45
  46. 46. Keyword SetsThe following search terms are Upwell Radian6 keyword sets for Upwell’s primary campaigntopics—Sustainable Seafood and Overfishing—as of the writing of this report, along with a briefdescription of what each keyword group is designed to capture. As online content and contextcontinually changes, keyword groups should ideally be monitored and refined on an ongoingbasis as well. A keyword that returns noise-free results for one period of time may be filled withunrelated results for another. Upwell’s keyword groups were designed for the time periodsspecified in each description.Fishing and Seafood: Sustainable SeafoodPrimary Keyword Group: Sustainable SeafoodEarliest Data: 10/17/2011Keywords: "#seafoodsummit", "#ss12hk", "@leodicaprio" AND "@upwell_us", "@seafoodwatch","@upwell_us" AND "vote4stuff", "alaska salmon", "alaskan salmon", "aquaculture dialogs","aquaculture dialogues", "aquaculture stewardship council", "barton seaver", "big listerner","bitly.com/wppomr", "bittman" AND "tuna" AND "safeway", "cannibal endtimes lobster"~6,"cannibalistic lobsters" AND "end times", "cannibalistic lobsters overfishing"~20, "casson trenor","catch limits", "catch shares", "chefs collaborative" AND "seafood", "cruel new fact of crustaceanlife" AND "lobster cannibalism", "davidsuzukifdn lobsters into cannibals"~9, "dungeness crab","environmentally responsible seafood", "f.a.d.-free" AND "tuna", "f.a.d.-free tuna comes tosafeway", "fad-free" AND "tuna", "fao" AND "fisheries", "fishing quotas", "fishphone", "fishwatch","food and agriculture organization" AND "seafood", "friend of the sea", "green chefs blue ocean","how safeway ended up selling cheap, responsibly-caught store brand tuna", "http:/ /bittman.blogs.nytimes.com/2012/10/02/fad-free-tuna-comes-to-safeway-affordably/", "http:/ /twitpic.com/bli9ak", "http:/ /www.bethkanter.org/listener/", "h"#biglistener", ttp:/ /www.fastcoexist.com/node/1680610", "http:/ /www.youtube.com/watch?v=jmkevhbejla","https://bitly.com/wppomr", "international seafood sustainability foundation", "issf" AND"fishing", "kanter" AND "big listener", "kanter" AND "big listening", "lack of predators lobster-on-lobster violence", "leodicaprio" AND "upwell", "marine stewardship council", "maximumsustainable yield", "menhaden" AND "sustainability", "menhaden" AND "sustainable", "nooverfishing guaranteed"~4, "ocean acidification" AND "google earth", "ocean wise", "ocean-friendly aquaculture", "ocean-friendly seafood", "oysters" AND "sustainability", "oysters" AND"sustainable", "political porpoise", "precautionary principle" AND "seafood", "responsibly caughttuna"~3, "reuters" AND "lobster cannibalism", "safeway sustainable tuna"~15, "seafood choicesalliance", "seafood consumer guide", "seafood ecolabel", "seafood fraud is a serious issue","seafood pocket guide", "seafood ratings", "seafood summit", "seafood sustainability", "seafoodwatch", "seafoodwatch", "sustainable fisheries", "sustainable fisheries act", "sustainablefisherman", "sustainable fishermen", "sustainable fishery", "sustainable seafood", "sustainableseafood"~9, "sustainable sushi", "sustainable" AND "tilapia", "the lobsters in maine are eating 46

×