Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Effective Measurement and Analytics in a Multichannel World by Gary Angel, President and CTO, Semphonic


Published on

The explosion of social media, digital media and digital channels has created a whole set of challenges to all marketing organizations. Not the least of these challenges is how to measure, analyze, …

The explosion of social media, digital media and digital channels has created a whole set of challenges to all marketing organizations. Not the least of these challenges is how to measure, analyze, and optimize this multi-headed beast. Traditional media tracking focuses on a small number of quasi-independent channels measured in the simplest terms (reach and audience). Digital is different in every respect. Digital channels are tightly bound, and the performance of each is heavily dependent on the performance of the entire system. Both cannibalization and support are common. Digital channels lack a single common metric because there‘re no equivalent to GRPs (gross rating points). Digital channels lack even a common type of metric – with Awareness, Engagement, Conversion and Retention (and a host of sub-metrics under these categories) vying for attention and credibility. Worse, focusing on top-line or siloed metrics will nearly always miss the point.

Published in: Technology, Business

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. White Paper Effective Measurement and Analytics in a Multichannel World Taming the Many-Headed Digital Marketing Beast by Gary Angel, President and CTO, Semphonic
  • 2. ABSTRACT The explosion of social media, digital media and digital channels has created a whole set of challenges to all marketing organizations. Not the least of these challenges is how to measure, analyze, and optimize this multi-headed beast. Traditional media tracking focuses on a small number of quasi-independent channels measured in the simplest terms (reach and audience). Digital is different in every respect. Digital channels are tightly bound, and the performance of each is heavily dependent on the performance of the entire system. Both cannibalization and support are common. Digital channels lack a single common metric because there‘re no equivalent to GRPs (gross rating points). Digital channels lack even a common type of metric – with Awareness, Engagement, Conversion and Retention (and a host of sub-metrics under these categories) vying for attention and credibility. Worse, focusing on top-line or siloed metrics will nearly always miss the point. All of this makes the effective measurement and optimization of digital marketing efforts extremely challenging. Challenging or not, however, it’s a problem that can’t be avoided. As marketing budgets shift into ever-increasing digital and social share, it’s no longer acceptable to ignore the problems around digital channel measurement and optimization. So the creation of enterprise-wide, digital marketing dashboards has become a priority at many leading businesses today. However, digital marketing dashboards, as commonly developed, are uninformative at best and misleading at worst. In this white paper, we’ll show why traditional best-practices in digital reporting fail to capture both the nature of the customer journey and, perhaps more surprising, even the right information about behavior WITHIN the silos. We’ll present a methodology for doing the job right. We’ll also highlight a technology solution that makes the integration of the necessary data, as well as the construction of the necessary analysis and reporting possible. For the purposes of the examples in this whitepaper, we’ll be using Anametrix for multichannel analytics and reporting, Google Analytics for web analytics, DART for ad serving, ViralHeat for social media analytics and for CRM. THE PROBLEM OF INTEGRATED MULTICHANNEL MARKETING MEASUREMENT Digital marketing in today’s world requires integrated multichannel measurement and analysis. However, enterprise measurement and dashboarding systems attempting to provide that measurement have suffered from three primary problems: (1) they lack methods for joining and understanding customer engagement across silos; (2) they focus on topline performance without providing the necessary context to create real understanding of the interrelationships leading up to them; and (3) they capture only the state of the system, but completely fail to describe the system itself and its potential levers. In the next three sections, we’ll explore each of these problems and show how they cripple most digital marketing measurement and analysis efforts.
  • 3. 1 The Problem With Silos Traditional mass-media buying featured relatively little interaction between channels. While Adstock might increase or decrease depending on exposure in multiple channels, the basic measurements of reach and audience were independent. You didn’t reduce or improve your radio GRPs by buying TV. In digital, however, that just isn’t true. Digital marketing works quite differently. Your search traffic (including your PAID search traffic) is heavily dependent on your offline media spend, your display ad budget and even your organic search listing efficiency. This deep interdependence is true across the board for digital: from social to mobile to fixed web to search and display. Unfortunately, as media buying and measurement have migrated over to digital, many of the traditional habits and expectations borne from traditional marketing have accompanied this transition. Most marketing measurement and analysis efforts are based on the aggregation of siloed channel data. When this data is aggregated, it’s not integrated at the prospect or customer level. There’s usually little or no effort made to understand the full customer journey or the multiple marketing and digital touchpoints along the way. This creates severe problems with campaign optimization and attribution. You can’t properly credit campaign success without understanding the full set of customer touchpoints and the sequence in which they’ve occurred. Reports showing revenue is up in every marketing channel but down in reality is all too common. If you are relying on the integration of siloed campaign measurement for each channel for optimization and dashboarding, you’re almost certainly seriously misrepresenting the reality of your marketing effectiveness. The problems don’t end with attribution. Siloed data makes almost every type of analysis problematic. • Want to understand the role of mobile in the customer journey? You can’t do that unless you can integrate mobile and fixed web data. • Want to identify segments to target by channel? It’s impossible without integrating customer data and demographics with channel behavior. • Want to measure the true ROI of your social marketing efforts? How do you do it without reviewing how social media correlates to changes in campaign returns across each marketing channel? • Want to understand campaign performance relative to your industry? You need to integrate competitive set and NPD -type online research data with your reporting. • Want to understand the true value of a content consumer? You need to integrate your DART-type ad serving data with your web behavioral data. • Want to understand the total impact of your spend across all channels? You simply have to combine spend, impact and outcome data in a single view to evaluate marketing performance. At every level of marketing, the failure to integrate data across silos creates huge – and dangerous – gaps in understanding. It’s dangerous because without the holistic understanding that only multichannel analytics can deliver, marketers will all too often come to incorrect conclusions regarding the effectiveness of their marketing spend WITHIN each silo.
  • 4. 2 The Problem With Focus on Top-Line Metrics It’s probably no surprise that siloed metrics are a plague on your measurement house. At least, inside each channel you have a clear and powerful strategy for reporting on success. Your measurement department has almost certainly discovered the current religion in digital measurement: don’t overload on numbers. The key to successful dashboarding and reporting is finding a small set of marketing key performance indicators (KPIs) that are understandable and immediately actionable. Chances are, that’s exactly what your enterprise has adopted – a small set of key metrics like Site Conversion Rate, Campaign Conversion Rate, Cost per Conversion (CPC), Customer Retention Rate, Customer Referral Rate, Average Number of Service Calls per Day, and other similar KPIs throughout the customer engagement lifecycle, all laid out in big numbers with great fonts, pretty colors, big trend arrows and lots of Tufte-inspired whitespace. Unfortunately, these reports deliver neither understanding nor actionability on their own. Suppose I walk into your office and tell you that Site Conversion rate is up 5 percent. You’ll probably be delighted. Now suppose I walk into your office and tell you that your Site Search Engine traffic is down 20 percent. That’s bad, right? But would you realize that, in all probability, the two metrics are related and are, in fact, telling you exactly the same story? As you drive less early-stage traffic to your site via natural search, your Conversion Rate will go up. Understanding the relationship between parts of the system is fundamentally different and more important than understanding the state of any single performance indicator. What happens if you find out that the CPC of a specific campaign is quite low compared to others? You’re happy about the high ROI, right? What if you found out your CRM call volume is up during that same time? That should signal bad news. Unless you look at the entire system, what you may not realize is that the high ROI campaign that you were delighted with happens to be driving customers that also lead to more service calls or higher return rates, significantly eroding your profit margins. What about revenue? Surely, it’s impossible for a revenue increase to be bad! Not only isn’t it impossible, it’s common. For example, company A creates a re-marketing program that sends a 10 percent discount offer to all cart abandons. Unfortunately, consumers quickly game the system and abandon the cart to get the coupon. To make matter worse, they share the news of the cart abandonment discount in social media to let their friends take advantage of this great offer! Gross Revenue increases with the remarketing program and the ensuing social buzz that you may or may not be tracking. However, Net Revenue declines due to reduced margins. Short-term revenue gains that sacrifice margin are frequent. Poor reporting systems make this type of system interdependency and correlation difficult or impossible to spot or understand. Worse yet, they can create the wrong indicators of and incentives for actual performance. There just is NO SINGLE METRIC THAT CAN BE MEANINGFULLY INTERPRETED WHEN VIEWED IN ISOLATION. What’s needed is a way to create meaningful KPI context so that movement and level can be easily consumed and understood.
  • 5. 3 The Problem With Showing the Current State You are the manager of the long-play videos site section. And, hurrah, your company has really gotten its act together. Your performance is tied to specific site goals, and your company has established a clear target for long-play video on the site: a 20 percent year-over-year increase. That’s great. There’s nothing like clear goals tied to real incentives to sharpen the mind and drive performance. You ask your analytics team to create a report on long-play videos. Here’s what they come back with: This is the classic measurement view for an organization with a well-defined success metric, a clear goal and robust ability to measure. You can, with this report, instantly know whether you are on-plan or not. That’s great. However, isn’t something missing from this report? While you can instantly see whether you’re on-plan or not, you have no idea why. As long as you’re on-plan, you’ll never look at this report. It isn’t really a report at all. It’s an alert. If the only function of a report is to alert you when you’re not on-plan, why not just deliver that: Really! Why not just send this if the only meaningful intelligence in your report is that you have something to worry about. It’s less expensive and more impactful. When report builders consider this kind of question, they often decide that they do, in fact, need to deliver a little more. So they start adding to the report. They add traffic sources since changes to Search Traffic might easily drive changes to the long-play video site section. They add a report on top Exit Pages, since maybe some videos lost traffic too easily. They add a report on top Entry Pages to help spot videos that might be getting more direct traffic and could be further supported. And so on. Pretty soon, there’s a report for every different kind of data that might help a decision-maker understand why they are not on-plan. Now there’s too much data. With no connections between all these reports it’s nearly impossible for a decision-maker to navigate the reports and decide what’s interesting, what’s meaningful and what’s just noise. What’s needed is a method of going beyond the “current state” to show the actual levers of change.
  • 6. SOLVING THE PROBLEMS OF MULTICHANNEL MEASUREMENT: ONE HEAD OF THE BEAST AT A TIME 1 Multichannel Data Integration: Solving the Silo Problem In today’s digital world, there isn’t one single integration challenge – there are many. With the explosion in digital and social channels and the increasing importance of understanding the relationship between offline and online touchpoints, the idea of doing siloed measurement just isn’t reasonable. On the other hand, that same explosion of sources makes integration daunting. It’s critical to have a real strategy about what you need to integrate and how that integration can occur. There are two common levels/types of integration in digital analytics: 1. Integration of Two Streams of Data at the Visitor Level 2. Integration of Stream Data Into Row-based Customer Data The challenges in joining multichannel data streams are solved by the same segmentation techniques that address the challenge of top-line metrics. Segmentation, while a classic, marketing-analysis technique, is also a very effective data reduction and aggregation technique. By creating the two-tiered segmentation we’ve described (visitor/visit-type), you can reduce a complex chunk of stream data into a small number of discrete variables attached to a single row. These variables describe the type of the visit, its measured success and its recency. By summing visit types, successes, recency and frequency at the visitor level, you have a powerful but very terse description of a large amount of customer behavior. This type of aggregation by segmentation is an obvious solution to our problem #1 since it creates a record that can easily be joined to a customer record. It also provides a ready-made solution to the first challenge. The beauty of the Two-Tiered model is that it can be applied to a very broad number of data streams, including offline data streams. It’s perfectly possible to model mobile engagement, social media buzz, web visits, calls to the call-center, ATM visits, branch or store visits and much more with the same simple model of type, recency, frequency and success. By supplying a unifying model to different types of data streams, each can be meaningfully reduced from stream data into discrete records that can then be intelligibly joined at the customer level. This delivers a holistic understanding of how prospects and customers engage with the brand across multiple channels throughout the lifetime of their engagement. 2 Segmentation Solves the Problem of Focus on Top-Line Metrics The simple fact is that aggregate reach, engagement, conversion and retention metrics – from number of Twitter followers and Facebook likes to Total Site Traffic to Conversion Rate to Campaign ROI to Churn Rate – are nearly all worthless. To be meaningful, a metric needs to be placed in the context of “who” it’s about and “what” those customers were trying to accomplish. Audience and Customer segmentations are the foundation of interesting measurement and analysis. If you walk into your bosses office and say something like, “Good news, traffic is up 5 percent on the Website,” or “bad news, we have 5 percent more customer churn,” I expect the boss to ask “With whom?” followed by “why?” Knowing the “who” behind a number is nearly always critical. With traffic, it’s particularly important. Bad traffic is ubiquitous
  • 7. on the Web. It’s cheap and too easy to find or buy. If you are going to measure campaign performance, you must measure something beyond traffic, otherwise you’ll almost certainly drown in unqualified and likely useless traffic. Basically, if you don’t know the audience behind that traffic increase, you really don’t know anything. Which campaign and channel is driving the visit? Do we have any demographic data to gain more insight into who the visitors are? What’s the K-factor (virality) associated with the visitors? Are they driving social-media driven visits in addition to the paid campaigndriven traffic? In traditional marketing, the “who” question was pretty much all there was to segmentation. In digital, that’s not true. Because once you’ve answered the “who” question, the next question I expect your boss to ask is “And why did they engage?” In pretty much every digital channel, your customers set the agenda for an engagement or touch. It’s your job to figure out what they had in mind and match your business goals to their intent. In essence, understanding what your customers are trying to accomplish during every engagement is part and parcel of understanding whether you are successful or not. What are your chances of making a product sale during a Customer Support visit? Zero. So if customer support visits are included in your site-wide Conversion Rate, what are you really measuring? In site-wide metrics, typically all you’re seeing is noise. This second type of segmentation – visit- or intent-based segmentation – is fairly unique to digital, and it’s why we say digital requires a two-tiered segmentation. The idea is simple: every metric should be in the context of a “who” and a “what.” Building this type of segmentation can be hard; indeed, creating visit-intent segments is one of the most challenging but fruitful tasks in digital analytics. It’s fruitful because your dashboards and reporting are only useful if they’re built around this concept. In essence, building good dashboards, scorecards and report sets starts with segmentation, however segmentation is only a piece of what’s required. 3 System Reports With Multichannel Integration and Embedded Segmentation Solve the “Current State” Problem Nearly all enterprise reporting has concentrated on finding variables that can show the state of a system. There’s no denying that understanding “where you’re at” is useful. But if knowing “where you’re at” is important, it’s largely in the context of knowing where you could go. We’ve developed great reports for showing latitude and longitude; we haven’t done much for filling the rest of the map. A really good report set will capture more than the “state” of the system by showing the connections between each variable in the system and report on the changes in those connections. To achieve this kind of report requires building a model of the system itself. Suppose, that you want to understand how social media – whether social marketing efforts driven by your company or other social buzz mostly out of your control – is impacting paid campaign effectiveness. The below Social Insights report from Anametrix overlays data from social analytics solutions such as ViralHeat or Radian6 with your campaign data across each channel to reveal interdependencies and relationships. It provides marketers with immediate insights into how social buzz correlates to campaign performance, including traffic and conversion metrics, highlighting whether social activity is impacting campaign results, or whether certain campaigns are going viral, and driving additional social media-driven visits.
  • 8. Further segmentation and filtering also enable you to ascertain if certain channels and / or geographies are more likely to being affected by any specific social buzz elements. As you can see, when you incorporate these multiple data sources in a report, you’ve done so much more than capture the state of the system. You’ve helped a decision maker understand the factors that drive the system, the degree to which each factor contributes and the places where potentially significant improvement opportunities exist.
  • 9. PUTTING IT ALL TOGETHER In this white paper, we’ve described three fundamental challenges in digital measurement, analysis and reporting. Nearly everyone can see and understand (and, unfortunately, has experienced!) these challenges. For each of these challenges, we’ve also described a solution. Segmentation, multichannel integration based on intelligent handling of streams, and system-based reporting using graphical flows, are the core methods that can drive your enterprise measurement to an entirely new level. It will probably be no surprise, however, that the most common existing toolkits for doing marketing analytics and reporting aren’t really up to the job. Web analytics tools have improved their segmentation capabilities considerably in recent years. However, many still lack significant segmentation capabilities. In addition to frequent limits on Visitor-based segments, they often make it difficult or impossible to overlay customer (visitor) segmentations from the offline world, lack capabilities for data-driven segmentation, limit cross-tabulation of segments (such as Visitor x Visit Type), lack cohort capabilities, and provide no way to integrate segments into back-end processes. Segmentation, as a result, is often unavailable in reports and analysis. At the integration level, the lack of segmentation is fatal to the strategies suggested here. In addition, there are numerous limits on the type of integrations that can be accomplished, the ability to effectively use integrated data and constraints caused by the cost and timeliness of creating and supporting the integrations themselves. Any attempt to integrate CRM data, Social Media Data, even Campaign Impression data into these systems is nearly always expensive AND fruitless. Most organizations have decided to tackle these integrations in internal warehouse-based systems. Given the poorly understood nature of joining digital streams, this strategy has not been widely effective. Furthermore, internal warehouses suffer from a fatal lack of agility. Digital isn’t just one source. It’s many. From web to mobile to display data to PPC to Pandora to Facebook to email, there’s a seemingly never-ending list of important channels and the associated and often quite complex data. Traditional IT just can’t keep up. Finally, at the user-interface level, it’s simply not possible to build really good “system” models in most measurement and analytics tools. The visualizations are challenging to create, as are the embedded models. It’s critical to have a reporting system focused on the customization of sophisticated dashboards instead of drag-and-drop of charts and tables. The Anametrix Digital Analytics platform has been designed from the ground-up to solve exactly this set of problems. Created by the team of analytics experts from WebSideStory, it’s engineered to tackle the next generation of digital analytics challenges, specifically the multichannel integration, analysis and visualization of data. Unlike other solutions, it provides a natural platform for the integration of digital and non-digital data sources. It even includes a number of built-in “external” integrations (such as U.S Census data) that are often extremely useful in building system models. It provides an open data model on which the User Interface can work, which is a critically important feature for handling multiple channels and types of data. As a segmentation engine, it provides all the front-end capabilities you’d expect including unlimited filtering at the Visit and Visitor level. It also supports segmentation on the back-end at the data-level, making it possible to fully support the type of integration with segmentation suggested here as the most appropriate method for joining disparate streams of data.
  • 10. From the User Interface perspective, Anametrix is specifically designed to support rich customization of any of number of data-extraction models, along with predefined reports that are customized to visualize the multichannel marketing process throughout the customer engagement lifecycle. This interface is built on top of the data model, so it works very well for all sorts of data types and data structures. It accommodates the most critical features in system-based reporting: extensive filtering, custom elements and complex calculations. Here, for example, is a sample Performance Overview Report designed to evaluate campaign ROI across channels… …and an Executive Overview Report designed to understand channel performance compared to previous periods and goals:
  • 11. Both reports illustrate many of the fundamental challenges and solutions we’ve discussed in this whitepaper. Anametrix uses multi-channel integration to measure channel performance for each campaign based on integrated cost data. It also uses segmentation and modeling to help create a powerful view of campaign success, across each channel, based on customer lifetime value.
  • 12. In this next Anametrix dashboard, Ad, Web, Social, and Business Goal data are integrated along with Lifetime Value modeling to produce a simple, comprehensive A/B comparison of end-to-end campaign effectiveness. Typically, a decisionmaker would focus on the first three-to-five metrics extracted from email marketing tools and web analytics, to determine performance. In this case, the top five metrics clearly favor the campaign on the right. It’s only after reviewing the entire system, including campaign virality, customer lifetime value and true ROI derived from CRM, social media and other data sources, that we realize the campaign on the left is the real winner. As you can see, combining digital segmentation, multichannel integration, and system-based reporting gives decision makers a holistic perspective into how customers are engaging with their brand through out the lifetime of their engagement. It lets them identify the correct levers to continually optimize marketing performance to further drive revenue and profitability. Ultimately, driving your digital marketing effectiveness to the next level takes both methodology AND technology.
  • 13. SUMMARY Traditional approaches to digital marketing measurement, analytics, reporting and dashboarding have assumed that it was possible to treat customer engagement with the brand in silos, to focus on high-level campaign or site metrics and to capture the “state” of a key variable or system. Each of these assumptions is wrong, and they have fed the creation of an ever-growing set of enterprise digital reports and dashboards that are neither interesting nore useful. The remedies to these problems aren’t necessarily simple or easy, but remedies do exist. By framing digital data in terms of audience and intent, it’s possible to create a meaningful context around the customer journey and eliminate the errant focus on top-line metrics. By navigating the tricky task of integrating digital stream data from multiple channels, it’s possible to drive down to a unified view of customer behavior – across all engagement touch points – that can solve the silo problem. By embedding models into reporting, it’s possible to capture not just the state of a digital system, but the levers of change, as well as the connections and interdependencies between each of them. As with most things (not just digital), good solutions take real work and require good tools. To solve these problems, you need a different set of capabilities than are inherent in most of today’s digital reporting tools. You need an open data model capable of flexible integration of stream data. You need to be able to build segmentations on the back end to facilitate that integration. You need to be able to create complex dashboards that can handle different data types, provide embedded segmentation, allow for the integration of models and calculations, and support the visual representation of flows and relationships and interdependencies. Digital Marketing is, truly, a many-headed beast. It’s multiplicity of channels present deep challenges to any technology and any approach. Traditional IT approaches lack the agility to cost-effectively integrate so many complex data sources. Web analytics solutions lack the openness to bring multichannel data together and lack the power to analyze and report on it effectively. It’s, nevertheless, a beast that must be tamed. Measurement is the key to effective digital marketing. And multichannel integration is absolutely essential to effective digital measurement and analysis. Putting the right methods and the right technology together can make all the difference for ensuring marketing is, indeed, a predictable and effective revenue driver for your business.
  • 14. About Semphonic ABOUT ANAMETRIX Semphonic is a cutting-edge digital measurement and data analytics consulting firm providing our clients with deep insight and strategy into their complex customer interaction challenges across the digital channel. Semphonic has deep roots in the analysis of large-scale databases, combining the power of segmentation, predictive modeling, and deep statistical analysis to help clients identify and execute high ROI, data-driven marketing campaigns. Founded in 1997, Semphonic’s US headquarters are in the San Francisco Bay Area with offices in Boston, New York, Washington DC, Portland and European headquarters in Berlin. Anametrix transforms marketing with digital analytics. We collect, analyze, and make sense out of data across all channels in real time to enable marketers to discover new truths about customers, prospects, and the market at large. Anametrix delivers 360-degree visibility into business data to uncover new trends and hidden correlations, explore new relationships, and deliver a bigger and more predictable impact on revenue. Founded in 2010 by the trailblazing web analytics team behind WebSideStory, Anametrix has headquarters in San Diego, Calif. For more information visit: Website: Twitter: @semphonic Blog: For more information visit: Website: Twitter: @anametrix Blog: