Presentation given to EMU's Center for Digital Engagement Summer Clinic.

1. Building Digital Marketing
Measurement Systems
Russ Merz, Ph.D.
Professor of Integrated Marketing Communication
Research Scientist and Analytics Consultant
The Center for Digital Engagement
© Russ Merz, Ph. D., 2016

2. Agenda
§ Why do Digital Marketers Need Measurement
Systems?
§ How Are Measurement Systems Built?
§ What are the Important Parts of Each Stage in the
Process?
§ What is the Role of Digital MarkeKng Technology?
2
The Center for Digital Engagement
© Russ Merz, Ph. D., 2016

3. “If you don’t measure it, you can’t manage it”
• OK, everybody has heard this trite phrase,…is it true?
• If it is, then it is only a necessary condiKon, not a sufficient one…because,
“What you choose to measure has a huge influence on the nature of your
business”
• Sounds good, but is it?
• SomeKmes managers will measure the things they manage, which might be good,
but only if they are measuring the right things?
• If they aren’t, then they will become what they measure and probably make bad
decisions…so then,
“What you choose to measure, and the quality of the measures used, affects
the financial yield of your management efforts and your business”
• Now this sounds beTer because quality measures are Ked to an outcome…but,
• Do you know how to do it?
Why do Digital Marketers Need
Measurement Systems?
3
© Russ Merz, Ph. D., 2016

4. How Are Measurement Systems Built?
The process for building digital markeKng measurement systems (DMMS) can be
summarized in terms of four key steps:
Design Develop DeployDiagnose
The goal here is to
thoroughly
understand how the
business is using
digital markeKng to
achieve desired
performance
outcomes:
• What is the digital
markeKng strategy
model?
• What assumpKons are
being made?
• How are acKons and
results currently
measured?
• Who are the audiences?
• What are the SWOTs?
Revise and modify the
exisKng measurement
system by:
• IdenKfying business
goals for each digital
markeKng strategy,
• Mapping goals to KPIs,
• Se[ng target
performance levels for
each KPI metric,
• Specifying how KPIs are
related (“cause and
effect”),
• Determine how metrics
will be sourced and
collected,
• PrioriKzing audience
segments.
Implement the
measurement plan by:
• CollecKng the relevant
metrics from data
sources,
• Organizing and
classifying metrics
according to role in data
model,
• ConducKng iniKal
analysis required by
model,
• Assess reliability and
make changes as needed,
• Perform analysis by
segments as needed.
Report findings and
communicate insights
by:
• Making formal
presentaKons of the
results to key
stakeholders,
• CreaKng customized
manager tools such as
dashboards, scorecards,
or calculators,
• Showing how results can
be used to improve
decision making,
• IdenKfy areas for
change.
Measurement
Strategy
Measurement
TacKcs
Measurement
AcKons
Measurement
Outcomes
4
© Russ Merz, Ph. D., 2016

5. Diagnose the Digital MarkeKng Strategy
A good starKng point for diagnosing the digital markeKng strategy of a business is to
compare the business’s strategy against a norma&ve model like the generic version below.
Use the comparison to idenKfy what is present, and what is missing, from the digital
markeKng strategy being evaluated.
5
Diagnose
© Russ Merz, Ph. D., 2016

6. Diagnose the Digital MarkeKng Strategy
TIP: Some templates for various
types of digital markeKng
strategy models can be found in
digital publicaKons. They can be
useful as starKng points for
diagnosis.
For example, this framework
from Search Engine Watch
presents a systemaKc plan for
building a social media strategy
measurement plan.
(source:
hTp://searchenginewatch.com/sew/
how-to/2202307/social-media-roi-how-
to-define-a-strategic-plan).
6
Diagnose
© Russ Merz, Ph. D., 2016

7. Design the Measurement System
Aher the digital markeKng strategy model is mapped out, then the next step is to designate
the measurement tac&cs to be used. Here’s an example of an approach primarily intended
for public relaKons pracKKoners developed by the Associa&on for the Measurement and
Evalua&on of Communica&on (AMEC: www.amecorg.com). It consists of the five
communicaKon effects shown below.
7
Design
© Russ Merz, Ph. D., 2016

8. Examples of Metrics that can be used for each model stage and type of media.
Social Media Metrics Website Metrics
8
Design the Measurement System Design
© Russ Merz, Ph. D., 2016

9. TIP: Use a worksheet to organize your planning. Here’s a useful one from AMEC.
9
Design the Measurement System Design
© Russ Merz, Ph. D., 2016

10. • AnalyKcs are metrics and models that
provide acKonable insight.
• Metrics are measurements
represenKng performance levels of
important variables to an
organizaKon (KPIs). They are used to
monitor situaKons and diagnose
problems. The dashboard in your car
is an analogy.
• Models are tools that combine
metrics to aid managers in making
decisions. They simplify and
represent the real world along a
dimension of interest (e.g. customer
experience). An analogy might be a
GPS system in your car.
• Both Metrics and Models are needed
for Insight.
AnalyKcs
Metrics
(Measures, KPIs)
Models
(Decision Tools)
AcKonable Insight
AnalyKcs-Based Measurement Systems
Combine Metrics with Models to Generate
AcKonable Insights for BeOer Decision Making
Metrics + Models = Insight
10
Design the Measurement System Develop
© Russ Merz, Ph. D., 2016
Adapted from: Sorger, Stephan (2013) MarkeKng
AnalyKcs: Strategic Models and Metrics

11. Metrics:
• Can be “Hard” (observable/countable)
ü Financials, units sold, clicks, page
views, bounce rates, etc.
• Can be “Soh” (unobservable)
ü Measures on a survey
ü Scaled
– Nominal/Categorical (e.g., gender)
– Ordinal (GT or LT)
– Interval
– RaKo
ü Open-ended
• Ohen Displayed in Dashboards
ü Grouped and Color coded
ü AutomaKcally updated
• Provide performance insight
ü Monitored and tracked
Models:
• Have Different “Styles”
ü Verbal
ü Pictorial
ü MathemaKcal
• Have Different “Forms”
ü DescripKve/ComparaKve
ü PredicKve
ü NormaKve
• Can be “Built” with
ü InducKve methods (data determines
theory), or,
ü DeducKve methods (theory
determines data)
• Require at least,
ü One dependent and one independent
variable [ Y = ƒ(X) ]
Necessary, But Not Sufficient Required for Decision-Making Value
11
Design the Measurement System Develop
© Russ Merz, Ph. D., 2016
Adapted from: Sorger, Stephan (2013) MarkeKng
AnalyKcs: Strategic Models and Metrics

12. 12
Design the Measurement System Develop
TIP: The value of the
insight created from the
measurement system is
largely a funcKon of the
types of models used.
Measurement systems
that integrate metrics
into wholisKc views of
strategy performance
are more useful for
decision-making.
© Russ Merz, Ph. D., 2016

13. The integraKon of metrics has gone
through at least three phases of
evoluKon:
ü Phase 1: Dials on a Dashboard—good,
but not true integraKon, it only
integrates metrics by presenKng them
side by side.
ü Phase 2: Trend line comparisons—
beOer, because it incorporates a
temporal perspecKve of changes in
metric values.
ü Phase 3: Quan:fied Rela:onships—
best, because it computes a
mathemaKcal value for the “cause and
effect” relaKonship between metrics.
Satisfaction, On-Line Purchase Intentions and Revenues over Time
-20.0%
-10.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
4/21/2003
4/28/2003
5/5/2003
5/12/2003
5/19/2003
5/26/2003
6/2/2003
6/9/2003
6/16/2003
6/23/2003
6/30/2003
7/7/2003
7/14/2003
7/21/2003
7/28/2003
8/4/2003
8/11/2003
8/18/2003
8/25/2003
9/1/2003
9/8/2003
9/15/2003
9/22/2003
9/29/2003
10/6/2003
10/13/2003
10/20/2003
10/27/2003
Weekly Revenue Dates
PercentageChange
Lagged %ChngPOL Lagged %ChngSAT %ChngREV
Design the Measurement System Develop
© Russ Merz, Ph. D., 2016

14. 14
Design the Measurement System Develop
The central tasks in designing a measurement system are: (1) determining the metrics to
be used, (2) idenKfying the role of the metrics, and (3) specifying the “cause and effect”
pathways between the metrics (if any). This “hypotheKcal model” forms the basis for the
development of the measurement system.
© Russ Merz, Ph. D., 2016

15. Example: A soh drink brand wanted to know if the
traffic generated by the a branded “naKve ad” based
website as measured by Google AnalyKcs would
affect the volume of social media conversaKons
related to the “NaKve Ad”, across a specified Kme
period.
15
DescripKve Analysis of the Data: The
correlaKon matrix showed the direcKon of the
relaKonships between the predictor and target
variable sets.
Design the Measurement System Develop
PredicKve Modeling Results: The results
supported the hypothesized relaKonships
between the branded “naKve ad” campaign
and social media conversa&ons related to
the campaign.
• The results demonstrated a significant
impact of Google AnalyKcs metrics onto
all three sets of brand related
conversaKons.
© Russ Merz, Ph. D., 2016

16. 16
Design the Measurement System Develop
Example: XXX soda wanted to know if social media
conversaKon topics related to the psychographic
characterisKcs of potenKal XXX users could predict
(1) XXX brand related conversaKons, and (2) XXX
website traffic, using 120 4-hour Kme periods.
PredicKve Modeling Results: The results
showed that the psychographic topics
were predicKve of the specific XXX brand
related conversaKons, and moderately
predicKve of website traffic. In addiKon,
XXX brand conversaKons were are also
jointly predicKve of website traffic.
DescripKve StaKsKcs: The summary
staKsKcs of the data used in the analysis
show relaKve sizes of the counts for
each metric.
© Russ Merz, Ph. D., 2016
Visits
SKckiness
Engagement

17. 17
Design the Measurement System Deploy
Deploying the measurement system is the structuring and organiza&on of the metrics and
modeling results so that they facilitate digital markeKng decision making.
ü Transforming data to informa&on is sufficient for monitoring performance levels.
ü But, if the measurement system will be used for prioriKzing changes and allocaKng resources, then
knowledge about how changes in one or more metrics affects another metric is necessary.
ü Thus the managerial uses of the measurement system determine the type of deployment.
Raw Data
InformaKon
about
Performance
Knowledge
about Cause
and Effect
DescripKve/ComparaKve
PredicKve/PrescripKve
© Russ Merz, Ph. D., 2016

18. Dashboards are useful tools for
presenKng descripKve or comparaKve
analysis results.
ü Dashboards are basically reporKng
and monitoring tools that allow the
display of various website metrics or
Key Performance Indicators (KPIs).
ü They do not show the user how
changes in one metric effects
changes in another metric.
ü It is up to the user to make
connecKons between metrics based
on observaKon rather than
quanKtaKve methods.
ü There are many vendors providing
SaaS based dashboards that allow
you to decide what to display, when
the metrics are displayed, and how
the metrics are organized by using
special widgets.
Design the Measurement System Deploy
© Russ Merz, Ph. D., 2016

19. Key Success Indicator (KSI) "What if" Calculator
Change PW Online Subscriptions
72423 979 Activity Cost Savings
235,375.64$ Change PW 3.25$
5,248.41$ Newsletter Enrollments 2.75$
Online Registrations Online Purchase Plan CSI 2,446.92$ Online Subscriptions 2.50$
175000 2537 0.88 2,836.65$ Balance Transfers 4.25$
Weekly Enrollments Resulting from
245,907.63$ Weekly Total Savings less call center
and mail service
1909 667
Newsletter Account Transfers
Enrollments
Instructions:
Enter the desired
values in the blue
boxes.
When knowledge is needed to inform the decision maker about how changes in metric
performance is needed then “what if” tools can be built to embody the analysis results.
ü For instance this online banking client wanted to know what total customer service
cost savings would be realized under different online registraKon scenarios and with
different cost savings per online acKon.
Design the Measurement System Deploy
© Russ Merz, Ph. D., 2016

20. Based on the modeling
results a “what if”
calculaKon tool was built for
an ecommerce client.
ü It demonstrates the
changes in conversion
levels that can occur with
changes in website
customer saKsfacKon.
ü The examples show the
effects of a 5 point gain in
saKsfacKon (65 to 70) on
the rest of the variables in
the causal network.
Design the Measurement System Deploy
© Russ Merz, Ph. D., 2016

21. 21
The Role of MarkeKng Technology in DMMS
Back-End Data
CollecKon/
Processing
Technology
Refining or
AnalyKc
Technology
Front-End
Technology
User Interface
Can the capabiliKes to build out a digital markeKng measurement system (DMMS) be
purchased off-the-shelf from a vendor?
ü The recent explosion in the number of marke&ng technology providers allow companies to create
a so-called marke&ng stack to provide the basic components of a digital markeKng measurement
system.
ü Three basic sub-plavorms would be needed and can be purchased from various vendors.
Cloud-based data
storage and
processing tools;
in PaaS, IaaS,
form
Data analysis and
modeling tools;
R, SAS, SPSS, etc.
in SaaP form
ReporKng and
presentaKon
tools; Excel,
iDashboard, in
SaaP form
© Russ Merz, Ph. D., 2016

22. MarkeKng Technology Landscape
Unfortunately, this will not be an easy task given the mindboggling number of vendors.
According to ScoT Brinker at chiefmartech.com , as of March 2016 there are currently 3874
markeKng technology (MarTech) soluKons1—expanding from only 150 in 2011!
Source: hIp://chiefmartec.com/2016/03/marke&ng-technology-landscape-supergraphic-2016/
1 SoluKons do not equal companies, some companies are in mulKple categories.
22
© Russ Merz, Ph. D., 2016

23. MarkeKng Technology Landscape
They can be roughly organized into six MarTech capability categories, each with a
number of sub-categories—represenKng a overwhelming number of players and choices.
Source: hIp://chiefmartec.com/2016/03/marke&ng-technology-landscape-supergraphic-2016/
23
© Russ Merz, Ph. D., 2016

24. 24
Summary and QuesKons
This presentaKon has aTempted to provide you with a brief overview of a generic process
for building out a digital markeKng measurement system (DMMS).
• The process as described is composed of four basic stages:
ü Diagnoses—the determinaKon of the underlying digital markeKng strategy model to be
measured;
ü Design—the idenKficaKon of the metrics necessary for understanding how well the strategy is
performing;
ü Develop—the applicaKon of analysis tools for calculaKng the values of the metrics, and
impacts of those metrics, related to key outcomes expected from the digital markeKng
strategy measured; and,
ü Deploy—the idenKficaKon of appropriate presentaKon methods for insuring appropriate levels
of communicaKon and usage of the DMMS by decision makers.
Any QuesKons?
© Russ Merz, Ph. D., 2016