A lecture in digital analytics at Aalto University. The lecture is a part of a module in Information Technology Program (ITP). Summer 2015, Helsinki -- Dr. Joni Salminen is a lecturer in digital marketing. Besides online marketing, his interests include startups and web platforms. Contact: joolsa@utu.fi

1. Information Technology Program
Aalto University, 2015
Dr. Joni Salminen
joolsa@utu.fi, tel. +358 44 06 36 468
DIGITAL ANALYTICS
1

2. About average levels
• CTR ~5% in SEM, 0.05% in display
• CVR ~1-2%
• Bounce ~40%, if over 60% is usually a bad thing
• CPM $2.80 (Johnston, 2014)
• The numbers obviously vary by industry / firm, but
these levels are typical according to my experience.
2

3. Standardizing online and offline metrics:
The CPM approach
A. Events
– Participation costs x €, y people participate
– CPM = x / (y/1000)
B. Magazine catalogue
– Distribution costs x €, circulation is y
– CPM = x / (y/1000)
• Compare these to other marketing channels (e.g.
AdWords, Facebook)
3

4. Standardization: example
4
FESTARI CPM AJANKOHTA KÄVIJÄMÄÄRÄ (2014) VUOKRA
Iskelmä 35€ 25-27.6.2015 34 000 1 200€
Jysäri 77€ 3-4.7.2015 13 000 1 000€
SuomiPop 67€ 9-11.7.2015 15 000 1 000€
Tammerfest 12€ 17-18.7.2015 80 000 1 000€

5. eCMP: another way to standardize
• impressions: 1,000,000
• cost: 50€
• clicks: 20
• CTR: ??
• CPC: ??
• eCPM: ??
5

6. eCMP: another way to standardize
• impressions: 1,000,000
• cost: 50€
• clicks: 20
• CTR: 0.002%
• CPC: 0.25€
• eCPM: 0.05€
6

7. You can use metrics to calculate…more
metrics!
• The problem: our AdWords client wants to know the
estimated reach and number of visitors
• The only number we have is the client’s budget
• How the hell we gonna get those estimates?
7

8. You can use metrics to calculate…more
metrics! (2/2)
• To calculate estimates for an (AdWords) campaign
plan, you only need to know three figures:
– budget
– goal CTR
– goal CPC
• Out of the previous figures, you can calculate other
metrics:
– clicks = budget / cpc
– impressions = clicks / ctr
8

9. (An example of the previous)
budget ctr cpc clicks impressions
250 0,05 0,2 1250 25000
9

10. In addition to being your way out of
complexity, metrics guide the optimization
process. In fact, they become the measures
for continuous improvement by optimization
activities.
10

11. Choosing the metrics a priori is crucial
• If you don’t rule metrics, they rule you! (remember
analysis paralysis)
• [JONI SHOWS: Facebook Ads reporting metric-o-
mania]
11

12. How to choose metrics?
There are several ways:
1. platform/channel specific
2. goal specific (derived from business objectives)
3. company type (age, business logic, cf. google
classification)
4. funnel stage (cf. elämyslahjat)
5. direct vs. indirect measures (cf. proxies)
12

13. Selection criteria for metrics
• is it actionable?
• is it useful?
• That’s it!
13

14. Engagement: is this good or bad?
14
(Metric: session duration)
Ask yourself:
• is this actionable?
• is this useful?

15. Engagement: is this good or bad?
15
(Metric: page depth)
Ask yourself:
• is this actionable?
• is this useful?

16. Neither is good! (I mean, the metrics ARE
good, but I don’t like what I’m seeing in the
data (data hurts :( )
• Solution ideas:
1. automatic lead magnet, when closing the page?
2. changes to product page template, encouraging to
discover gifts
– random product
– gift search engine in the left column
• You know you’ve chosen good metrics, when they
make you think!
16

17. The idea of ”proxy”
• Proxy is an indirect measure of a phenomenon
• We use proxies when the real data is not available
• For example, an impression is a proxy for capturing
attention (i.e. building ”awareness”)
• (This is similar to construct validity in statistics – how
do you measure e.g. trust?)
• Be careful with proxies, some of them approach
vanity metrics!
17

18. The proxy problem: How would you
measure ”engagement”?
Amunwa (2014):
– Time on Site
– Avg. Pages per Session
– Pageviews
– Return visits
– Site search keywords
– Submitting a contact form
– Downloading whitepaper / e-book
– Subscribing blog
– Reading one or more key blog posts related to the
offering
18

19. “How would you measure user engagement
at service like Dropbox?”
”Here are some metrics that I would want to know about to measure
user-engagement for a file storage and sharing platform similar to
Dropbox:
– Unique user accounts (UU): Technically, just the number of sign-
ups is not a user-engagement metric, but this is definitely an
important success metric, and a foundation stone for future user-
engagement.
– Active users (MAU and DAU): A health UU metric is good, but what
matters most is how many of these come back frequently. Monthly
Active Users (MAU) and Daily Active Users (DAU) are two most
popular measures, but depending on your needs, you may have a
different level of granularity.
– Upload frequency: For a service like Dropbox, I would like to know
how frequently users are actually uploading files to their accounts.
An even stronger indicator can be how many users have set-up
automatic sync between their devices and the service.”
19

20. “How would you measure user engagement
at service like Dropbox?”
”Here are some metrics that I would want to know about to measure
user-engagement for a file storage and sharing platform similar to
Dropbox:
– Content access frequency: Another good user-engagement metric is how
frequently the uploaded content is being accessed. This will include
downloads and users accessing their content directly on site and others
accessing users' shared contentthe ettteteee.
– Storage used: Another user-engagement metric specific to this case would be
how much storage is being used. If active users are using up a good share of
what is available to them, that is a good sign.
– Upgrades: With so many free storage options available, if your users decide
to pay you for more storage, they are definitely engaged!
– Referrals: If users consider your service good enough to be referred to their
friends, they definitely like your service. Even if they do this for additional free
storage, this is still a strong positive, as this indicates theirs interest in using
your service more.
– Device-mix: The mix of devices from which your service is being accessed
can tell you a lot about how they intend to use it, and how strong you can
expect the future engagement to be. I would definitely want the device mix to
have a good representation of mobile (smartphones/tablets), as this indicates
users' interest in accessing your service on the go.”
20

21. Engagement metrics differ by platform
• FB: likes, comments, engagement ratio
• GA: pages/visit, bounce rate, time on site, etc.
• own app: [custom]
21

22. Platform-specific metrics
• for example,
• Google: PageRank, Quality Score, search
impression share
• Facebook: Relevance Score, engagement ratio
– impSh = ad shown / all possibilities of showing the ad
– engRatio = users who shared, clicked, liked or
commented / all users who saw the post
22

23. Example of Quality Score (Google, 2010)
BEFORE QUALITY SCORE
Max. CPC CPC Position
Advertiser 1 0,4 0,3 1
Advertiser 2 0,3 0,2 2
Advertiser 3 0,2 0,1 3
Advertiser 4 0,1 - -
+QUALITY SCORE
Max. CPC QS Score Position
Advertiser 1 0,4 1 0,4 -
Advertiser 2 0,3 3 0,9 2
Advertiser 3 0,2 6 1,2 1
Advertiser 4 0,1 8 0,8 3
CLICK PRICE
Max. CPC QS Score CPC
Advertiser 1 0,4 1 0,4 -
Advertiser 2 0,3 3 0,9 (0,80/3) = 0,24
Advertiser 3 0,2 6 1,2 (0,90/6) = 0,15
Advertiser 4 0,1 8 0,8 Minimum price
23
Vickrey-style
second price
sealed auction
QS
changes
positions!
…and
prices

24. Relevance Score (Facebook, 2015)
• measures the potential of
the ad to succeed in a
chosen target group (1–10)
• good relevance score =
cheaper clicks and
impressions (and vice versa)
24

25. = if you know the systems, you gain a
competitive advantage
• however, there is a catch…
25

26. Optimization for platform metrics can be in
conflict with optimizing for business goals
26
Which ad is more successful?
Ad A Ad B
Quality score 10 3
CTR 10 % 3 %
Impressions 1000 1000
Clicks 100 30
Conversions 15 15
Revenue 1500 € 1500 €
Cost 500 € 150 €

27. Optimization for platform metrics can be in
conflict with optimizing for business goals
27
Ad A Ad B
Quality score 10 3
CTR 10 % 3 %
Impressions 1000 1000
Clicks 100 30
Conversions 15 15
Revenue 1500 € 1500 €
Cost 500 € 150 €
ROI 200 900%

28. The metric conflict can be seen as an issue
of local vs. global maximum
• This is a common computer science problem
– Platform-specific metrics: local maximum
– Business goals: global maximum
• It can be very very hard to achieve a global
maximum, but metrics should be chosen to support
the path towards it…
28

29. eCommerce metrics (Fellman, 2015)
• Visitors
• Conversion rate
• Average basket
• Margin
• Example (monthly sales):
• 100,000 x 0.02 x 100 € x 0.40 = 80,000 €
29
Visitors Conversion rate Average
basket
Margin

30. eCommerce metrics in GA (Promodo, 2013)

31. Metrics for newsletters
31
Subscribers
x delivery rate
x open rate
x click rate
x conversion rate
x avg. basket
x margin
= profit
(cf. Drake’s equation)

32. Metrics for newsletters: example
32
Subscribers (20,000)
x delivery rate (0.90 → 18000)
x open rate (0.25 → 4500)
x click rate (0.40 → 1800)
x conversion rate (0.03 → 54)
x avg. basket (100 € → 5400 €)
x margin (0.40)
= profit (2160 €)
(cf. Drake’s equation)
…I guess you already
see the importance of
volume & frequency.
damn them
numbers
change!

33. Company lifecycle (Wojcik, 2013)
1. “Infant: traffic, followers, subscribers, reviews,
social media shares
2. Adolescent: number of sales, revenue,
conversion rate, time on site, customer
satifaction
3. Mature: profit, retention length, churn rate,
revenue per customer, costs of goods sold,
societal/business impact”
First to build awareness, then to make sales, and
finally to optimize. (Follows the logic of company
building.)
33

34. Startup metrics (startups are cool, yej!)
“There are two types of startups out there:
– ones with very low CAC (usually because they offer
the product for free, and usually having users with low
willingness to pay) and
– ones with very high CAC (usually selling enterprise
software). The best position is to have an offering with
low CAC and strong willingness to pay (translating to
high CLV).”
• Startups usually focus on metrics measuring growth
and viability, as these are their goals. Corporations
tend to be more defensive and focus on efficiency
metrics and market share.
34

35. AARRR: Startup metrics for pirates
(McClure, 2007)
35

36. Business logic also matters. Let’s see
how…
36

37. Sanoma & Facebook both sell ads… Should
they optimize for the same metrics?
Why/why not?
• In my opinion, they should not
• The key difference is not the revenue model, but the
wider business logic, meaning that…
– Sanoma runs on editorial content and media sales
people
– Facebook runs user-generated content and on and
real-time bidding
• For both, engagement, impressions and revenue are
important. But their strategy to achieve them is
different, and so the metrics should be too.
37

38. The story of Kiosked
• CPC → CPM
• I call it ”funnel transferral”: moving up in the funnel
• from performance-based to awareness-based
• this is the major reason why e.g. Google cancelled
their affiliate program: impressions still sell!
38

39. Funnel stage
• Assign the following metrics to their proper funnel
stage: CPC, CPA, CPM, CPL
• Awareness
• Interest
• Desire
• Action
39

40. Funnel stage
• Assign the following metrics to their proper funnel
stage: CPC, CPA, CPM, CPL
• Awareness – CPM
• Interest – CPC
• Desire – CPL
• Action – CPA
40

41. Actionable metrics vs. vanity metrics
• “An actionable metric is one that ties specific and
repeatable actions to observed results.”
– Examples: conversion rate in a direct response
campaign, number of leads generated from a lead
magnet campaign, bounce rate of campaign + landing
page in comparison to site average
• “The opposite of actionable metrics are vanity
metrics (like web hits or number of downloads)
which only serve to document the current state of
the product but offer no insight into how we got
here or what to do next.”
– Examples: Facebook fans, ad impressions, even
visitors in some cases
41

42. Fine, you have chosen metrics! Now what??
• …well, you make a dashboard showing them.
42

43. Let’s build a dashboard!
The case is ElämysLahjat.fi, an ecommerce company
selling activity gifts.
1. First, choose metrics (how do we do this? how many
we take?)
2. Then, let’s build it in Google Analytics…
43

44. ”Ihmisten ymmärryskyky on aika limited.”
44