Do you know how well your content is performing? Is it achieving the goals you set for it? If it is, do you know why? And even more importantly, if it isn’t, do you know why not? It can be difficult to answer these questions. We know we want to measure something, but we might not know what to measure. We might not even be exactly sure how to articulate the answer we hope to get so that we can “ask the right question.” In this session, Andrea will describe a method for evaluating your problem space starting with the result you hope to achieve. She’ll discuss the merits of this approach, how it will help you to determine what data you want to collect, and how to best collect and analyze the right data to determine the effectiveness of various kinds of content.

1. @ a a m e s • # i n t e l c o n t e n t
Measuring the Effectiveness of Content
Creating a content measurement framework
Andrea L. Ames
Enterprise Content Experience Strategist, IBM
@aames
@ a a m e s • # i n t e l c o n t e n t

2. @ a a m e s • # i n t e l c o n t e n t
AGENDA
@ a a m e s • # i n t e l c o n t e n t
AGENDA
① What is “effective” content?
② Why measure content?
③ Gotchas to watch out for
④ Types of content measures and measurement
⑤ Building a closed-loop content evaluation framework
⑥ Q & A
@ a a m e s • # i n t e l c o n t e n t

3. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
A metric is something you
can measure… However, just
because they are
measureable, it doesn’t mean
those measures are
informative.
Jared M. Spool
Founding Principal, UIE
@ a a m e s • # i n t e l c o n t e n t

4. @ a a m e s • # i n t e l c o n t e n t
AGENDA
@ a a m e s • # i n t e l c o n t e n t
Moves customers
successfully
through their
journey
@ a a m e s • # i n t e l c o n t e n t

5. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Characteristics of effective content
Reach—number of potential customers exposed
Awareness—how well known you are
Engagement—connecting with potential customers
Satisfaction—how well you meet customer expectation
Value—difference between what customer gets vs. gives for product
Editorial quality—grammatical and stylistic accuracy
Usability—ease of use

6. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Measure content to
Tell a story … sell your content initiative
Demonstrate the value of your initiative
Strategize and make decisions throughout a project
Beginning: identify opportunity, prove the strategy is right
Middle: show incremental progress, course-correct
End: prove value and earn investment for the future
Transform opinion to fact … remove emotion from analysis

7. @ a a m e s • # i n t e l c o n t e n t
AGENDA
@ a a m e s • # i n t e l c o n t e n t
Telling the
right story … to
the right people
@ a a m e s • # i n t e l c o n t e n t

8. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Tell the right story to the right people
Business stakeholders
Make a direct connection between your content metrics and the
metrics that drive business
Prove the value of content and the content experience using
metrics that matter to business
Content stakeholders
Find out what matters most … define common ground

9. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Engaging content stakeholders
Who are their executives, sponsors, and stakeholders?
How is their progress or result being measured?
Who measures their performance?
Who funds them?
What matters to them?
Where do their goals align with yours? Build bridges!
Where do their goals conflict with yours? Build business cases!

10. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Example metrics for stakeholder conversations
Stakeholder Example metrics Example associated content teams Example content metrics
Marketing
Executive
 ROI
 Cost per lead
 Campaign performance
 Conversion metrics
 Web team
 Social team
 Event team
 Web traffic
 Click-throughs
 Likes and shares
 Conversions
 Collateral distributed
 Cost per unit produced
Sales
Executive
 Viable leads
 Sales growth
 Product performance
 Sales enablement
 Education & training
 Beta programs
 Proofs of Concept (PoCs) to sale
 Number of classes
 Beta program participants
 Cost per unit produced
Support
Executive
 Call volume
 Call length
 Customer sat.
 Ticket deflection
 Web support team
 Call center team
 Amount of web information produced
 Number of calls reduced
 Time of calls reduced
 Cost per unit produced
Development
Executive
 Dev cost
 Market share
 Lines of code
 Compliance
 Quality and test
 Product documentation team
 Developers who publish whitepapers
and case studies
 Product community forums and wikis
 Lines of text, number of pages, etc.
 Cost per unit produced
 Web traffic
 Number of forum participants

11. @ a a m e s • # i n t e l c o n t e n t
AGENDA
@ a a m e s • # i n t e l c o n t e n t
What
How many
Why
Compliance
@ a a m e s • # i n t e l c o n t e n t

12. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Types of measures
Quantitative
Describe the what, or how many of the what
Can be measured with numbers—
absolutely, mathematically
Examples: Conversions, likes, shares,
number of calls
Qualitative
Describe intangibles, like the why
Non-numerical
Examples: Sentiment, how important
something is, how much the respondents
like something, how likely they are to
recommend
Compliance items
Quantitative—you can count them
But you’re only counting “1”
More important is the perceived value of that “1”
Examples: Models applied, templates used,
processes followed
Heuristics
Rules of thumb
No one single “rule” or set of “rules”
Inspection method
Performed by “experts”
Examples: system visibility, use of affordances

13. @ a a m e s • # i n t e l c o n t e n t
AGENDA
@ a a m e s • # i n t e l c o n t e n t
Close the loop—
frameworks
make it easy!
@ a a m e s • # i n t e l c o n t e n t
Measure
periodically
for mid-
course
correction
Take final
measures to
determine
overall project
impact
Use final
measure as
baseline for
next project
Start with a
baseline

14. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Building a content evaluation framework
Before you begin—tell the right story to the right people
Identify what to measure and collect
Identify and begin managing (communicating to) stakeholders! *
Determine types of metrics to include in framework *
Build a framework, aka, scorecard, to evaluate how you’re doing
Normalize “results” to “scores” *
Categorize and weight metrics *
Create summaries *
Validate the framework *

15. @ a a m e s • # i n t e l c o n t e n t
AGENDA
@ a a m e s • # i n t e l c o n t e n t
Select
Normalize
Categorize
& weight
@ a a m e s • # i n t e l c o n t e n t

16. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Example: Select metrics
Quantitative
Time
Distance
Heartrate
Qualitative
How I feel at start of run
How I feel at end of run
How the weather conditions were
Compliance
Doping

17. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Example: Normalize—select a scale
How I feel at start of run
Horrible OK Good Great!
How I feel at end of run
Horrible OK Good Great!
What the weather conditions were
Freezing Cold Cool Warm/humid Hot/humid
How I feel at start of run
Horrible OK Good Great!
How I feel at end of run
Horrible OK Good Great!
What the weather conditions were
Horrible OK Good Great!
(Hot/humid or freezing  cool/dry)

18. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Example: Normalize—how good is “good?”
Time
Beginner: 15 minute mile
Intermediate: 11 minute mile
Athlete: 8 minute mile
Distance: % of goal
25-50%
50-75%
75-100%
Heartrate
Below fat-burning zone
Fat burning zone
Aerobic zone
Time
Horrible  implied
OK  Beginner: 15 minute mile
Good  Intermediate: 11 minute mile
Great  Athlete: 8 minute mile
Distance: % of goal
Horrible  implied
OK  25-50%
Good  50-75%
Great  75-100%
Heartrate
Horrible  implied
OK  Below fat-burning zone
Good  Fat burning zone
Great  Aerobic zone

19. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Example: Categorize and weight
How do combinations of metrics help tell a story?
Fitness = relationship across distance, time, and heartrate
Improved fitness =
Increased distance
Decreased time
Decreased heartrate

20. @ a a m e s • # i n t e l c o n t e n t
AGENDA
@ a a m e s • # i n t e l c o n t e n t
Apply to many
diverse subjects
Compare results
for consistency and
to uncover gaps
@ a a m e s • # i n t e l c o n t e n t

21. @ a a m e s • # i n t e l c o n t e n t
AGENDA
@ a a m e s • # i n t e l c o n t e n t
Q & A
@ a a m e s • # i n t e l c o n t e n t

22. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Resources
Jared Spool: KPIs are metrics, but not all metrics are KPIs from UIE Brainsparks
blog, Oct 5, 2012: http://bit.ly/VFYvF2
Bhapkar, Neil. 8 KPIs Your Content Marketing Measurements Should Include.
Content Marketing Institute. Web. 12 April 2013. http://bit.ly/Wnb7Cy
Klipfolio. The KPI Dashboard—Evolved. Web. 12 April 2013. http://bit.ly/LhzeL9
Muldoon, Pamela. 4 metrics every content marketer needs to measure:
Interview with Jay Baer. Content Marketing Institute. Web. 12 April 2013.
http://bit.ly/X8IvMJ
Thompson, Rachel. Stakeholder management: Planning stakeholder
communication. MindTools. Web. 12 April 2013. http://bit.ly/8UnUdj

23. @ a a m e s • # i n t e l c o n t e n t@ a a m e s • # i n t e l c o n t e n t
Importance of high-quality technical content
Based on survey of IBM clients and prospective clients (n=215) as of November 2014
Perception of the company
(84.9%)
Satisfaction with the product/solution
(92%)
Perception of product/solution quality
(96.3%)
Initial purchase decision
(87.3%)

24. @ a a m e s • # i n t e l c o n t e n t
Andrea L. Ames
Enterprise Content
Experience Strategist
IBM
@ a a m e s • # i n t e l c o n t e n t