The presentations of the keynote and various speakers at Analytics Academy on March 19, 2015.

1. WHY ARE WE TALKING
ABOUT ANALYTICS
TODAY?

2. Photo: Tobias Seiderer, Flickr

3. Photo: Scott Akerman, Flickr

7. INSIGHTS ACTIONS
YOU GET WHAT
YOU MEASURE

8. “I’m gonna write me a new
minivan this afternoon!”

9. WE DO ALL THIS AGAINST
A BACKDROP OF A
WORLD
OF DATA
Max Hancock for the Noun Project

11. Data for storytelling

14. Thank you.
Enjoy the day!

15. We Are What
We Measure
Steve Mulder

16. 16

17. 17

18. 18
What we measure
becomes
what we reward

19. 19
What we reward
becomes
what we do
and
what we are

20. 20
What are you
measuring now?

21. 21
In the world of NPR…
 Broadcast radio listening
 Digital analytics
 Donations and membership
 Underwriting revenue
Important things to measure, but do they tell the whole
story of what you are trying to do and be?
Are there things you’re doing less of simply because they’re
not as easy to measure?
We tend to measure what is familiar and
easy to count

22. 22
Community impact
 More informed public, influence on culture, policy
change, etc.
Audience insights
 Listener attitudes, unmet needs, changing behaviors
Digital
 Less familiar to many people in our organizations
 Constantly changing and hard to keep up with
 Overwhelming amount of data at our fingertips
It’s harder to measure what is less quantifiable,
newer, changing, or difficult to track

23. 23
Using data
effectively
is a lot about
mindset

24. 24

25. 25

26. 26

27. 27
Trap #1
We measure the wrong thing

28. 28
Trap #2
We make decisions by blindly following data

29. 29

30. 30
Trap #3
We become optimizers instead of free thinkers

31. 31
Optimization via A/B
or usability testing
The danger of optimizing to a local maximum
Local maximum

32. 32

33. 33
1. Data is the platform from which we leap

34. 34
For NPR, ratings and analytics data tell us about
listening

35. 35
Third-party research validates trends in media
fragmentation and digital adoption

36. 36
Channels proliferate, presenting both
opportunity and challenge
“Radio isn’t going away,
it’s going everywhere.”

37. 37
Our own research and personas reveal
changing consumer behaviors & opportunities

38. 38
It’s not about building things that users know
to ask for

39. 39
New digital interfaces in familiar locations
(connected car)

40. 40
Experiments with new listening experiences:
NPR One
Exploring
experiences
that meet needs
users can’t yet
articulate

41. 41
2. Data helps us know
how high we’ve jumped
and whether to keep going

42. 42
The Facebook experiment: Local station stories
geo-targeted on NPR’s Facebook stream

43. 43
Local stories saw consistently higher
engagement and grew local audience

44. 44
Pivot and double-down: We created a
workflow tool so we can scale this offering

45. 45
and
more about
continuous learning
3. It’s less about
achieving goals

46. 46
Yes, this sounds like Lean Startup thinking
Assumption/hypoth
esis
Minimum Viable
Product
Analytics,
research, testing

47. 47
Designing choice in the new NPR One app
Original design: Give
users more
control/choices
Hypothesis to test:
Fewer immediate
choices + simplicity
= longer listening
Results: Listening
time up 8%

48. 48
Walking the narrow path

49. 49
Ingredients of an
analytics culture

50. 50
Bring data to life, make it memorable and
actionable

51. 51
Bring data to life to make it memorable and
actionable

52. 52
Emphasize what matters most

53. 53
With great power comes great complexity.
Don’t let analytics feel like this.

54. 54

55. 55
Thanks for the metric.
Is that good or bad?
Context is the engine
of meaning

56. 56
Context can turn data
into insight and action

57. 57
Context over time, against averages, or against
custom cohorts

58. 58
Embed data in regular processes and habits

59. 59
Embed data in regular processes and habits

60. 60
Data without analysis
and storytelling is like
food without taste

61. 61
Fewer
dashboards,
more insights!

62. 62

63. Putting Data to Work
How Digital Analytics Informs Operations
Analytics Academy
March 19, 2015

64. Overview
What is your business goal?
How does digital communications
support that goal?
What are the goals of digital strategy?
What can you learn about user behavior to
determine if you are achieving that goal?
How can you use that knowledge to improve
your strategy?

66. Digital Recruitment Strategy

67. WEB & SOCIAL
Strategies for inbound traffic

68. Goals of the Website
• Help prospective applicants discover the
Harvard experience
• Hear authentic current student voices
• Understand the application process and
successfully complete the application
• Access information about application
process and admission decision

69. Analyzing Web Traffic
Google Analytics Monthly Report
Metrics Report Interpret

70. Interpreting the Metrics
• Seasonality - The Admissions cycle affects traffic to the
site. By understanding that traffic we can plan for server
capacity and uptime during our most critical time.
Regular Decision Early Decision

71. Understand Traffic Sources
• Traffic Acquisition - How does it change
over time?
January 1, 2014
January 1, 2015

72. Analytics-driven Content
• Landing pages: identify
where users are
entering your site, other
than the homepage
• Highlight key messages
• Make content easily
accessible across the
site

73. Understand Traffic Sources
• Mobile Traffic - Mobile traffic has grown from 15% in
2014 to 20.5% of all traffic in one year on smartphones
alone. On EA decision day, mobile traffic was nearly one
quarter of all traffic.

74. Mobile Strategy
• Optimize for mobile.
Assume users will view
on mobile first
–Responsive design. All
content must work on
mobile– web, email,
tour
–Test early and often
on many devices,
browsers

75. Social Media & Google
Analytics
Acquisition > Social > Users Flow

76. Social Media
Many sources, constant change
• What can we measure?
We can’t we measure?
• Aggregating social data
from various platforms
• Measuring engagement at
the post level to inform
content strategy

77. Social Analytics
Hootsuite
TwitterCounter
Iconosquare
Facebook
Tumblr
YouVisit

78. Informing Social Media Content
• Our audience are 16-18 years old.
• We are monitoring usage to go where they
are, rather than asking them to come to
us.

79. EMAIL & OUR SEARCH LIST
Strategies for outbound communication

80. Goals of Outreach
• Search list are highest priority users–
actively recruiting these prospective
applicants
• Targeted outreach focused on students
from diverse socioeconomic and cultural
backgrounds
• Drive traffic to our website, engage with
our students, start an application

81. Measuring Email Effectiveness
• Engagement is measured in
open rates, clickthroughs
• Deliverability, bounces, quality
of list
• Use UTM codes to link back
to website
• Send through Slate,
integrated with admissions
database. More on that later!

82. Google URL Builder
● utm_source: Identify the advertiser, site, publication, etc. that is
sending traffic to your property, for example: google, citysearch,
newsletter4, billboard.
● utm_medium: The advertising or marketing medium, for example: cpc,
banner, email newsletter.
● utm_campaign: The individual campaign name, slogan, promo code,
etc. for a product.
● utm_term: Identify paid search keywords.

83. Screenshot of Email Metrics

84. Email Report

85. Email Content
Optimize based on learning
• 70% of emails opened on mobile. Design
for mobile
– Responsive design
– Length of message
– Placement of links
• Know when to send based on audience –
3-5pm for our audience
• Test subject lines
– A/B testing

86. SLATE (DATABASE OR CRM)
converting prospects to applicants

87. CRM & Admissions Database
• Content strategy is based in goals
for recruitment & yield. CRM tracks
progress towards that goal.
• Also makes online reading of
applications possible
• Implemented this year- opportunity
to reevaluate strategies and better
analyze our data!

88. Student’s Path in App Cycle
• Applicant data is centralized throughout all
stages of the process– as a prospect,
inquiry, and applicant.
Prospect Inquiry Applicant

89. Getting Specific with our
Message
Recruitment communications for targeted
prospective students
Low socio-economic
status
seniors
Hispanic, African
American, Asian
American, Native
American
Low Application
schools
Low SES Juniors
HFAI UMRP HCC
First in the
family to graduate
from a four-year
college
First
Gen

90. Measuring Digital Recruitment
Track offline interactions
and aggregate data in
Slate
• Interactions with
current students
• History of engagement
is kept in the applicant
record
• Query and pull reports
based on interactions

91. Measuring Digital Recruitment
• Widgets, queries and reports demonstrate how
recruitment leads to applications.

92. 15797 17173
18498
20132
2018 2019
Searched
+8.71%
Not Searched
+8.83%
Number of Applicants to the
Class of 2018 and 2019

94. Thank you
Amy Lavoie | aalavoie@fas.harvard.edu
Victoria Marzilli | vmarzilli@fas.harvard.edu

95. Hit or miss?
How to tell, and what to do next

97. What does ‘success’ mean?
● Metrics paired to business goals
o Pageviews (reach)
o Time on site (attention)
o Email signups (loyalty, audience building)
● Anecdotal feedback
o internal (local leadership, faculty)
o external (media, influencers, awards)

98. What does ‘failure’ mean?
● Failed to meet expectations
● Elicited negative responses
● Opportunity cost

99. From the Harvard channels

100. From the Harvard channels

101. Campaign checklist
Using reporting from start to finish
1. Decide what’s worth sharing
2. Build consistent tracking methodology
3. Share prepared content
4. Implement analytics collection process
5. Put analytics through feedback loop
6. Evaluate effectiveness, determine
takeaways

102. Deciding what to share — and how to share it
Editorial decisions
*Editorial calendar
*Trending topics (Twitter, FB, Iconosquare)
Google trends
Topsy
Encore Alert and similar services
Format
Platform updates
Mobile proliferation
*Current research (Sharing Fast and Slow)

103. Choose content: Editorial

104. Choose content: Format
Sharing Fast and Slow
104
POST PRIMED FOR SHARING/REACH POST PRIMED FOR INTEREST/CLICKS
200 clicks to story176 retweets, 13 new followers

105. Build and execute consistent
reporting framework

106. Put analytics through feedback loop

107. Evaluate effectiveness, determine takeaways

108. Thank you!
Any questions?
Let’s talk!
rebecca_wickel@harvard.edu

109. The art and science of effective social
media in campaign messaging.
Painting By Numbers

110. Usable Knowledge #ThroughEducation
Launch/learning/measurement Launch/learning/measurement

111. Usable Knowledge #ThroughEducation
Launch/learning/measurement Launch/learning/measurement
CAMPAIGN PRIORITY (DEAN) CAMPAIGN ENGAGEMENT

112. Usable Knowledge: A History

113. Strategy

114. Strategy
Validate and experiment for hypothesis: Does social deliver?

115. Case Study
(Experiment)

116. Strategy

117. Strategy

120. Strategy
Social
-Toiling
-Training
-Tweeting
-Timing
(OBSERVE)
-Practice
-Partner
-Performance

124. Social is like Moneyball

125. Sabermetrics and scouting

126. Small tweaks
Primetime!
Platform?
Passion.

128. #ThroughEducation

129. Strategy
Social
-Toiling
-Training
-Tweeting
-Timing
(OBSERVE)
-Practice
-Partner
-Performance

130. National trending

131. #ThroughEducation

137. @HGSE Impressions over the
past month – focus on 9/19

138. Momentum…

139. Brad Paint By Numbers

140. Et cetera

148. thanks.

149. Copyright © President & Fellows of Harvard College
Behind the Retweet: Learning from Social Analytics
Harvard Analytics Academy, March 19, 2015
Robert Bochnak, Assistant Director, Alumni Marketing and Communications, Harvard Business
School, Harvard Analytics Academy, March 19, 2015

150. -Assistant Director of Alumni Marketing and
Communications at the Harvard Business
School where I manage the office’s
social media channels
-Have helped generate over 18,000 unique
alumni interactions via Twitter since January
2013
-Graduate of UMass-Amherst and the Tufts
University Graduate School of Arts and
Sciences
-Author, Social Media Matters blog,
http://robertbochnak.wordpress.com/
-Superhero sidekick of Aniceta
15
0
A little bit about me….

151. Our Approach: Twitter
15
1
Identify
EngageLeverage

152. Our Approach: Identify HBS alumni who are on
Twitter and add them to the appropriate list.
15
2

153. Our Approach: Engage alumni through retweets, original
tweets promoting their work, tweets sharing events, tweets
connecting alumni with each other, tweets to alumni based
on their interests, etc.
15
3

154. Our Approach: Leverage the relationships we have
developed with alumni so they participate in discussions.
15
4

155. But before all this engagement could happen we
needed to collect data on our alumni….
So we decided that each alumni interaction we had
on Twitter would be tracked on an excel
spreadsheet. This sheet would include information
drawn from the Twitter profile and alumni directory
file of each alumnus/a and would include
information such as city and state of residence,
interest, year of graduation, and section.
15
5

156. Data Collection
15
6

157. Data Use
With this data—as earlier slides conveyed—we’re
able to send targeted tweets to alumni based on
their interests.
15
7

158. Reporting and disseminating the data
-Each interaction with an alumnus/a is “scored” on
the tracking spreadsheet: 1-2 “touches” a month
equals “minimally engaged”; 2-4 “touches” a
month equals “moderately engaged”; and 5 or
more “touches” equals “highly engaged.”
-Interactions are retweets, favorited tweets, or
original tweets by individual alumni.
15
8

159. Reporting and disseminating the data cont.
15
9

160. Reporting and disseminating the data cont.
-We also measure the connections we help make
16
0

161. Reporting and disseminating the data cont.
-Twitter-based interactions are included in monthly
reports.
16
1

162. Reporting and disseminating the data cont.
16
2

163. Reporting and disseminating the data cont.
16
3

164. Thank You!
Questions?
16
4

165. YOUR GUIDE TO EMAIL METRICS
DETOX

166. HELLO!
Catherine Conway
catherine_conway@harvard.edu
@catjconway

167. EMAIL METRICS
Which camp do you fall into?

168. DOES YOUR
DASHBOARD
NEED A
DASHBOARD?

169. WHAT
DASHBOARD?

170. BEWARE OF METRICS BLOAT
○ Numbers replace your business goals
○ Confusion among team members
○ Lost alignment within organization
○ Spread thin aggregating metrics
○ Metrics no longer used to inform business
decisions

171. HOW DO YOU TREAT
METRICS BLOAT?
a. Invest in more fancy tools
b. Wither and die
c. Other

172. METRICS ARE
PEOPLE, TOO
Show some empathy, would you?

173. COMMUNICATION
IS ABOUT
BUILDING HUMAN
RELATIONSHIPS

174. “How are you going to build that
relationship?

175. PROVIDE VALUE
Events
○ Workshops
○ Lectures
○ Screenings
Interesting content
○ Articles
○ Online videos
○ Digests
Service
○ Online application
○ Customer support
Stewardship
○ Funds
○ Volunteer opportunities

176. MEASURE VALUE
Your business goals should align with how
you provide value.

177. SEPARATE EMAIL
METRICS FROM
SYSTEMS

178. THINK ABOUT METRICS IN
TERMS OF PEOPLE AND VALUE

179. EXAMPLE

180. EVENTS
Place your screenshot here

181. EVENTS
attendance
event
invitation
registration

182. Place your screenshot here
PUBLISHING CONTENT

183. PUBLISHING CONTENT
sharenewsletter read

184. STEWARDSHIP
Place your screenshot here

185. STEWARDSHIP
donationappeal awareness

186. Place your screenshot here
SERVICE

187. SERVICE
inquirypromotion awareness

188. ONCE YOU FIND
YOUR BEACON...

189. START SIMPLE
Email Attendees
December invitation 25
January invitation 30
February invitation 5
March invitation 25

190. BUILD TRENDLINES

191. ASK QUESTIONS
○ What happened in February?
○ Was the creative different?
○ Was the email sent at a different time?
○ Did the email go to the wrong list?

192. CHOOSE YOUR ADVENTURE
○ Wait for more ‘beacon’ data
○ Tweak your strategy (one variable at a
time)
○ Follow the trail with more analytics

193. CATHERINE’S
GUIDE
To Email Metrics Detox

194. 1. IDENTIFY BUSINESS GOALS
○ What value do you offer?
○ How are you building a relationship with your
audience?

195. 2. CONNECT YOUR COMMS
STRATEGY TO YOUR GOALS
What is the path to your end goal?

196. 3. TRACK THE END GOAL
Use the trendline as a proxy for success.

197. 4. WORK BACKWARDS
○ Ask questions about anomalies.
○ Dig in deeper, if you must.

198. 4. TWEAK
One variable at a time!

199. CATHERINE’S EMAIL
METRICS DETOX PLAN
1. Identify your
business goals.
2. Connect your
communications
strategy to your
goals.
3. Track the end goal.
4. Work backwards. 5. Tweak

200. THANKS!
Any questions?
@catjconway
catherine_conway@harvard.edu

201. Tracking the Life of an Event
Using Analytics

202. Event Life Cycle

203. Define Success and Plan
Accordingly

204. Measures of Success
• Registration
• Revenue
• Attendee satisfaction
• Target demographic representation
• Smooth event flow
• Numbers of first-time attendees
• Social amplification
• Smooth registration process
• Email engagement
• Social connections during/after event

205. Communications Strategy
• Who is your audience?
• What channels?
• How frequently?
• How long is your campaign?
• What is your content strategy?

207. Stay Nimble

208. Establish a Benchmark

209. Save the
Date
1st Invite 2nd Invite
Save the
Date
1st Invite 2nd Invite
Your
Harvard:
Mexico
Your
Harvard:
China

210. Tell the Whole Story

213. Dive Into Attendee Data

218. • Registration
• Revenue
• Attendee satisfaction
• Target demographics
• Effectively applying learnings from past events
• Smooth event flow
• Numbers of first-time attendees
• Social amplification
• Smooth registration process
• Email engagement
• Social connections during/after event
Measure Success

219. Thank you!
Questions?
kellie_celia@harvard.edu

220. Escape Excel Hell:
Make Your Reports
Memorable and Actionable

221. Hi!
I’m Mike Petroff
• Digital Content Strategist
• Loves GIFs (but not emojis)
• Twitter: @mikepetroff
• New Dad (excuse any puns)

222. IS THIS YOU?

223. IS THIS YOU?

224. IS THIS YOU?

225. IS THIS YOU?
GOOD
LUCK!

226. I’ve been there.
I’ve tried.

227. I’ve been there.
I’ve tried.
I’ve failed.

228. If you build it, they will come?

229. If you build it, they will come?

230. Data alone is not
inherently
helpful.
A brief history
lesson.

231. Data and maps.

232. London Underground Tube Map
▣1908
Source: datavizblog.com/

233. London Underground Tube Map
▣1908
Underground lines were
laid out geographically,
often superimposed over
the roadway of a city map.
Source: datavizblog.com/

234. London Underground Tube Map
▣Meet Henry Beck
an English engineering draftsman.

235. London Underground Tube Map
▣1933
Source: datavizblog.com/

236. London Underground Tube Map
▣1933
Beck believed that
passengers riding the
trains were not too
bothered about the
geographical accuracy,
but were more interested
in how to get from one
station to another, and
where to change.
Source: datavizblog.com/

237. Format matters.
What is the intent of the reader?

238. Bring focus.

239. Bring focus.

240. But…
We’re buried
in raw data.
How do we bring
focus to what’s
important?

241. ‘’
From 2013 to 2020, the
digital universe will grow by
a factor of 10 – from 4.4
trillion gigabytes to 44
trillion. It more than
doubles every two years.
Source: emc.com

242. ‘’
Most companies estimate
that they are analyzing a
mere 12% of the data they
have, according to a recent
study.
Source: Forrester Research

243. Fix the reporting
process.
Find what
matters.

244. The Failed Reporting Process
collect
report

245. The Failed Reporting Process

246. The Successful Reporting Process
collect
analyze
present

247. Collect
▣ Good data in, good data out
▣ Track consistently
▣ Share the collection process, and make it transparent

248. Analyze
▣ Provide insights
▣ Highlight opportunities
▣ Identify things that can be changed

249. Present
▣ Bring focus to the the most important thing
▣ Find the best delivery method
▣ Be consistent, but be memorable

250. Simplify
Show the goal’s performance.

251. Simplify

252. Simplify

253. Brand Examples
Annual Reports

254. Kickstarter

255. Gates Foundation

256. Insights
Show the reason it matters.

259. Daily Gazette report
What: Quick view of daily Gazette
email performance
Who: Senior leaders, Gazette
editors, Media Relations, Digital
Strategy
Why: Identify what resonates with
readers, engagement trends
Place your screenshot here

260. Daily Gazette report
Not very
helpful.

261. Daily Gazette report
Better, but
room to
grow.

262. Weekly Gazette web
traffic
What: Google Analytics dashboard
Who: Senior leaders, Gazette,
Media Relations, Digital Strategy
Why: Identify overall traffic, explain
sources of traffic, anomalies, review
owned traffic performance, social
impact

263. Weekly Gazette web traffic
Identifies not only top overall stories, but
highlights traffic generated from owned
sources like Harvard.edu, Daily Gazette
email, and Harvard’s social channels.

264. Weekly Gazette web traffic
▣ A written summary of select
referral traffic, including
influencers on social media
▣ Identifies media pickup and
impact on traffic

265. Monthly report
What: Combine metrics across
various digital platforms
Who: Senior leaders, HPAC, AA&D
Why: Analyze trends, large spikes
and dips, measure against previous
month/year

266. Monthly report
▣ Compare against previous
month and previous year, use
conditional formatting

267. Monthly report
▣ Provide most important findings and with each
report; explain changes and future opportunities

268. How to get started
▣ Define the goal and the audience
▣ Identify metrics that measure the goal
▣ Collect metrics consistently
▣ Simplify to show performance against the goal
▣ Present insights that are memorable and lead to
action
YOU control the context.

271. Thanks!
Michael_Petroff@harvard.edu
@mikepetroff