In addition to measuring newsroom and reporter performance, students at the Cronkite News ran a number of experiments on audience engagement during the Fall semester.

1. Cronkite News & Parse.ly
Cronkite News’ favorite Parse.ly experiments of Fall 2016
Cronkite News is the news division of AZ-PBS, part of the Walter Cronkite
School of Journalism & Mass Communication at Arizona State University.

2. We used Parse.ly to measure...
Newsroom performance
Our giant Parse.ly screen shows
all of our 100+ reporters and
producers how stories are
performing RIGHT NOW, and how
we can/should react in order to
grow in a meaningful way.
Weekly and semester reports of
our site help us understand
long(er) term performance and
evergreen-ness.
Reporter performance
All reporters created their own
author reports (sent to their email
each week) to track their personal
performance.
Special experiments
We used Parse.ly to help us
answer three questions.
Let’s explore them!

3. Question 1
Does placement on
our home page affect
story engagement?
Our website’s home page elevates three
stories each day: The story that appears in the
top hero position, and two just below it; seeing
other content requires a scroll. We sought to
know whether appearing prominently on our
home page affects engagement. Were users
more likely to spend time on our stories or
share them if they had prime home page real
estate?
Working with 40 stories across three weeks,
we tagged stories “hp” if they were placed in
one of the top three positions, and “nohp” if
they were not.

4. What we
learned
Home page placement
and social interactions
are correlated.
HP NO HP
AVG. TIME 0:55 0:52
SOCIAL INTERACTIONS
(MEAN)
222 58
SOCIAL INTERACTIONS
(MEDIAN)
31 22
Average engaged time for stories with home page
placement and those without were relatively similar.
Social interactions (likes, shares, etc.) saw a bigger
divide: Average interactions for “hp” stories were
much greater than those without. However, the mean
may have been skewed by viral-ish, highly newsworthy
election stories, so the median may be more notable
… and the “hp” median was higher, too. We want to
continue to examine this relationship in the
post-election news climate.

5. Question 2
Does video placement within
stories affect engaged time?
Depending on available multimedia assets, we
place videos in various locations throughout
our stories: sometimes at the top, sometimes
throughout the body, sometimes at the
bottom - and sometimes, stories have no
videos at all. We sought to learn whether we
could keep audiences engaged longer simply
by identifying the best position for a video and
optimizing toward that.
We added these tags to posts over the course
of three weeks: no video, top video (videos
that appear at the top of the page), middle
video (within the body of the page), and
bottom video (at the very bottom of the page).

6. What we
learned
Video placement
didn’t significantly affect
engagement time.
(Plus some hard lessons
about re-tagging and
re-crawling posts.)
AVG. TIME
NO VIDEO 1:24
BOTTOM 1:24
TOP 1:12
MIDDLE 1:06
The “time” metric may not be the best way to explore
this. Users spent more time reading “no video” stories
than stories with videos in any position. “Bottom”
video stories had the same average time, though we
published less of those … plus, logic follows that
users who reach the bottom of a story at all spend
more time on it; they may not watch the video at all.
High engaged time tends to more strongly correlate
with beat.
We also learned the hard way how to use Parse.ly to
re-crawl for new tags. We’ll try this again in spring!

7. Question 3
Do stock photos or original
photos perform better?
Cronkite News doesn’t have dedicated
photojournalists; reporters take their own
photos. And like many newsrooms, we can’t
always capture an idyllic image to draw
readers into every story, so we sometimes
turn to stock (or Creative Commons)
photography. We sought to understand
whether the photos we took (or the ones we
didn’t) performed better.
To evaluate performance, we used tags to
mark stories that contained original photos or
stock photos, then measured the tags in
Parse.ly.

8. What we
learned
Stock photos perform better.
(But that doesn’t mean we
should use them.)
ORIGINAL STOCK
AVG. VIEWS 132 185
AVG. USERS 108 165
AVG. TIME 0:49 0:40
Stock photos did a better job of bringing users to our
site; average views and users were higher for stories
that led with a stock image than stories that used our
own photography. This suggests not that we should
stop shooting, but rather that we must stress
higher-quality photography.
Indeed, once they’re actually on our site, the audience
seems to prefer original photography; stories with our
own photos (which obviously speak more to the news
we’re covering!) kept users engaged longer.

9. Question 4
Do our Friday shows perform
better when we use Parse.ly
metrics to pick the content?
Fact: You can use Parse.ly to inform your TV content!
We’re a converged newsroom; we air a news show on
PBS throughout Arizona every day at 5 with repeats at
11. Monday through Thursday, the show is live;
Friday’s show is our “Social Refresh” - a compilation
of the stories that performed best across social
media that week.
We typically choose Friday’s content based on stories
that earn the most likes in Twitter and/or Facebook.
This fall, we used Parse.ly to pick the content: We
chose the week’s top performers by Parse.ly’s “social
referrers” and “shares, likes, tweets & pins” metrics.
We used our daily Nielsen reports to analyze
performance of this semester’s Friday shows against
the performance of the spring semester’s Friday
shows.

10. What we
learned
Spring Friday shows
perform better
… and we should examine this
question differently.
SPRING ’16 FALL ’16
AVG.
HOUSEHOLDS
4,772 4,248
AVG. RATING .25 .22
AVG. SHARE .62 .56
Spring Friday shows performed better: They reached
more households, earned higher ratings and higher
ratings shares (among our timeslot competitors).
But reach, of course, is not the same as engagement;
instead, we should assess drop-off between quarter
hours (the difference between viewers of the top and
bottom halves of the show), which suggests whether
viewers like what they see! This requires some different
Nielsen data collection; we’ll initiate that in spring.

11. Moving forward...
1
We’ll tag better
Now that we have a
handle on Parse.ly, we’ll
be smarter about tagging
our content and be
diligent about re-crawling
pages we’ve tweaked to
ensure more precise
sampling.
2
We’ll measure
dynamically
Often, there’s not just
one perfect metric that
gives you “the answer.”
When exploring a single
question, we’ll aim to
look at groups of metrics
and the stories they
tell together.
3
We’ll focus on social
We’re particularly curious
about our social media
audiences; in spring, we
want to focus our
Parse.ly experiments on
uncovering the behaviors
of our social referrals
and leveraging that
knowledge for growth.

12. Above all, we LEARNED!
Here’s to measuring smarter in 2017!