17. 17
Channels proliferate, presenting both
opportunity and challenge
“Radio isn’t going away,
it’s going everywhere.”
18. 18
Our own research and personas reveal
changing consumer behaviors & opportunities
19. 19
It’s not about building things that users know
to ask for
20. 20
New digital interfaces in familiar locations
(connected car)
21. 21
Experiments with new listening experiences
Exploring
experiences
that meet needs
users can’t yet
articulate
22. 22
2. Data helps us know
how high we’ve jumped
and whether to keep going
23. 23
The Facebook experiment: Local station stories
geo-targeted on NPR’s Facebook stream
24. 24
Local stories saw consistently higher
engagement and grew local audience
25. 25
Pivot and double-down: We created a
workflow tool so we can scale this offering
26. 26
3. It’s less about
achieving goals
and
more about
continuous learning
27. 27
Yes, this sounds like Lean Startup thinking
Assumption/hypoth
esis
Minimum Viable
Product
Analytics,
research, testing
28. 28
Designing choice in the new NPR app
Original design: Give
users more
control/choices
Hypothesis to test:
Fewer immediate
choices + simplicity
= longer listening
Results: Listening
time up 8%
Qualitative and quantitative
User interviews, surveys, third-party research
Usability testing, A/B testing, concept testing
http://www.flickr.com/photos/chicagobart/4650478963/
UX and design community: fundamental fear and skepticism of research & metrics
Can feel like data’s purpose is to mug design and steal creativity from us, restrict what we do
http://www.flickr.com/photos/spencerbbclark/6483073463/
False: Research = doing what users tell you to do
Driving directions, prescriptive, restrictive
Lycos marketing survey on colors
Sony boombox color
What users say isn’t always what they do
Or measuring only one thing
Everything hinges on what you decide to measure
http://www.adprint.ro/files/executions/8/356_topinterior_mag_sofa_low.jpg_900.jpg
Former Soviet Union: managers of glass plants rewarded by tons of sheet glass produced. Most plants produced glass so thick it was nearly opaque. Changed to square meters of glass. Led to glass so thin that it was easily broken.
*Unintended consequences*
For us: Session conversion vs longer-term loyalty effects
Upsells
Leads to false optimization, taking us in the wrong direction
What we choose to measure becomes what we reward -> what we do, what we are
http://compliance.saiglobal.com/community/know/blogs/item/4392-unintended-consequences-and-perverse-incentives
Instead of using our creative skills, it’s just about optimization and making very small adjustments
Becoming machinists, tending to the robots that make decisions
Cult of A/B testing
Doug at Google on testing blues
http://www.flickr.com/photos/seattlemunicipalarchives/4089729743/
What is it about A/B testing that makes it so easy to fall into this trap?
It’s the baseline we start from, not the end
It provides context and constraints that focus us before freeing us
http://www.flickr.com/photos/bfishadow/4407857237/
Opportunity: reaching new audience where they live
Challenge: More competitors for “earshare” than ever before
Channel usage by segment
Likelihood to consider public radio
Brand perception and affinity: NPR vs station vs program
BUT all this data doesn’t tell us exactly what to do
Smartphone app works with Ford Sync, voice control
App in private beta
Mobile version of NPR Infinite Player
Remixing public radio content, algorithms and editorial curation for a lean-back or lean-forward experience
Measure how high we've jumped (and whether it's worth continuing investment)
It should inspire us to reach higher
Without overly limiting our design choices
http://www.flickr.com/photos/10710442@N08/8265087493/in/photolist-dAmJyv-aoYjU9-aoYk3q-LJAUR
Initially: manual process – concierge MVP
Mindset: the goal is learning
There is no “finished”
http://www.deviantart.com/morelikethis/artists/198956566?view_mode=2
Flips the product development model on its head (build first, then figure out what to measure)
Findability and discoverability
Testing alone didn’t answer the question
But it gave us insights so WE could answer it
Falling into the cult of data
vs
Falling into the cult of ego
Without data, design is just guessing
Why is it so hard for teams to walk the narrow path?
Data doesn’t give us the answers, it helps us ask better questions