1. THE VALUE OF EXPERIMENTATION
A GUEST PANEL DISCUSSION
Host: Tremis Skeete, Principal Product Manager, nexTier Innovations
Sponsor: Alpha
PRODUCT THINKING NYC
3.29.2017

2. Ellen Chisa
VP of Product
Lola
Daniella Patrick
Innovation Lab Product Manager
Accenture
Jordan Bergtraum
Management Consultant
(formerly VP of Product Management at ServiceChannel)
Nis Frome
Co-Founder, Product Manager & Content Lead
Alpha

3. It’s a future the CEO of Netflix says he couldn’t even have
predicted five years ago, when the company was still
primarily shipping DVDs to customers in the U.S. while
grappling with its emerging video streaming service. Even
now, the Internet continues to scramble the game so fast
that Hastings said his company is racing to keep up with all
the changes.
When asked, he didn’t even want to hazard a guess as to
where Netflix might be five years from now.
“We don’t really know,” Hastings said. “It’s not Netflix that’s
making the changes. It’s the Internet. We’re figuring out
every year how to use the Internet to make a great
consumer experience. Every year is an experiment.”
Reference:
https://venturebeat.com/2016/05/07/netflixs-reed-hastings-and-ted-s
arandos-every-year-is-an-experiment

4. 1. What does it mean when you perform an experiment?
2. How can we communicate the value of experiments?
3. What performing experiments can do for you as designer?
4. What will users get out of being a part of an experiment?

5. What does it mean
when you perform an experiment?

7. “The objective of a Product Manager is to take an
idea and break it down into its core assumptions, and
find ways to test as cheap as possible.”

8. Experiments enable product managers to have autonomy. They
can more effectively push back against feature-creep and design by
committee by simply arguing that they are running an experiment
rather than launching a product.
Experiments drastically limit brand exposure to prospective
customers. A startup can launch a failed product and go back to the
drawing board without missing a beat. A major credit card company
cannot do the same without risking its reputation and damaging its
ability to launch trusted products in the future.
Experiments can transport user feedback from the end of the
product lifecycle to the beginning.
Reference: What Product Managers Need to Know About Minimum
Viable Experiments
By Mike Fishbein, Alpha
(http://insights.alphahq.com/minimum-viable-experiments)

9. “In the past few months, I've been trying to embrace
experiments. Instead of waiting until I had a bank of 30
[youtube] videos in place, I just wanted to do a test, see
how people responded, and then go from there.
Now, you're probably thinking ... don't you tell your clients to
start small, to launch fast, and to experiment? Yes, I do.
But, as many of you can probably relate, sometimes we
are our own worst client. Because as designers, we
want to make sure it's really well done.
So if you are struggling to act on your ideas then figure out
how you can start small, learn, and then go from there.”
Sarah Doody
User experience designer, consultant, and writer.
http://www.sarahdoody.com

12. “User Experience Design is a way to reduce risk,
improve productivity, and reduce cost.”
Reference: UX Briefing For Executives & Managers
(https://blog.prototypr.io/ux-briefing-for-executives-managers-8331160747f1)

14. [Product] Pricing is both the crudest, and most subtle, signal
that you can send to your customers, and they don’t really
think about their reaction to that signal – they feel it. The only
way to really understand your customers and design an
effective pricing strategy is to run experiments to work
out exactly how they’ll behave.
Reference: Designing Pricing Strategies: Don’t Ask – Experiment!
http://www.mindtheproduct.com/2016/11/designing-pricing-strategies-dont-ask-experiment/

15. A/B Testing
Comparing two versions of a web page to see which one performs
better. You compare two web pages by showing the two variants (let's
call them A and B) to similar visitors at the same time. The one that
gives a better conversion rate, wins!
Reference: The Complete Guide to A/B Testing (https://vwo.com/ab-testing/)

16. Reference: RankScience.com

17. Reference: RankScience.com

18. Reference: Casey Winters on Pinterest's Retention Wins, and Why 90% of SEO Advice is Wrong
(http://reforge.com/blog/casey-winters-pinterest-retention-seo)

19. A special thanks to our guest panelists:

20. A special thanks to Thor Ernstsson, Nis Frome,
and the team at…
Alpha
http://alphahq.com

21. NexTierInnovations.com

22. AlphaHQ.com

23. productthinkingnyc@gmail.com