Fail Safe, Fail Smart, Succeed
Kevin Goldsmith, CTO
@KevinGoldsmith
@kevingoldsmith #failsafe
Invention requires failure
@kevingoldsmith #failsafe
–Albert Einstein
“Anyone who has never made a mistake has never tried
anything new.”
@kevingoldsmith #failsafe
“Failure is success training.”
-Kevin Smith
@kevingoldsmith #failsafe
@kevingoldsmith #failsafe
Creating a fail-safe environment
@kevingoldsmith #failsafe
Fail Smart
@kevingoldsmith #failsafe
–Mikael Krantz
"we want to be an internal combustion engine, not a
fuel-air bomb. Many small controlled explosions
propelling us in a generally ok direction, not a huge blast
leveling half the city”
Minimize the blast radius
@kevingoldsmith #failsafe
Find the fastest, cheapest path to learning.
Fail fast
@kevingoldsmith #failsafe
The lesson of Clippy
@kevingoldsmith #failsafe
– Thomas J. Watson
“If you want to increase your success rate, double your
failure rate.”
@kevingoldsmith #failsafe
@kevingoldsmith #failsafe
punish not learning from failure
Do not punish failure
@kevingoldsmith #failsafe
Each failure tells you something about
• your process
• your team
• your perception of your customers
• your understanding of your product
• yourself
Extract the lessons from failure
@kevingoldsmith #failsafe
Do project retrospectives with the intent to not only figure
out what happened on the project, but with the intent to
make future projects better.
Extract the lessons from failure
@kevingoldsmith #failsafe
Create a shared repository of the lessons learned.
Catalog the lessons
@kevingoldsmith #failsafe
Failures hidden or forgotten are failures that will be
repeated.
Catalog the lessons
@kevingoldsmith #failsafe
@kevingoldsmith #failsafe
@kevingoldsmith #failsafe
@kevingoldsmith #failsafe
@kevingoldsmith #failsafe
@kevingoldsmith #failsafe
– Bob Sutton
“The most creative people — and companies — don’t
have lower failure rates, they fail faster and cheaper, and
perhaps learn more from their setbacks, than their
competitors.”
@kevingoldsmith #failsafe
@kevingoldsmith #failsafe
Making Failure Safer
@kevingoldsmith #failsafe
Minimize the cost of failure
@kevingoldsmith #failsafe
Minimize the cost of failureInvestment
Time
Think it Build it Ship it Tweak it
@kevingoldsmith #failsafe
Minimize the cost of failureInvestment
Time
Think it Build it Ship it Tweak it
@kevingoldsmith #failsafe
Minimize the cost of failureInvestment
Time
Think it Build it Ship it Tweak it
Paper Prototypes
UI Mockups
User Testing
MVP
Employee
Testing
Gradual
Rollout
A/B Testing
User Testing
Analytics
Validation
A/B Testing
User Testing
Analytics
ValidationBeta

Testing
@kevingoldsmith #failsafe
The lesson of Spotify Now
@kevingoldsmith #failsafe
Expect failure all the time
@kevingoldsmith #failsafe
http://techblog.netflix.com/2010/12/5-lessons-weve-learned-using-aws.html
@kevingoldsmith #failsafe
Smaller/Micro Services, loosely connected, mean smaller
failures
Small systems have small failures
@kevingoldsmith #failsafe
Roll new things out over time, watch how your users react,
watch how your systems scale, watch your customer service
queues, watch your analytics, watch your crash logs.
Take your time
@kevingoldsmith #failsafe
Putting this into practice @ Avvo
@kevingoldsmith #failsafe
Creating a supportive culture
@kevingoldsmith #failsafe
Moving from planning around commitments to planning
around priorities
Creating a supportive culture
@kevingoldsmith #failsafe
Talk about when things don’t go well with an eye to future
improvement rather than blame
Creating a supportive culture
@kevingoldsmith #failsafe
Talk about when things don’t go well with an eye to future
improvement rather than blame
Creating a supportive culture
@kevingoldsmith #failsafe
Talk about when things don’t go well with an eye to future
improvement rather than blame
Creating a supportive culture
@kevingoldsmith #failsafe
Do retrospectives
In your retrospectives, think beyond the current project to
future projects.
The goal: continuous improvement.
Creating a supportive culture
@kevingoldsmith #failsafe
Monoliths are natural when a small team (or a startup) are
building a product.
Building lots of small services means building infrastructure
to support them.
The trick is knowing when it is time to make the transition.
Fail-safe infrastructure
@kevingoldsmith #failsafe@kevingoldsmith #failsafe
@kevingoldsmith #failsafe
–Henry Ford
“Failure is simply the opportunity to begin again, this
time more intelligently”
@kevingoldsmith #failsafe
@kevingoldsmith #failsafe
Questions?

Fail Safe, Fail Smart, Succeed