SMOKE TESTS
HUSTLE YOUR EXPERIMENTS w/o DEVS
@DistroDom - #WMD2015
WHO IS THIS GUY DOMINIC?
● Co-Founded GarmentValet
● Co-Founded Curebit / Talkable
Now I’m @DistroDom @500
Startups investing time and $ in
promising startups #500STRONG
@DistroDom - #WMD2015
PRODUCT GROWTH+
@DistroDom - #WMD2015
A little story...
@DistroDom - #WMD2015
MY OWN F*CKUP - one of the many ;)
@DistroDom - #WMD2015
BAD LINES FROM THE “OPS” APP
Sales
Cash
@DistroDom - #WMD2015
WAITING ON
ONE
MODULE
1 YEAR WASTED
@DistroDom - #WMD2015
WE’VE
ALL BEEN
HERE
@DistroDom - #WMD2015
Growth Is SLOWER than we want
@DistroDom - #WMD2015
Product Backlog is LONGER than we want
@DistroDom - #WMD2015
YOU EXPECT….
@DistroDom - #WMD2015
But you get….
@DistroDom - #WMD2015
#ProductGROWTHDilemma
@DistroDom - #WMD2015
KILL THAT SHIT
Before it KILLS YOU
@DistroDom - #WMD2015
SMOKE TEST
@DistroDom - #WMD2015
WTF is this Smoke Test?
WAY TO PROVE THAT YOUR PRODUCT
BACKLOG IS WORTH IT
task: validate do
@DistroDom - #WMD2015
@DistroDom - #WMD2015
- Alberto Savoia
LET’S RUN A PROCESS
1. What is the MINIMUM PROOF You Need?
2. How can you get that proof with NO ENGINEERS?
3. Drive the RIGHT TRAFFIC to your experiment
4. MEASURE Conversion
5. Analyze and OPTIMIZE
6. Make a DECISION
@DistroDom - #WMD2015
WHAT IS THE PROOF YOU NEED?
@DistroDom - #WMD2015
CAN I DO IT WITHOUT CODE?
@DistroDom - #WMD2015
AUDIENCE MATCH
@DistroDom - #WMD2015
KEEP SCORE
@DistroDom - #WMD2015
NEXT STEPS
@DistroDom - #WMD2015
DECIDE
@DistroDom - #WMD2015
ITERATE
CONSTANTLY
@DistroDom - #WMD2015
Vs. This Design
- Asked for email
and zip before live
- Had a <3% Email
Capture Rate
@DistroDom - #WMD2015
RESULTS
<= 30% of visitors
entered their
zipcode
&
25 % of people
entered their
email =>
&
10% shared
@DistroDom - #WMD2015
AN EXAMPLE FROM LAST WEEK
@DistroDom - #WMD2015
ADD TO BOX
@DistroDom - #WMD2015
“TYPICAL” PRODUCT WAY
BUILD a New
Feature Called
“Add To Box”
(~2 Weeks)
PLAN a New
Feature Called
“Add To Box”
(~1 Week)
TEST &
RELEASE
a New Feature
Called “Add To
Box”
(~1 Week)
ONE MONTH POTENTIALLY WASTED
@DistroDom - #WMD2015
“TYPICAL” PRODUCT WAY
LAUNCH
(THAT DAY)
PLAN
IDEA =>
HACK
(~1 hour)
MAKE $$$$
BOoooOM
:)
And the RESULTS….
@DistroDom - #WMD2015
FIRE
● 9.55% of Active Subs converted
● Overall monthly gross profit grew by
45% because of this feature
● Marginal gross profit per PijonBox, on
those boxes, grew over 357%.
●
@DistroDom - #WMD2015
BUT EVEN IF WE LOST….
● Saved a month of dev time
● Learned about our customers
● FELT GOOD AS A TEAM
●
WE WOULD HAVE STILL WON!!
@DistroDom - #WMD2015
dom@500.co
@DistroDom
www.facebook.com/dominic.coryell
HUSTLE WITH ME
@DistroDom - #WMD2015
GROWTH => PRODUCT
@DistroDom - #WMD2015

Smoke Tests - the hustle'rs guide to experimentation