0
How Validation and Vision Co-exist
Andres Glusman
Chief Product-Revenue Officer
Meetup
DEC 2013
2
3
BEST THING ABOUT LEAN:
“Let’s test it!”
WORST
THING ABOUT LEAN:
“Let’s test it!”
MILLIONS OF WAYS TO FAIL

8
3 KINDS OF WINS

9
10
SMALL
WINS TIED
TO LIFE
SOURCE

11
12
BREAKTHROUGH
WINS

13
BREAKTHROUGH
WINS

14
MEANINGLESS WINS

15
Hunter Walk

16
LOW EFFORT

???

BIG
WIN
Stars

SMALL
WIN

HIGH EFFORT

Worthy Bets

???
Junk Food

Dogs
OUR LESSONS AT MEETUP

18
Craft a
design-driven
vision

19

Validate &
iterate using
usability lab

Break into
iterative tests
to (in)validate
visio...
20
21
WE

UX TESTING
On pace to do 400 - 500 sessions a year
Everything user-facing gets tested (at different stages)
Test 5 days a week (mo, t...
COSTS AS MUCH AS
2 OUTSOURCED USABILITY
STUDIES
“I thought there
would be more
colors“

“I thought there
would be more
colors“

“I thought there
would be more
colors“
IM from recent usability test

26
Shorten the time between someone
at Meetup wondering how someone
would react to something
and
actually watching someone re...
MALKOVICH
THE TENDENCY TO BELIEVE THAT
BIAS:
EVERYONE USES TECHNOLOGY
LIKE YOU DO
Links to our playbook:
glusman.com
30
WE

LIVE TESTING
2
1

32

3
33
34
WHAT ABOUT SMALL WINS?

35
BEWARE THE SMALL WIN
“This will only take a day or so”
“We saw a 12% lift in adoption from this flow”
FOCUS ON THE PRIZE
Run tests in core UX that could move vital metrics
Keep them super low-cost

37
SPLIT TEST
BANDIT
38
The Bandit automatically
optimizes tests in real-time
Allows us to run many variants on smaller
drivers with minimal human...
Go after big visions with
systematic validation

Ditch meaningless wins
Take some of the small wins
that supports the caus...
Andres Glusman

andres@meetup.com
@glusman

41
Upcoming SlideShare
Loading in...5
×

How Validation and Vision Co-exist by Andres Glusman

1,241

Published on

How Validation and Vision Co-exist by Andres Glusman

@glusman Meetup

The Lean Startup Conference 2013
http://leanstartup.co/

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,241
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
27
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • http://www.flickr.com/photos/cjcam/2662387745/lightbox/
  • http://www.flickr.com/photos/cjcam/2662387745/lightbox/There’s such a temptation to test that it absolves you of creative integrality and responsibility. More importantly you start chasing meaningless wins. One of the hardest lessons is learning what to test and what not to test. We don’t have a crystal ball. So how do you think about it.
  • Wins that give a little fuel for the mission AND are on route to the goal – usually related to business vital signs - adoption, revenue
  • Wins that give a little fuel for the mission AND are on route to the goal – usually related to business vital signs - adoption, revenue
  • Break thru win
  • Break thru win
  • Meaningless wins. That’s the problem with lean. We spend lots of times validating assumptions that don’t matter. Ironic, because I got into lean cause it helps you focus on what matter most.
  • Here’s the diagram of our testing setup with the changes I’ve just mentionedThere’s the user in the testing room, and me next to them with my laptop We’re running GTM, and I can monitor that, take notes and IM on my laptop Teams of developers now sit at their desks and watch the test on their own computers through the power of GoToMeeting, while the audio comes through their speakers (hopefully they are wearing earphones so they are not sharing the session with the whole office)
  • Meaningless wins. That’s the problem with lean. We spend lots of times validating assumptions that don’t matter. Ironic, because I got into lean cause it helps you focus on what matter most. Next time you hear speed to launch as the reason to do something… think of will
  • Wins that give a little fuel for the mission AND are on route to the goal – usually related to business vital signs - adoption, revenueIf we can find a $20 in our pocket we’ll take it. But not if we spend $50 to get it.
  • Transcript of "How Validation and Vision Co-exist by Andres Glusman "

    1. 1. How Validation and Vision Co-exist Andres Glusman Chief Product-Revenue Officer Meetup DEC 2013
    2. 2. 2
    3. 3. 3
    4. 4. BEST THING ABOUT LEAN:
    5. 5. “Let’s test it!”
    6. 6. WORST THING ABOUT LEAN:
    7. 7. “Let’s test it!”
    8. 8. MILLIONS OF WAYS TO FAIL 8
    9. 9. 3 KINDS OF WINS 9
    10. 10. 10
    11. 11. SMALL WINS TIED TO LIFE SOURCE 11
    12. 12. 12
    13. 13. BREAKTHROUGH WINS 13
    14. 14. BREAKTHROUGH WINS 14
    15. 15. MEANINGLESS WINS 15
    16. 16. Hunter Walk 16
    17. 17. LOW EFFORT ??? BIG WIN Stars SMALL WIN HIGH EFFORT Worthy Bets ??? Junk Food Dogs
    18. 18. OUR LESSONS AT MEETUP 18
    19. 19. Craft a design-driven vision 19 Validate & iterate using usability lab Break into iterative tests to (in)validate vision Repeat
    20. 20. 20
    21. 21. 21
    22. 22. WE UX TESTING
    23. 23. On pace to do 400 - 500 sessions a year Everything user-facing gets tested (at different stages) Test 5 days a week (mo, tu, wed, th, fri) Set up tests prior to knowing what will be tested Recruit participants from community and through craigslist Anyone can request to have something tested Follow each session with discussion & notes/video
    24. 24. COSTS AS MUCH AS 2 OUTSOURCED USABILITY STUDIES
    25. 25. “I thought there would be more colors“ “I thought there would be more colors“ “I thought there would be more colors“
    26. 26. IM from recent usability test 26
    27. 27. Shorten the time between someone at Meetup wondering how someone would react to something and actually watching someone react to something
    28. 28. MALKOVICH THE TENDENCY TO BELIEVE THAT BIAS: EVERYONE USES TECHNOLOGY LIKE YOU DO
    29. 29. Links to our playbook: glusman.com
    30. 30. 30
    31. 31. WE LIVE TESTING
    32. 32. 2 1 32 3
    33. 33. 33
    34. 34. 34
    35. 35. WHAT ABOUT SMALL WINS? 35
    36. 36. BEWARE THE SMALL WIN “This will only take a day or so” “We saw a 12% lift in adoption from this flow”
    37. 37. FOCUS ON THE PRIZE Run tests in core UX that could move vital metrics Keep them super low-cost 37
    38. 38. SPLIT TEST BANDIT 38
    39. 39. The Bandit automatically optimizes tests in real-time Allows us to run many variants on smaller drivers with minimal human intervention & debate
    40. 40. Go after big visions with systematic validation Ditch meaningless wins Take some of the small wins that supports the cause
    41. 41. Andres Glusman andres@meetup.com @glusman 41
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×