TEAMS MEMES AND METRICS
 Connecting the Lean UX Dots

                          Jon Innes
               September 20th, 2012
The Origins of Lean

  Toyota’s Taiichi Ohno defined values and practices for
  removing three types of inefficiencies:
  • 無駄 = muda wasted outputs
  • 斑 = mura unnecessary inconsistencies
  • 無理 = muri wasted efforts

        Improving quality while decreasing costs
        Toyota rises from obscurity to leadership


UX INNOVATION LLC
                                                           2
Three Key Success Factors for Lean


  1. TEAMS: Small focused and cohesive
  2. MEMES: Ideas refined by iterative testing
  3. METRICS: Define goals and inform decisions




UX INNOVATION LLC
                                                  3
Teams

                    Small is better




                    Why?
UX INNOVATION LLC
                                      4
Dunbar’s Number




                              150
http://en.wikipedia.org/wiki/Dunbar%27s_number



UX INNOVATION LLC
                                                 5
Social Loafing




http://en.wikipedia.org/wiki/Social_loafing



UX INNOVATION LLC
                                              6
The Tragedy of the Commons




Garrett Hardin, Science, 162(1968):1243-1248
Image from: http://www.akbars.net/images/haiti-border.gif


UX INNOVATION LLC
                                                            7
Diffusion of Responsibility




UX INNOVATION LLC
                                8
Small Teams, Shared Values & Clear Goals




     http://flickr.com/photos/15489034@N00/420003710



UX INNOVATION LLC
                                                       9
Memes

                Iteration is key



UX INNOVATION LLC
                                   10
MEMES: Ideas That Evolve

  Geneticist Richard Dawkin’s term for an idea, behavior or style
  that spreads from person to person within a culture. They
  evolve iteratively and are:
  • Easily and often copied due to medium they exist in
  • Subject to mutation
  • Respond to selective pressures

          Product designs are essentially memes


UX INNOVATION LLC
                                                                    11
ITERATION: The Philosophy of Refinement




         Take your best shot, assess & try again

UX INNOVATION LLC
                                                   12
From Idea to Product Using Scrum
                                                              Daily Scrum
                                                               Meeting


                                                    24                  Potentially
       Product                   Sprint             hrs
                                                                        Shippable
       Backlog                  Backlog                                  Product
                                                      2-4               Increment
                                                     week
                                                     Sprint




Based on image from: http://www.mountaingoatsoftware.com/



UX INNOVATION LLC
                                                                                      13
PROBLEM: Some Ideas Are Duds




     http://en.wikipedia.org/wiki/File:Carbonfilament.jpg


UX INNOVATION LLC
                                                            14
Infinite Monkeys Typing




     http://en.wikipedia.org/wiki/Infinite_monkey_theorem

UX INNOVATION LLC
                                                            15
LEAN: Customer Development
                 Don’t build a company to serve
                  customers that don’t exist!




                        Get out of the building!

     Based on image from Steve Blank http://steveblank.com/


UX INNOVATION LLC
                                                              16
LEANER: Lean Startup
 Don’t build companies until you have a product!




                             Build & measure it!
     Based on image from Eric Ries http://theleanstartup.com/

UX INNOVATION LLC
                                                                17
LEANEST: Lean UX
  Don’t build products when prototypes will do!




                        Prototype & measure it!
     Based on image from Janice Fraser http://www.slideshare.net/clevergirl /

UX INNOVATION LLC
                                                                                18
Metrics

         Data beats opinion



UX INNOVATION LLC
                              19
Good Metrics Provide Clear Goals




     Define objective metrics tied to team goals
UX INNOVATION LLC
                                                   20
Fitness Functions

  A fitness function is an objective metric that is used to
  summarize how close a given design solution is to achieving a
  goal. Good fitness function metrics are:
  • Clearly defined, understood by team members
  • Regularly calculated to inform team decisions
  • Proven to actually measure what they are supposed to

        Metrics in Lean serve as fitness functions


UX INNOVATION LLC
                                                                  21
Dave Mclure’s Categories of Metrics
      Acquire                 Activate                 Retain                    Refer
•   Marketing          •    Get user to         •   Get user to        •    Get user to
    oriented, get user      engage with             integrate offering      recommend to a
    to know offering        product for initial     into their lives        friend
    exists                  1st use                 and use long term



                       These cover the entire
                       product lifecycle or UX


     Categories based on Dave McClure’s AARRR Startup Metrics for Pirates see:
     http://www.youtube.com/watch?v=irjgfW0BIrw


UX INNOVATION LLC
                                                                                             22
Making Measurable Progress?
  Candidates for design metrics to use in fitness functions:
       • # of interactions with user segments (personas)
       • Sign-ups
       • Return rates
       • Task completion rates
       • Time on task
       • Error rates
       • Satisfaction scores
       • Net Promoter Scores
        Avoid vanity metrics focus on learning
               how to solve problems
UX INNOVATION LLC
                                                               23
Winning Teams Keep Score & Learn Together
  Don’t be an IDIOT
      Individual
      Deciding on
      Intuition
                  Intuition & theory are great
      Or                      BUT
      Theory      learning requires data
                          GET FACTS!

UX INNOVATION LLC
                                                 24
Final thoughts

            It is not the strongest of
            the species that survives,
            nor the most intelligent. It
            is the one most adaptable
            to change.           Charles Darwin




UX INNOVATION LLC
                                                  25

TEAMS MEMES AND METRICS Connecting the Lean UX Dots

  • 1.
    TEAMS MEMES ANDMETRICS Connecting the Lean UX Dots Jon Innes September 20th, 2012
  • 2.
    The Origins ofLean Toyota’s Taiichi Ohno defined values and practices for removing three types of inefficiencies: • 無駄 = muda wasted outputs • 斑 = mura unnecessary inconsistencies • 無理 = muri wasted efforts Improving quality while decreasing costs Toyota rises from obscurity to leadership UX INNOVATION LLC 2
  • 3.
    Three Key SuccessFactors for Lean 1. TEAMS: Small focused and cohesive 2. MEMES: Ideas refined by iterative testing 3. METRICS: Define goals and inform decisions UX INNOVATION LLC 3
  • 4.
    Teams Small is better Why? UX INNOVATION LLC 4
  • 5.
    Dunbar’s Number 150 http://en.wikipedia.org/wiki/Dunbar%27s_number UX INNOVATION LLC 5
  • 6.
  • 7.
    The Tragedy ofthe Commons Garrett Hardin, Science, 162(1968):1243-1248 Image from: http://www.akbars.net/images/haiti-border.gif UX INNOVATION LLC 7
  • 8.
  • 9.
    Small Teams, SharedValues & Clear Goals http://flickr.com/photos/15489034@N00/420003710 UX INNOVATION LLC 9
  • 10.
    Memes Iteration is key UX INNOVATION LLC 10
  • 11.
    MEMES: Ideas ThatEvolve Geneticist Richard Dawkin’s term for an idea, behavior or style that spreads from person to person within a culture. They evolve iteratively and are: • Easily and often copied due to medium they exist in • Subject to mutation • Respond to selective pressures Product designs are essentially memes UX INNOVATION LLC 11
  • 12.
    ITERATION: The Philosophyof Refinement Take your best shot, assess & try again UX INNOVATION LLC 12
  • 13.
    From Idea toProduct Using Scrum Daily Scrum Meeting 24 Potentially Product Sprint hrs Shippable Backlog Backlog Product 2-4 Increment week Sprint Based on image from: http://www.mountaingoatsoftware.com/ UX INNOVATION LLC 13
  • 14.
    PROBLEM: Some IdeasAre Duds http://en.wikipedia.org/wiki/File:Carbonfilament.jpg UX INNOVATION LLC 14
  • 15.
    Infinite Monkeys Typing http://en.wikipedia.org/wiki/Infinite_monkey_theorem UX INNOVATION LLC 15
  • 16.
    LEAN: Customer Development Don’t build a company to serve customers that don’t exist! Get out of the building! Based on image from Steve Blank http://steveblank.com/ UX INNOVATION LLC 16
  • 17.
    LEANER: Lean Startup Don’t build companies until you have a product! Build & measure it! Based on image from Eric Ries http://theleanstartup.com/ UX INNOVATION LLC 17
  • 18.
    LEANEST: Lean UX Don’t build products when prototypes will do! Prototype & measure it! Based on image from Janice Fraser http://www.slideshare.net/clevergirl / UX INNOVATION LLC 18
  • 19.
    Metrics Data beats opinion UX INNOVATION LLC 19
  • 20.
    Good Metrics ProvideClear Goals Define objective metrics tied to team goals UX INNOVATION LLC 20
  • 21.
    Fitness Functions A fitness function is an objective metric that is used to summarize how close a given design solution is to achieving a goal. Good fitness function metrics are: • Clearly defined, understood by team members • Regularly calculated to inform team decisions • Proven to actually measure what they are supposed to Metrics in Lean serve as fitness functions UX INNOVATION LLC 21
  • 22.
    Dave Mclure’s Categoriesof Metrics Acquire Activate Retain Refer • Marketing • Get user to • Get user to • Get user to oriented, get user engage with integrate offering recommend to a to know offering product for initial into their lives friend exists 1st use and use long term These cover the entire product lifecycle or UX Categories based on Dave McClure’s AARRR Startup Metrics for Pirates see: http://www.youtube.com/watch?v=irjgfW0BIrw UX INNOVATION LLC 22
  • 23.
    Making Measurable Progress? Candidates for design metrics to use in fitness functions: • # of interactions with user segments (personas) • Sign-ups • Return rates • Task completion rates • Time on task • Error rates • Satisfaction scores • Net Promoter Scores Avoid vanity metrics focus on learning how to solve problems UX INNOVATION LLC 23
  • 24.
    Winning Teams KeepScore & Learn Together Don’t be an IDIOT Individual Deciding on Intuition Intuition & theory are great Or BUT Theory learning requires data GET FACTS! UX INNOVATION LLC 24
  • 25.
    Final thoughts It is not the strongest of the species that survives, nor the most intelligent. It is the one most adaptable to change. Charles Darwin UX INNOVATION LLC 25

Editor's Notes

  • #3 Ever write a usability report nobody read? That’s an example of muda. Did the report note UI inconsistencies due to a lack of communication on the team. That’s mura. Did you stay up all night write design specs nobody followed. That’s muri.
  • #4 In my experience, three factorsmust exist for teams to transition to Lean.
  • #5 When it comes to teams, small is better.
  • #6 While in today’s age of 500 Facebook friends, or something similar on LinkedIn, the evidence still supports Dunbar’s theory that an individual can really only have meaningful interactions with 150 other people in life. It’s a limit of cognitive function, just like the famous 7 plus or minus two. In fact, 7+/- 2 is the ideal size for a lean team.
  • #7 And when it comes to teams 1+1 does not equal 2. The bigger the team, the overall output tends to drop per person.
  • #8 Here’s another effect of group size. Thisis a slide showing the forest on a single island in the Caribbean, half is Haiti, known largely due to the recent tragic hurricane. The other half is the Dominican Republic. Note the state of the forest. Haiti’s side is barren. Due to the sad state of that country’s economy, and the fact that they have a high rate of poverty, people have burned or cut down the forest in Haiti. This is an example of the tragedy of the commons. The group took advantage of the resource, and it came depleted. Everyone thought about their own short term interest and now the country’s forest is destroyed. This wouldn’t happen in a small community or team.
  • #9 So the problem is similar in large companies. Does this look familiar?
  • #10 Small teams, with shared values, and clear goals tend to perform better. That’s what Agile is about, and that’s what lean is about as well. Get small and focused, and define clear goals.
  • #11 The next key is iteration of ideas.
  • #12 RichardDawkin’s invented the term meme to describe the idea of ideas that evolve. In our field, product designs are memes. For those of you in the dark, this is basically the scientific method. You may not believe in science, but I do…
  • #13 And if you want to improve your odds of succeeding, wouldn’t you want more than one shot?
  • #14 Scrum stories that go into the product backlog are a great of describing requirements. The Scrum process is essentially like an iterative algorithm, but like they say in computer science, if you put garbage in, you get garbage out. So how can we make this more lean?
  • #15 We avoid putting bad ideas through the full development iteration.
  • #16 There’s an old question, “Given an infinite amount of time, could a monkey type the works of Shakespeare? Probably not. And an infinite amount of monkey’s probably would not do much better.” Yet we often see companies hiring a team of developers to find a viable design for a product just by using A/B testing. If I were a VC, I wouldn’t fund that. Would you?
  • #17 There are some important things to ask about any product. One—Is there a customer for it? Lots of companies fail for this reason. No amount of iteration in development is going to help you here. Steve Blank’s advice. Don’t build a company to learn this. Get out of the building and validate you have a customer first.
  • #18 The Lean Startup guys are making a lot of progress here, and the good news is they have started to consider UX concepts. I’m even more excited that they’ve refocusedtheagile community on hypothesis testing, which is really core to both UX and Agile. This is leaner…but can we improve it?
  • #19 I would say yes. Thatbrings us to Lean UX. This slide is from Janice Fraser. Jeff just talked about this earlier. The key message here is you don’t always need to build a product to test an idea. It’s not lean to do it that way. I say think Minimum Viable PROTOTYPE rather than Minimum Viable Product, products are good for testing business models. But they aren’t necessarily required for testing designs ideas.
  • #20 This brings me to the last of my three factors, metrics or data.
  • #21 If a team is playing a sport, be it rugby or baseball, the team knows what the goal is. It’s clear if the team is doing well. It’s also clear to the team if they are getting better, or if they are better than the other team. Define clear goals, they will guide you along the way.
  • #22 There is this concept of a fitness function. Metrics serve as a test if the idea is fit, getting fitter, or fit enough to meet a goal.
  • #23 Dave McClure, Silicon Valley internet entrepreneur and investor has advised startups to think about 5 key types of metrics using his AARRR acronym. I’ve listed the first 4 here which are UX centric to give you a sense of why UX is not just about the product’s UI design.
  • #24 So ask yourself and your team. Are you making measurable progress? How do you know? Don’t kid yourself, activity does not equal accomplishment. In the early stage of a product design process, the key progress is about learning how well you are solving a problem for your users. If you don’t do that well, not much else matters.
  • #25 So you summarize, keep score, and learn together about what works and what doesn’t. Don’t be an IDIOT, nobody wants' one of those on their team. It’s OK to make decisions fast, but for key decisions, get data. Even if it’s after the fact, you don’t want to think you were right when you are wrong. Otherwise you don’t learn much, and you won’t do very well.