SlideShare a Scribd company logo
Testing Your Baby
“How do I get Janice and the
luxr mafia off my back?”

                  Rick Boardman
                     @uxrick
                 luxr Jun/17/2011
Cast
                     Product/Biz              UX


                                              User
                       Eng


Thanks to Braden Kowitz for smiley graphics
Think                                                                                                                                     Usability?
             Intro to                          Intro to
(Speaker/                                                                        How to                          Lean UX                    Task
           generative                          Personas           Intersect                       What are       methods?                 Analysis?
 Lecture) interviewing       Distinguish
                                               & POVs
                                                                                Brainstorm
                                                                                                 you building
                             btw needs                            w/ Biz &                                                                Analytics
                                                                   Distro                       in next week
                             & insights                                                                                         Agile?
                                                                                                & next 3 mo?

               User                          Customer/                           Solution
                               Needs                               Uses                           Features      Prototype      Story/       Test
              Empathy                         Problem                            Ideation                                      Release

                    How do we test our       Definition



Make                   hypotheses?           Make persona
(Method/ Practice with                      of target user(s)                    Framing                        Wireframes
                                                                                                                                             Mock
 Activity) partner                                                              Brainstorm                                                 dashboard
                                              POV madlib
                               Practice                         Ecosystem &                                                  Storyboard
                                                                                                  Sketches
                             synthesizing                       Distribution                                                    Flows
                            empathy map                             map                                                                    How to test
                                                                                                                                           conversion
             Refine survey                  Compare personas                      Prioritize                      How to                      events?
              questions                     created by diff                    based on tech/                     test?                   Test with real
                                            team members                         biz/distro                                                  users &
Check                                                                                                                                        Product
                                                                                                                                          Design Guild
                                                                  What are                                                    How to
(Method/                                                           riskiest                       How to
                                                                                                                               test?
 Activity)                  Show partner                                                           test?
                                                                hypotheses?
                                map
                                                                  Interview
                                                                 real users?



                                 Thurs                                                           Fri                                           Sat
http://www.all-tribes.com/eng/catalog/rtfm-vol2-girls-mastic-p-51.html
How do I do Unix?




      http://www.all-tribes.com/eng/catalog/rtfm-vol2-girls-mastic-p-51.html
RTFM!
How do I do Unix?




Read The F****** Manual
      http://www.all-tribes.com/eng/catalog/rtfm-vol2-girls-mastic-p-51.html
Pic: http://www.flickr.com/photos/victoriapeckham/164175205
How do we
              test our
            hypotheses?




Pic: http://www.flickr.com/photos/victoriapeckham/164175205
How do we                           LTYFU!
              test our
            hypotheses?




   Listen To Your F****** Users
Pic: http://www.flickr.com/photos/victoriapeckham/164175205
Head in sand        Guess
  Ask Janice
                                                Ask Jason
        Which method?
Demo at              “Launch and cognitive walkthrough
 SXSW                    see”
                                        Analytics
 expert review      Usability testing     Friends n’ Family
 A/B testing
                    Dogfooding          Funnel analysis
   remote testing                  Contextual interviews
                     Beta
                            click/5sec tests
Focus groups     Surveys                         Show to
                                                David Pogue
Head in sand      Guess
  Ask Dave
                                         Ask Enrique
        Which method?
Demo at
 SXSW
             H ere’s a       dirty
                    “Launch and cognitive walkthrough
                        see”

            secUsability
 expert review
                         ...
                ret .testing Friends n’ Family
                                       Analytics


 A/B testing
                s shhhh Funnel analysis
            sssDogfooding
   remote testing                  Contextual interviews
                    Beta
                           click/5sec tests
Focus groups   Surveys                         Show to
                                              David Pogue
Head in sand               Guess
       Ask Dave
                                                                 Ask Enrique
                 Which method?
   Demo at
                 H ere’s a    dirty “Launch and cognitive walkthrough

         TheeUsability.testingsFriends kFamily
                    yeta.l.l. ucn’ *
    SXSW                                see”
                                                       Analytics
                s cr
     expert review
     A/B testing
                    s shhhh Funnel analysis
                sssDogfooding
        remote testing                                     Contextual interviews
                                     Beta
                                                 click/5sec tests
  Focus groups                Surveys                                Show to
                                                                    David Pogue
*More diplomatically: they have strengths and weaknesses
Head in sand               Guess
       Ask Dave
                                                                 Ask Enrique
                 Which method?
   Demo at
                 H ere’s a    dirty “Launch and cognitive walkthrough

         TheeUsability.testingsFriends kFamily
                    yeta.l.l. ucn’ *
    SXSW                                see”
                                                       Analytics
                s cr
     expert review
     A/B testing
                    s shhhh Funnel analysis
                sssDogfooding
        remote testing                                     Contextual interviews
                                     Beta
Focus groups Surveys                             Especiallytests one
                                                 click/5sec this
                                                                     Show to
                                                                    David Pogue
*More diplomatically: they have strengths and weaknesses
WTF? ... thats not very
     helpful ...
Sol  ution:
  Trian  gulate
Mult iple M ethods
               Analytics




Usability
              Offer Janice beer
A nalytic s suck


*Analytics are useful but have strengths and weaknesses
Why Analytics Suck
1. Tell you <what> not <why>
2. Hard to isolate specific effects
3. Which numbers do I pay
  attention to?
Oooo
AAARRRR, got a
  dashboard?




       Link: Metrics
       for Pirates
Example Conversion Dashboard
                 (note: *not* actuals… your mileage may vary)


  Stage                    Conversion Status                            Conv.   Est. Value
                                                                         %      (*not* cost)

Acquisition         Visitors -> Site/Widget/Landing Page                 60%        $.05
                (2+ pages, 10+ sec, 1+ clicks = don’t abandon)

Activation              “Happy” 1st Visit; Usage/Signup                  15%        $.25
              (clicks/time/pages, email/profile reg, feature usage)

Retention            Users Come Back; Multiple Visits                    5%         $1
                 (1-3x visits/mo; email/feed open rate / CTR)

 Referral                    Users Refer Others                          1%         $5
                       (cust sat >=8; viral K factor > 1; )

 Revenue                    Users Pay / Generate $$$                     2%         $50
                   (first txn, break-even, target profitability)




                                                              Link: Metrics for Pirates
Re mote (  autom  ated)
  Usa bility S ucks*

*Remote usability is useful but has strengths and weaknesses
Why Remote
     (Automated)
    Usability Sucks
1. Participant quality
2. Hard to dig deep
3. Need high-fidelity prototype
4. Don’t get to meet fun people
  (i.e. miss out on useful context)
Why Remote
      (Automated)
    Usability Sucks
1. Participant quality
     Useful tool but
2. Hard to dig deep
     dangerous too
3. Need high-fidelity prototype
4. Don’t get to meet fun people
  (i.e. miss out on useful context)
Live U sability
            S ucks*

*Live usability is useful but has strengths and weaknesses
Why Live Usability
      Sucks
1. Pricey? ($$$ and time)
2. Hard to cover large sample
3. You have to talk to people
Usability testing
• Watch your product in action
  with a target user
• Follow some tricks to make as
  objective as possible
           erview bonus : opportunity
  Live int
  for m ore generative research!
Usability Howto
1. Identify research goal -> Scenario
2. Recruit
3. Prepare interview outline, pilot test!
4. Test!

  -   Intro (welcome/confidential/reassure/thinkaloud)

  -   Record / take notes

  -   Debrief
5. Review, GET CODING, and iterate
Golden Rule of
   Usability
Golden Rules of
    Usability
Participant should do 90% of the talking



                  Shut up
Golden Rules of
    Usability
Participant should do 90% of the talking



     An Sh
       d ITE RuTEup
              At !
Things to look for

• Obvious usability issues
• Bugs
• Opportunities
• Don’t forget Delights!
Lean UX questions

1. What gear do I need?
2. How many users?
3.Which users?
observation
     test room                          room

                 1-way mirror
eyetracking
                                            UI
     usb mic

                                           Eng

Moderator      User
                                           PM




                                32
Observer /
                        notetaker /
                        prototype
                        debugger /
Moderator   User        coffee maker
                        (optional but nice)




                   33
Lean UX questions

1. What gear do I need?
2. How many users?
3.Which users?
Lean UX questions

  1. What gear do I need?
  2. How many users? >=0*
  3.Which users?
* 2: Ideally, small n and iterate. If no one available, do a
walkthrough!
Lean UX questions

   1. What gear do I need?
   2. How many users? >=0*
   3.Which users? Any*
* 2: Ideally, small n and iterate. If no one available, do a
walkthrough! Focus on core interaction flow.
* 3: Start with your neighbour or Janice, then focus.
Awesome
2 hour read
              http://www.sensible.com/rocketsurgery/index.html
Interview Practice
Interview Practice

        I’m super
      nervous ... am I
       being tested?
Interview Practice
 What do I do
   next?

                Should I sign up?
                  Is this OK?
Interview Practice

         I HATE IT.
      THIS IS &^**(}ED
         UP, MAN.
Interview Practice

      I LOVE THIS. ITS THE
        BEST THING EVER
Interview Practice

           <click>
      <click> <done>
         <silence>
Interview Practice
      Well it looks like you can
    use this menu to browse and
     you can use that button to
                signup
     <i.e. here’s a guided tour>
Interview Practice
      Weather wafflewaffle
     waffle Superbowl waffle
     waffle waffle waffle waffle
     waffle waffle waffle waffle
        waffle waffle waffle
           Superbowl ...
Interview Practice

           ?
Interview Practice

    Let me show you the
    product! What we’re
     trying to do here ...
Interview Practice

    This is my baby. What
    do you love about it?
Interview Practice

    This is my baby. What
    annoys you about it?
Interview Practice
      Weather wafflewaffle
     waffle Superbowl waffle
     waffle waffle waffle waffle
     waffle waffle waffle waffle
        waffle waffle waffle
           Superbowl ...
Interview Practice

    Lets do a focus group
Interview Practice

         ?
GO FORTH & LTYFU!



  Q & A!

More Related Content

What's hot

Design Scripts: Designing (inter)action with intent
Design Scripts: Designing (inter)action with intent Design Scripts: Designing (inter)action with intent
Design Scripts: Designing (inter)action with intent
Bas Leurs
 
2008 | A conversation about Interaction Design
2008 | A conversation about Interaction Design2008 | A conversation about Interaction Design
2008 | A conversation about Interaction Design
francesca // urijoe
 
MIT General Exams. Grounding interfaces: shifting the body boundaries, 2008.
MIT General Exams. Grounding interfaces: shifting the body boundaries, 2008.MIT General Exams. Grounding interfaces: shifting the body boundaries, 2008.
MIT General Exams. Grounding interfaces: shifting the body boundaries, 2008.
Cati Vaucelle
 
Reacting in real time
Reacting in real timeReacting in real time
Reacting in real time
cxpartners
 
Aum workshop paper_presentation
Aum workshop paper_presentationAum workshop paper_presentation
Aum workshop paper_presentation
Ahmad Ammari
 
2011 Taiwan UX Summit_Workshop B
2011 Taiwan UX Summit_Workshop B2011 Taiwan UX Summit_Workshop B
IDecideFast Final NSF I-Corp Presentation
IDecideFast Final NSF I-Corp PresentationIDecideFast Final NSF I-Corp Presentation
IDecideFast Final NSF I-Corp Presentation
Stanford University
 
Sandhya's portfolio
Sandhya's portfolioSandhya's portfolio
Sandhya's portfolio
Sandhya
 
Demola affective robotics_20120502
Demola affective robotics_20120502Demola affective robotics_20120502
Demola affective robotics_20120502
Rod Walsh
 
GdAM Midterm
GdAM MidtermGdAM Midterm
GdAM Midterm
pasteinplace
 
Midterm Presentation
Midterm PresentationMidterm Presentation
Midterm Presentation
pasteinplace
 
Ux bootcamp small
Ux bootcamp smallUx bootcamp small
Ux bootcamp small
Frank Kloos
 
Pedro Nakazato Andrade | Portfolio
Pedro Nakazato Andrade | PortfolioPedro Nakazato Andrade | Portfolio
Pedro Nakazato Andrade | Portfolio
Pedro Nakazato Andrade
 
Patent valuation using MDMP methodology
Patent valuation using MDMP methodologyPatent valuation using MDMP methodology
Patent valuation using MDMP methodology
Navneet Bhushan
 
App Design, from Concept to Completion
App Design, from Concept to CompletionApp Design, from Concept to Completion
App Design, from Concept to Completion
Howie Chang
 
Coherence, Engagement, and Usefulness as Sensemaking Criteria in Participato...
Coherence, Engagement, and Usefulness as Sensemaking Criteria in Participato...Coherence, Engagement, and Usefulness as Sensemaking Criteria in Participato...
Coherence, Engagement, and Usefulness as Sensemaking Criteria in Participato...
Simon Buckingham Shum
 
Fishing or a Z?: Investigating the Effects of Error on Mimetic and Alphabet D...
Fishing or a Z?: Investigating the Effects of Error on Mimetic and Alphabet D...Fishing or a Z?: Investigating the Effects of Error on Mimetic and Alphabet D...
Fishing or a Z?: Investigating the Effects of Error on Mimetic and Alphabet D...
Abdallah El Ali
 
Leading with language module 3 ocbi feedback
Leading with language   module 3 ocbi feedbackLeading with language   module 3 ocbi feedback
Leading with language module 3 ocbi feedback
crumbled
 

What's hot (18)

Design Scripts: Designing (inter)action with intent
Design Scripts: Designing (inter)action with intent Design Scripts: Designing (inter)action with intent
Design Scripts: Designing (inter)action with intent
 
2008 | A conversation about Interaction Design
2008 | A conversation about Interaction Design2008 | A conversation about Interaction Design
2008 | A conversation about Interaction Design
 
MIT General Exams. Grounding interfaces: shifting the body boundaries, 2008.
MIT General Exams. Grounding interfaces: shifting the body boundaries, 2008.MIT General Exams. Grounding interfaces: shifting the body boundaries, 2008.
MIT General Exams. Grounding interfaces: shifting the body boundaries, 2008.
 
Reacting in real time
Reacting in real timeReacting in real time
Reacting in real time
 
Aum workshop paper_presentation
Aum workshop paper_presentationAum workshop paper_presentation
Aum workshop paper_presentation
 
2011 Taiwan UX Summit_Workshop B
2011 Taiwan UX Summit_Workshop B2011 Taiwan UX Summit_Workshop B
2011 Taiwan UX Summit_Workshop B
 
IDecideFast Final NSF I-Corp Presentation
IDecideFast Final NSF I-Corp PresentationIDecideFast Final NSF I-Corp Presentation
IDecideFast Final NSF I-Corp Presentation
 
Sandhya's portfolio
Sandhya's portfolioSandhya's portfolio
Sandhya's portfolio
 
Demola affective robotics_20120502
Demola affective robotics_20120502Demola affective robotics_20120502
Demola affective robotics_20120502
 
GdAM Midterm
GdAM MidtermGdAM Midterm
GdAM Midterm
 
Midterm Presentation
Midterm PresentationMidterm Presentation
Midterm Presentation
 
Ux bootcamp small
Ux bootcamp smallUx bootcamp small
Ux bootcamp small
 
Pedro Nakazato Andrade | Portfolio
Pedro Nakazato Andrade | PortfolioPedro Nakazato Andrade | Portfolio
Pedro Nakazato Andrade | Portfolio
 
Patent valuation using MDMP methodology
Patent valuation using MDMP methodologyPatent valuation using MDMP methodology
Patent valuation using MDMP methodology
 
App Design, from Concept to Completion
App Design, from Concept to CompletionApp Design, from Concept to Completion
App Design, from Concept to Completion
 
Coherence, Engagement, and Usefulness as Sensemaking Criteria in Participato...
Coherence, Engagement, and Usefulness as Sensemaking Criteria in Participato...Coherence, Engagement, and Usefulness as Sensemaking Criteria in Participato...
Coherence, Engagement, and Usefulness as Sensemaking Criteria in Participato...
 
Fishing or a Z?: Investigating the Effects of Error on Mimetic and Alphabet D...
Fishing or a Z?: Investigating the Effects of Error on Mimetic and Alphabet D...Fishing or a Z?: Investigating the Effects of Error on Mimetic and Alphabet D...
Fishing or a Z?: Investigating the Effects of Error on Mimetic and Alphabet D...
 
Leading with language module 3 ocbi feedback
Leading with language   module 3 ocbi feedbackLeading with language   module 3 ocbi feedback
Leading with language module 3 ocbi feedback
 

Viewers also liked

Acw presentatie meten is weten do 14.06.2012
Acw presentatie meten is weten   do 14.06.2012Acw presentatie meten is weten   do 14.06.2012
Acw presentatie meten is weten do 14.06.2012
Fishtank
 
Talk to your Fantastic Users @Angelpadorg
Talk to your Fantastic Users @AngelpadorgTalk to your Fantastic Users @Angelpadorg
Talk to your Fantastic Users @Angelpadorg
Rick Boardman
 
Acw studiedag sociale media 24.11.2011
Acw studiedag sociale media   24.11.2011Acw studiedag sociale media   24.11.2011
Acw studiedag sociale media 24.11.2011
Fishtank
 
Skillteam workshop social media final v1.0 05.10.2011
Skillteam workshop social media final v1.0 05.10.2011Skillteam workshop social media final v1.0 05.10.2011
Skillteam workshop social media final v1.0 05.10.2011
Fishtank
 
Sàng lọc và chẩn đoán trước sinh bệnh thalassemia
Sàng lọc và chẩn đoán trước sinh bệnh thalassemiaSàng lọc và chẩn đoán trước sinh bệnh thalassemia
Sàng lọc và chẩn đoán trước sinh bệnh thalassemia
Võ Tá Sơn
 
Dgc segurança dos consumidores no carnaval 2014
Dgc   segurança dos consumidores no carnaval 2014Dgc   segurança dos consumidores no carnaval 2014
Dgc segurança dos consumidores no carnaval 2014
Caia Évora
 

Viewers also liked (6)

Acw presentatie meten is weten do 14.06.2012
Acw presentatie meten is weten   do 14.06.2012Acw presentatie meten is weten   do 14.06.2012
Acw presentatie meten is weten do 14.06.2012
 
Talk to your Fantastic Users @Angelpadorg
Talk to your Fantastic Users @AngelpadorgTalk to your Fantastic Users @Angelpadorg
Talk to your Fantastic Users @Angelpadorg
 
Acw studiedag sociale media 24.11.2011
Acw studiedag sociale media   24.11.2011Acw studiedag sociale media   24.11.2011
Acw studiedag sociale media 24.11.2011
 
Skillteam workshop social media final v1.0 05.10.2011
Skillteam workshop social media final v1.0 05.10.2011Skillteam workshop social media final v1.0 05.10.2011
Skillteam workshop social media final v1.0 05.10.2011
 
Sàng lọc và chẩn đoán trước sinh bệnh thalassemia
Sàng lọc và chẩn đoán trước sinh bệnh thalassemiaSàng lọc và chẩn đoán trước sinh bệnh thalassemia
Sàng lọc và chẩn đoán trước sinh bệnh thalassemia
 
Dgc segurança dos consumidores no carnaval 2014
Dgc   segurança dos consumidores no carnaval 2014Dgc   segurança dos consumidores no carnaval 2014
Dgc segurança dos consumidores no carnaval 2014
 

Similar to The RTFM of Usability at LUXR June 2011

500 Startups Lean UX Bootcamp
500 Startups Lean UX Bootcamp500 Startups Lean UX Bootcamp
500 Startups Lean UX Bootcamp
Enrique Allen
 
Advisoryboard2
Advisoryboard2Advisoryboard2
Advisoryboard2
garagenoda
 
More Elements of UX: real-world design deliverables
More Elements of UX: real-world design deliverablesMore Elements of UX: real-world design deliverables
More Elements of UX: real-world design deliverables
Peter Boersma
 
Design for People, Effective Innovation and Sustainability
Design for People, Effective Innovation and SustainabilityDesign for People, Effective Innovation and Sustainability
Design for People, Effective Innovation and Sustainability
Musstanser Tinauli
 
Akendi
AkendiAkendi
Akendi
adubidada
 
Everything you ever wanted to know about UX (*but were afraid to ask)
Everything you ever wanted to know about UX  (*but were afraid to ask)Everything you ever wanted to know about UX  (*but were afraid to ask)
Everything you ever wanted to know about UX (*but were afraid to ask)
Patrick Quattlebaum
 
Usability Training - UDSM 06/2010
Usability Training - UDSM 06/2010Usability Training - UDSM 06/2010
Usability Training - UDSM 06/2010
Marko Teräs
 
Lean UX, LUXr @ 500 Startups Edition
Lean UX, LUXr @ 500 Startups EditionLean UX, LUXr @ 500 Startups Edition
Lean UX, LUXr @ 500 Startups Edition
Janice Fraser
 
I decide fast final presentation
I decide fast final presentationI decide fast final presentation
I decide fast final presentation
Stanford University
 
Service Design
Service Design Service Design
Service Design
Francesco Sardu
 
Why UX Design Needs Content Strategy
Why UX Design Needs Content StrategyWhy UX Design Needs Content Strategy
Why UX Design Needs Content Strategy
Karen McGrane
 
Anchovi NSF final Presentation
Anchovi  NSF final PresentationAnchovi  NSF final Presentation
Anchovi NSF final Presentation
Stanford University
 
Qualitative techniques for assessing agricultural innovation systems
Qualitative techniques for assessing agricultural innovation systemsQualitative techniques for assessing agricultural innovation systems
Qualitative techniques for assessing agricultural innovation systems
SteveLeGrand
 
Designing an MVP that works for users (2 and 1/2 hours) @Lean UX NYC 2013
Designing an MVP that works for users (2 and 1/2 hours) @Lean UX NYC 2013Designing an MVP that works for users (2 and 1/2 hours) @Lean UX NYC 2013
Designing an MVP that works for users (2 and 1/2 hours) @Lean UX NYC 2013
Ariadna Font Llitjos
 
Tour of UX deliverables - dev/haag
Tour of UX deliverables - dev/haagTour of UX deliverables - dev/haag
Tour of UX deliverables - dev/haag
Peter Boersma
 
[Imr]week6
[Imr]week6[Imr]week6
[Imr]week6
jylee6977
 
Conversion Conference - What's in YOUR toolkit?
Conversion Conference - What's in YOUR toolkit?Conversion Conference - What's in YOUR toolkit?
Conversion Conference - What's in YOUR toolkit?
Craig Sullivan
 
Guerrilla Usability: Insight on a Shoestring
Guerrilla Usability: Insight on a ShoestringGuerrilla Usability: Insight on a Shoestring
Guerrilla Usability: Insight on a Shoestring
David Sturtz
 
Building an mvp that works for users
Building an mvp that works for users Building an mvp that works for users
Building an mvp that works for users
Ariadna Font Llitjos
 
An introduction to Hypothesis Based Testing
An introduction to Hypothesis Based TestingAn introduction to Hypothesis Based Testing
An introduction to Hypothesis Based Testing
STAG Software Private Limited
 

Similar to The RTFM of Usability at LUXR June 2011 (20)

500 Startups Lean UX Bootcamp
500 Startups Lean UX Bootcamp500 Startups Lean UX Bootcamp
500 Startups Lean UX Bootcamp
 
Advisoryboard2
Advisoryboard2Advisoryboard2
Advisoryboard2
 
More Elements of UX: real-world design deliverables
More Elements of UX: real-world design deliverablesMore Elements of UX: real-world design deliverables
More Elements of UX: real-world design deliverables
 
Design for People, Effective Innovation and Sustainability
Design for People, Effective Innovation and SustainabilityDesign for People, Effective Innovation and Sustainability
Design for People, Effective Innovation and Sustainability
 
Akendi
AkendiAkendi
Akendi
 
Everything you ever wanted to know about UX (*but were afraid to ask)
Everything you ever wanted to know about UX  (*but were afraid to ask)Everything you ever wanted to know about UX  (*but were afraid to ask)
Everything you ever wanted to know about UX (*but were afraid to ask)
 
Usability Training - UDSM 06/2010
Usability Training - UDSM 06/2010Usability Training - UDSM 06/2010
Usability Training - UDSM 06/2010
 
Lean UX, LUXr @ 500 Startups Edition
Lean UX, LUXr @ 500 Startups EditionLean UX, LUXr @ 500 Startups Edition
Lean UX, LUXr @ 500 Startups Edition
 
I decide fast final presentation
I decide fast final presentationI decide fast final presentation
I decide fast final presentation
 
Service Design
Service Design Service Design
Service Design
 
Why UX Design Needs Content Strategy
Why UX Design Needs Content StrategyWhy UX Design Needs Content Strategy
Why UX Design Needs Content Strategy
 
Anchovi NSF final Presentation
Anchovi  NSF final PresentationAnchovi  NSF final Presentation
Anchovi NSF final Presentation
 
Qualitative techniques for assessing agricultural innovation systems
Qualitative techniques for assessing agricultural innovation systemsQualitative techniques for assessing agricultural innovation systems
Qualitative techniques for assessing agricultural innovation systems
 
Designing an MVP that works for users (2 and 1/2 hours) @Lean UX NYC 2013
Designing an MVP that works for users (2 and 1/2 hours) @Lean UX NYC 2013Designing an MVP that works for users (2 and 1/2 hours) @Lean UX NYC 2013
Designing an MVP that works for users (2 and 1/2 hours) @Lean UX NYC 2013
 
Tour of UX deliverables - dev/haag
Tour of UX deliverables - dev/haagTour of UX deliverables - dev/haag
Tour of UX deliverables - dev/haag
 
[Imr]week6
[Imr]week6[Imr]week6
[Imr]week6
 
Conversion Conference - What's in YOUR toolkit?
Conversion Conference - What's in YOUR toolkit?Conversion Conference - What's in YOUR toolkit?
Conversion Conference - What's in YOUR toolkit?
 
Guerrilla Usability: Insight on a Shoestring
Guerrilla Usability: Insight on a ShoestringGuerrilla Usability: Insight on a Shoestring
Guerrilla Usability: Insight on a Shoestring
 
Building an mvp that works for users
Building an mvp that works for users Building an mvp that works for users
Building an mvp that works for users
 
An introduction to Hypothesis Based Testing
An introduction to Hypothesis Based TestingAn introduction to Hypothesis Based Testing
An introduction to Hypothesis Based Testing
 

More from Rick Boardman

Checklistux interaction flowandusability
Checklistux interaction flowandusabilityChecklistux interaction flowandusability
Checklistux interaction flowandusability
Rick Boardman
 
Balsamiq interviews
Balsamiq interviewsBalsamiq interviews
Balsamiq interviews
Rick Boardman
 
02 analysis
02 analysis02 analysis
02 analysis
Rick Boardman
 
Ttyfu v7-angelpadorg
Ttyfu v7-angelpadorgTtyfu v7-angelpadorg
Ttyfu v7-angelpadorg
Rick Boardman
 
Hyperscope UX analysis
Hyperscope UX analysisHyperscope UX analysis
Hyperscope UX analysis
Rick Boardman
 
PhD Defense: Improving tool support for personal information management
PhD Defense: Improving tool support for personal information managementPhD Defense: Improving tool support for personal information management
PhD Defense: Improving tool support for personal information management
Rick Boardman
 
No IM Please, Were Testing
No IM Please, Were TestingNo IM Please, Were Testing
No IM Please, Were Testing
Rick Boardman
 

More from Rick Boardman (7)

Checklistux interaction flowandusability
Checklistux interaction flowandusabilityChecklistux interaction flowandusability
Checklistux interaction flowandusability
 
Balsamiq interviews
Balsamiq interviewsBalsamiq interviews
Balsamiq interviews
 
02 analysis
02 analysis02 analysis
02 analysis
 
Ttyfu v7-angelpadorg
Ttyfu v7-angelpadorgTtyfu v7-angelpadorg
Ttyfu v7-angelpadorg
 
Hyperscope UX analysis
Hyperscope UX analysisHyperscope UX analysis
Hyperscope UX analysis
 
PhD Defense: Improving tool support for personal information management
PhD Defense: Improving tool support for personal information managementPhD Defense: Improving tool support for personal information management
PhD Defense: Improving tool support for personal information management
 
No IM Please, Were Testing
No IM Please, Were TestingNo IM Please, Were Testing
No IM Please, Were Testing
 

Recently uploaded

Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
David Wilson
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
Bhajan Mehta
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
DianaGray10
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
ZachWylie3
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
SynapseIndia
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
Zilliz
 
Patch Tuesday de julio
Patch Tuesday de julioPatch Tuesday de julio
Patch Tuesday de julio
Ivanti
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
alexjohnson7307
 
Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)
Debmalya Biswas
 
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
FIDO Alliance
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
SubhamMandal40
 
Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10
ankush9927
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
shyamraj55
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
shanihomely
 
Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3
DianaGray10
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
DianaGray10
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
SynapseIndia
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 

Recently uploaded (20)

Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
 
Patch Tuesday de julio
Patch Tuesday de julioPatch Tuesday de julio
Patch Tuesday de julio
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
 
Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)
 
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
 
Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
 
Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 

The RTFM of Usability at LUXR June 2011

  • 1. Testing Your Baby “How do I get Janice and the luxr mafia off my back?” Rick Boardman @uxrick luxr Jun/17/2011
  • 2. Cast Product/Biz UX User Eng Thanks to Braden Kowitz for smiley graphics
  • 3. Think Usability? Intro to Intro to (Speaker/ How to Lean UX Task generative Personas Intersect What are methods? Analysis? Lecture) interviewing Distinguish & POVs Brainstorm you building btw needs w/ Biz & Analytics Distro in next week & insights Agile? & next 3 mo? User Customer/ Solution Needs Uses Features Prototype Story/ Test Empathy Problem Ideation Release How do we test our Definition Make hypotheses? Make persona (Method/ Practice with of target user(s) Framing Wireframes Mock Activity) partner Brainstorm dashboard POV madlib Practice Ecosystem & Storyboard Sketches synthesizing Distribution Flows empathy map map How to test conversion Refine survey Compare personas Prioritize How to events? questions created by diff based on tech/ test? Test with real team members biz/distro users & Check Product Design Guild What are How to (Method/ riskiest How to test? Activity) Show partner test? hypotheses? map Interview real users? Thurs Fri Sat
  • 5. How do I do Unix? http://www.all-tribes.com/eng/catalog/rtfm-vol2-girls-mastic-p-51.html
  • 6. RTFM! How do I do Unix? Read The F****** Manual http://www.all-tribes.com/eng/catalog/rtfm-vol2-girls-mastic-p-51.html
  • 8. How do we test our hypotheses? Pic: http://www.flickr.com/photos/victoriapeckham/164175205
  • 9. How do we LTYFU! test our hypotheses? Listen To Your F****** Users Pic: http://www.flickr.com/photos/victoriapeckham/164175205
  • 10. Head in sand Guess Ask Janice Ask Jason Which method? Demo at “Launch and cognitive walkthrough SXSW see” Analytics expert review Usability testing Friends n’ Family A/B testing Dogfooding Funnel analysis remote testing Contextual interviews Beta click/5sec tests Focus groups Surveys Show to David Pogue
  • 11. Head in sand Guess Ask Dave Ask Enrique Which method? Demo at SXSW H ere’s a dirty “Launch and cognitive walkthrough see” secUsability expert review ... ret .testing Friends n’ Family Analytics A/B testing s shhhh Funnel analysis sssDogfooding remote testing Contextual interviews Beta click/5sec tests Focus groups Surveys Show to David Pogue
  • 12. Head in sand Guess Ask Dave Ask Enrique Which method? Demo at H ere’s a dirty “Launch and cognitive walkthrough TheeUsability.testingsFriends kFamily yeta.l.l. ucn’ * SXSW see” Analytics s cr expert review A/B testing s shhhh Funnel analysis sssDogfooding remote testing Contextual interviews Beta click/5sec tests Focus groups Surveys Show to David Pogue *More diplomatically: they have strengths and weaknesses
  • 13. Head in sand Guess Ask Dave Ask Enrique Which method? Demo at H ere’s a dirty “Launch and cognitive walkthrough TheeUsability.testingsFriends kFamily yeta.l.l. ucn’ * SXSW see” Analytics s cr expert review A/B testing s shhhh Funnel analysis sssDogfooding remote testing Contextual interviews Beta Focus groups Surveys Especiallytests one click/5sec this Show to David Pogue *More diplomatically: they have strengths and weaknesses
  • 14. WTF? ... thats not very helpful ...
  • 15. Sol ution: Trian gulate Mult iple M ethods Analytics Usability Offer Janice beer
  • 16. A nalytic s suck *Analytics are useful but have strengths and weaknesses
  • 17. Why Analytics Suck 1. Tell you <what> not <why> 2. Hard to isolate specific effects 3. Which numbers do I pay attention to?
  • 18. Oooo AAARRRR, got a dashboard? Link: Metrics for Pirates
  • 19. Example Conversion Dashboard (note: *not* actuals… your mileage may vary) Stage Conversion Status Conv. Est. Value % (*not* cost) Acquisition Visitors -> Site/Widget/Landing Page 60% $.05 (2+ pages, 10+ sec, 1+ clicks = don’t abandon) Activation “Happy” 1st Visit; Usage/Signup 15% $.25 (clicks/time/pages, email/profile reg, feature usage) Retention Users Come Back; Multiple Visits 5% $1 (1-3x visits/mo; email/feed open rate / CTR) Referral Users Refer Others 1% $5 (cust sat >=8; viral K factor > 1; ) Revenue Users Pay / Generate $$$ 2% $50 (first txn, break-even, target profitability) Link: Metrics for Pirates
  • 20. Re mote ( autom ated) Usa bility S ucks* *Remote usability is useful but has strengths and weaknesses
  • 21. Why Remote (Automated) Usability Sucks 1. Participant quality 2. Hard to dig deep 3. Need high-fidelity prototype 4. Don’t get to meet fun people (i.e. miss out on useful context)
  • 22. Why Remote (Automated) Usability Sucks 1. Participant quality Useful tool but 2. Hard to dig deep dangerous too 3. Need high-fidelity prototype 4. Don’t get to meet fun people (i.e. miss out on useful context)
  • 23. Live U sability S ucks* *Live usability is useful but has strengths and weaknesses
  • 24. Why Live Usability Sucks 1. Pricey? ($$$ and time) 2. Hard to cover large sample 3. You have to talk to people
  • 25. Usability testing • Watch your product in action with a target user • Follow some tricks to make as objective as possible erview bonus : opportunity Live int for m ore generative research!
  • 26. Usability Howto 1. Identify research goal -> Scenario 2. Recruit 3. Prepare interview outline, pilot test! 4. Test! - Intro (welcome/confidential/reassure/thinkaloud) - Record / take notes - Debrief 5. Review, GET CODING, and iterate
  • 27. Golden Rule of Usability
  • 28. Golden Rules of Usability Participant should do 90% of the talking Shut up
  • 29. Golden Rules of Usability Participant should do 90% of the talking An Sh d ITE RuTEup At !
  • 30. Things to look for • Obvious usability issues • Bugs • Opportunities • Don’t forget Delights!
  • 31. Lean UX questions 1. What gear do I need? 2. How many users? 3.Which users?
  • 32. observation test room room 1-way mirror eyetracking UI usb mic Eng Moderator User PM 32
  • 33. Observer / notetaker / prototype debugger / Moderator User coffee maker (optional but nice) 33
  • 34. Lean UX questions 1. What gear do I need? 2. How many users? 3.Which users?
  • 35. Lean UX questions 1. What gear do I need? 2. How many users? >=0* 3.Which users? * 2: Ideally, small n and iterate. If no one available, do a walkthrough!
  • 36. Lean UX questions 1. What gear do I need? 2. How many users? >=0* 3.Which users? Any* * 2: Ideally, small n and iterate. If no one available, do a walkthrough! Focus on core interaction flow. * 3: Start with your neighbour or Janice, then focus.
  • 37. Awesome 2 hour read http://www.sensible.com/rocketsurgery/index.html
  • 39. Interview Practice I’m super nervous ... am I being tested?
  • 40. Interview Practice What do I do next? Should I sign up? Is this OK?
  • 41. Interview Practice I HATE IT. THIS IS &^**(}ED UP, MAN.
  • 42. Interview Practice I LOVE THIS. ITS THE BEST THING EVER
  • 43. Interview Practice <click> <click> <done> <silence>
  • 44. Interview Practice Well it looks like you can use this menu to browse and you can use that button to signup <i.e. here’s a guided tour>
  • 45. Interview Practice Weather wafflewaffle waffle Superbowl waffle waffle waffle waffle waffle waffle waffle waffle waffle waffle waffle waffle Superbowl ...
  • 47. Interview Practice Let me show you the product! What we’re trying to do here ...
  • 48. Interview Practice This is my baby. What do you love about it?
  • 49. Interview Practice This is my baby. What annoys you about it?
  • 50. Interview Practice Weather wafflewaffle waffle Superbowl waffle waffle waffle waffle waffle waffle waffle waffle waffle waffle waffle waffle Superbowl ...
  • 51. Interview Practice Lets do a focus group
  • 53. GO FORTH & LTYFU! Q & A!

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. 1. None\n2. Doesn&amp;#x2019;t matter - start small. Key thing, iterate!\n3. Steve Krug: &amp;#x201C;Recruit loosely and grade on a curve&amp;#x201D;\n
  18. \n
  19. \n
  20. \n
  21. 1. None\n2. Doesn&amp;#x2019;t matter - start small. Key thing, iterate!\n3. Steve Krug: &amp;#x201C;Recruit loosely and grade on a curve&amp;#x201D;\n
  22. 1. None\n2. Doesn&amp;#x2019;t matter - start small. Key thing, iterate!\n3. Steve Krug: &amp;#x201C;Recruit loosely and grade on a curve&amp;#x201D;\n
  23. \n
  24. 1. None\n2. Doesn&amp;#x2019;t matter - start small. Key thing, iterate!\n3. Steve Krug: &amp;#x201C;Recruit loosely and grade on a curve&amp;#x201D;\n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. 1. None\n2. Doesn&amp;#x2019;t matter - start small. Key thing, iterate!\n3. Steve Krug: &amp;#x201C;Recruit loosely and grade on a curve&amp;#x201D;\n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. Reassure - no pressure, joke\n
  39. Reassure - no pressure, joke\n
  40. Imagine you are at home, what would you do next?\n
  41. Probe. Why?\n
  42. Probe. Why?\n
  43. probe. Thinkaloud\n
  44. I&amp;#x2019;d like US to finish on time ...\n
  45. I&amp;#x2019;d like US to finish on time ...\n
  46. I&amp;#x2019;d like US to finish on time ...\n
  47. I&amp;#x2019;d like US to finish on time ...\n
  48. I&amp;#x2019;d like US to finish on time ...\n
  49. I&amp;#x2019;d like US to finish on time ...\n
  50. I&amp;#x2019;d like US to finish on time ...\n
  51. I&amp;#x2019;d like US to finish on time ...\n
  52. I&amp;#x2019;d like US to finish on time ...\n
  53. \n