SlideShare a Scribd company logo
UX SMACKDOWN!
Usability Testing Techniques in the Ring

                      #UXSmackdown
Aviva Rosenstein             Rebecca Sherrill
Sr Mgr, User Research        UX Lead
Salesforce.com               Beaconfire.com
@uxresearch                  @beccas



Kyle Soucy                   Michael Summers
Founding Principal           Global User Experience, Paypal
UsableInterface.com          UserResearch.com
@kylesoucy                   @ez2use



John Whalen
UX Lead & Founder
Brilliant Experience.com
@johnwhalen
                           #uxSmackdown


                                                #UXSmackdown
ROUND 1


      Focus   Groups
             Vs.
      Fie ld Res earch

                         #UXSmackdown
Focus Groups


               #UXSmackdown
?
Why are we proponents of focus groups?


                                         #UXSmackdown
1. Focus groups do not have to be this...

Photo	
  copyright	
  AMC                   #UXSmackdown
1. Focus groups do not have to be this...

Photo	
  copyright	
  AMC                   #UXSmackdown
2. You don’t have to dress up like a $#%^&*@$!!
#fieldstudyninja to get good data.

                                              #UXSmackdown
3a. Focus groups can be “in situ”


                                    #UXSmackdown
3b. Focus groups can be remote

                                 #UXSmackdown
4. Focus groups can have individual assignments before
group discussion to produce better data.

                                               #UXSmackdown
Drip Coffee?!                     No M&M’s?
      I must make a                        Totally
      Starbucks run!                    unacceptable!
                       I can’t work
                       under these
                        conditions!
     Diet Pepsi is
                                     ...and only 12
   no substitute for
                                    menu choices?!!
      Diet Coke!
                                   Who has the Zagats!



Finally, think of what you’d miss behind the glass...
                                         Check out skydeckcartoons.com for more.


                                                             #UXSmackdown
Summary
Reasons for conducting Focus Groups:

1. Focus groups can be “small group
workshops,” not “focus groups”.

2. You don’t have to dress up like a field
studies ninja.

3. Focus groups can be “in situ” or
remote.

4. Focus groups can have
individual assignments before
group discussion to improve data
quality.



                                             #UXSmackdown
Field Research


                 #UXSmackdown
“What	
  people	
  say,	
  
what	
  people	
  do,	
  and	
  
what	
  they	
  say	
  they	
  do	
  
are	
  entirely	
  different	
  
things.”	
  
                —Margaret Mead


                                        #UXSmackdown
Focus groups aren’t enough:
If you want to find out:
 •   What users want & need
 •   What they’re doing now
 •   How they feel about it
 •   What you can improve
Immerse yourself in the user’s world




                               #UXSmackdown
ROLE: Business Owner
TASK: Approve visual design direction

CONTEXT: Waterfall dev process. Supervises multiple
product managers. Frequently Approve visual design direction
                      TASK: mobile; uses iPhone.

CHARACTERISTICS: Short aention span.
Under significant time pressure.               PM
                          ID
Focuses on visuals and metrics.                                BO

CONTENT CRITERIA: Brief, clear presentation in common
formats consumable on mobile devices

                                           Dev Mgr
                       VzD



                                                          #UXSmackdown
Stakeholder participation in analysis =
   everyone owns the study findings




Photo	
  by	
  Jane	
  Mejdahl,	
  	
  used	
  under	
  CC	
  BY-­‐SA	
  2.0.	
     19
Focus group falsehoods:
 •   Human memory fail
 •   Suggestibility
 •   Social norms
 •   Self-enhancement bias
 •   Self serving facilitators




                                 #UXSmackdown
Imagine you’re asking
about



                        #UXSmackdown
Oh!
 How do you
                        Thank
like the gift I
                         you!
 made you?

                  (Ugh, it’s
                  horrible.)


                               #UXSmackdown
How do you             Oh!
like the app          Great!
we built you?

                (Yuck, it’s
                 useless.)


                              #UXSmackdown
Our study found that
focus group participants
are more generous,
more law abiding, and
floss more often.




                           #UXSmackdown
Summary
Field studies let you:
 •   Understand user needs
 •   Uncover tacit knowledge that isn’t
     consciously available
 •   Find deltas between how users
     think they do something and their
     performance
 •   Develop actionable insights
 •   Identify opportunities for
     innovation


                                          #UXSmackdown
ROUND 1



Make Some Noise!

             #UXSmackdown
ROUND 1



     Questions

                 #UXSmackdown
ROUND 2


       Eye Tr acking
            Vs.
                mote T esting
Un moderated Re

                        #UXSmackdown
Eye Tracking


               #UXSmackdown
#UXSmackdown
Unmoderated
Remote Usability Testing:



           ?

                       #UXSmackdown
#UXSmackdown
Unmoderated Usability Testing




                          #UXSmackdown
Benefits



           #UXSmackdown
Test hundreds of people simultaneously while keeping
them in their own natural environment.




                                            #UXSmackdown
Test multiple websites simultaneously.




                                         #UXSmackdown
Reduced costs... depending on the tool you use.




                                              #UXSmackdown
Great way to plant the seed of UCD methodologies
and introduce usability testing into a company.




                      UX
                                            #UXSmackdown
Fewer logistics to manage and no need to worry about
no-shows and getting last-minute replacements.




                                            #UXSmackdown
Fast Results




               #UXSmackdown
In-context Feedback

                                  “...I’m only offered a
                               search by model number.
                                  How ridiculous.”



                “...I would like to see the
            products availability and delivery/
                shipping information listed.”




Image source: Loop11
                                                           #UXSmackdown
The danger...

    I wonder
how a real customer
would actually use this
     website?


                          Panelist




                                     #UXSmackdown
The danger...


          BLOCKED
BLOCKED




                                    #UXSmackdown
There are no
  shortcuts
to conducting
good research.
             #UXSmackdown
YO U B E
            THE JU
                  DG   E!

Good?                  Evil?

                                   #UXSmackdown
                   Photo	
  copyright	
  New	
  Line	
  Cinema
ROUND 2



Make Some Noise!

             #UXSmackdown
ROUND 2



     Questions

                 #UXSmackdown
ROUND 3


          elity Proto  types
H igh Fid
              Vs.
         Itera tive T esting
  Rapid
                         #UXSmackdown
High Fidelity
 Prototypes

                #UXSmackdown
Why #HighFidelityRules...   ...and #IterativeTestingFails




Photo	
  copyright	
  Apple                                               #UXSmackdown
Weight                2,061       2,954

      Drive Train           Rear        Rear

      HP                    350         320

      Torque                292         273

      Top Speed             156         171

      0-60                   5.3         4.9




Why High-Fidelity Prototypes?
1. Everything looks great on paper...

                                                #UXSmackdown
...but sometimes the final experience is not the same.


                                                 #UXSmackdown
...but sometimes the final experience is not the same.


                                                 #UXSmackdown
In today’s swipe and swish world you need the full experience.



                                                                 #UXSmackdown
Pause
               Suppli   Perfor         HIGHSCORE       0
                es      mance
                                           SCORE       0




2a. Iterative testing can’t tell you if you’re creating flow.


                                                     #UXSmackdown
2a: Iterative testing can’t tell you if you’re creating flow.


                                                     #UXSmackdown
You should see the front.


        blue fade                               yellow fade




2b: Iterative testing can’t tell you if you’re creating
persuasive design.

                                                          #UXSmackdown
2b: Iterative testing can’t tell you if you’re creating
persuasive design.

                                                     #UXSmackdown
3. High fidelity prototypes identify the full implications of
“stakeholder design”

                                                    #UXSmackdown
#UXSmackdown
John,
     couldn’t you iteratively test
something simple, like a login without a
       high fidelity prototype?!




                                           #UXSmackdown
John,
     couldn’t you iteratively test
something simple, like a login without a
       high fidelity prototype?!




             YES
                                           #UXSmackdown
...but my argument is that increasingly we need the
full experience to understand a design’s success.

                                                #UXSmackdown
Final note: I too find great value in iterative design in the
right circumstance!

                                                     #UXSmackdown
Summary
Reasons for High-Fidelity
Prototypes:

1. In today’s swipe and swish world
you need the full experience.

2. You can’t really “iteratively test”,
persuasive or multi-channel designs.

3. High-fidelity prototypes identify the
full implications of
“stakeholder design”.


                                           #UXSmackdown
High Fidelity         Rapid Iterative
 Prototype      vs.     Usability
   Testing               Testing




                               #UXSmackdown
Low-Fidelity Prototyping




                    viously
           Ad d pre




                                      rk
                              N etwo
                                     ity
                               Activ

                                           #UXSmackdown
RITE Usability Testing

                   Test!     Rev
                                  ised
                            prob       as s
                                 lem        oon
                                      is id      as a
                                           enti
                                                fied




                                   >
        Revise               Revise
      Prototype!           Prototype!



                   Test!
                                        #UXSmackdown
How do you make
design decisions
based on only testing
1 person?




                        #UXSmackdown
Modified RITE Process


                             ent ! Revise !
 Test!        Debrief!   Pres ngs         type
(All users)              Findi     P roto



                                     Test!        Debrief!
                                    (All users)


                                                   #UXSmackdown
Benefits of choosing
    low-fidelity
    prototypes


                   #UXSmackdown
Getting end user feedback
earlier in the design process.




                           #UXSmackdown
It’s easier to throw away
paper than code.




                            #UXSmackdown
Provides the opportunity to
think outside the box.




                          #UXSmackdown
It gets the entire team involved.




                           #UXSmackdown
Participants are less apt to give
negative feedback on designs that
have a "finished" appearance.

                   Oh, it’s lovely!
                   I wouldn’t
                   change anything.



                              #UXSmackdown
i FT W!
        n Lo w-F
RIT E o




                       #UXSmackdown
ROUND 3



Make Some Noise!

             #UXSmackdown
ROUND 3



     Questions

                 #UXSmackdown
Focus
                                     G   roups
                                   Vs.
                          Field R
            racking                esearc
       Eye T                             h
            Vs.
           ed Remot e Testing
U nmoderat
              High Fidelity Prototypes
                         Vs.
              Rapid Iterative Testing
                                     #UXSmackdown
#UXSmackdown

More Related Content

Similar to UX Smackdown! Usability Testing Techniques in the Ring

UX Joburg: Mobile UX Workshop
UX Joburg: Mobile UX WorkshopUX Joburg: Mobile UX Workshop
UX Joburg: Mobile UX Workshop
Rob Enslin
 
Fail Fast, Learn Fast, Move Fast: My UX journey to move faster
Fail Fast, Learn Fast, Move Fast: My UX journey to move fasterFail Fast, Learn Fast, Move Fast: My UX journey to move faster
Fail Fast, Learn Fast, Move Fast: My UX journey to move faster
Jeremy Johnson
 
Resize your UX
Resize your UXResize your UX
Resize your UX
Marti Gukeisen
 
Usability Testing
Usability TestingUsability Testing
Usability Testing
Andy Budd
 
What is UX v1
What is UX v1What is UX v1
What is UX v1
Mike Gallers
 
Macadamian product camp - uxd for product managers research and design tech...
Macadamian   product camp - uxd for product managers research and design tech...Macadamian   product camp - uxd for product managers research and design tech...
Macadamian product camp - uxd for product managers research and design tech...Dan Arra
 
eXtreme User eXperience (XUX) - How one team melded UX with XP
eXtreme User eXperience (XUX) - How one team melded UX with XPeXtreme User eXperience (XUX) - How one team melded UX with XP
eXtreme User eXperience (XUX) - How one team melded UX with XP
Michael Rawling
 
What is UX v2
What is UX v2What is UX v2
What is UX v2
Mike Gallers
 
Beyond Staggered Sprints: Integrating User Experience and Agile
Beyond Staggered Sprints: Integrating User Experience and AgileBeyond Staggered Sprints: Integrating User Experience and Agile
Beyond Staggered Sprints: Integrating User Experience and Agile
Jeff Gothelf
 
Getting into User Experience Design
Getting into User Experience DesignGetting into User Experience Design
Getting into User Experience Design
Jess McMullin
 
The Evolution of UX
The Evolution of UXThe Evolution of UX
The Evolution of UX
Kieron Leppard
 
Web valley talk - usability, visualization and mobile app development
Web valley talk - usability, visualization and mobile app developmentWeb valley talk - usability, visualization and mobile app development
Web valley talk - usability, visualization and mobile app development
Eamonn Maguire
 
Catching up on UX
Catching up on UXCatching up on UX
Catching up on UX
Ryo Sampei
 
What is User Experience?
What is User Experience?What is User Experience?
What is User Experience?
Chuck Mallott
 
UXcellence: The Importance Of Human-Centered Design
UXcellence: The Importance Of Human-Centered DesignUXcellence: The Importance Of Human-Centered Design
UXcellence: The Importance Of Human-Centered Design
Mike Townson
 
Fred Spencer: Designing a Great UI
Fred Spencer: Designing a Great UIFred Spencer: Designing a Great UI
Fred Spencer: Designing a Great UI
Axway Appcelerator
 
Designing a Great UI
Designing a Great UIDesigning a Great UI
Designing a Great UIFred Spencer
 
Winning Hearts and Minds: Tips for Embedding User Experience in Your Organisa...
Winning Hearts and Minds: Tips for Embedding User Experience in Your Organisa...Winning Hearts and Minds: Tips for Embedding User Experience in Your Organisa...
Winning Hearts and Minds: Tips for Embedding User Experience in Your Organisa...
Michele Ide-Smith
 
Helping Your Company Adopt a User-Centered Process
Helping Your Company Adopt a User-Centered ProcessHelping Your Company Adopt a User-Centered Process
Helping Your Company Adopt a User-Centered Process
Zack Naylor
 

Similar to UX Smackdown! Usability Testing Techniques in the Ring (20)

UX Joburg: Mobile UX Workshop
UX Joburg: Mobile UX WorkshopUX Joburg: Mobile UX Workshop
UX Joburg: Mobile UX Workshop
 
Fail Fast, Learn Fast, Move Fast: My UX journey to move faster
Fail Fast, Learn Fast, Move Fast: My UX journey to move fasterFail Fast, Learn Fast, Move Fast: My UX journey to move faster
Fail Fast, Learn Fast, Move Fast: My UX journey to move faster
 
Resize your UX
Resize your UXResize your UX
Resize your UX
 
Usability Testing
Usability TestingUsability Testing
Usability Testing
 
What is UX v1
What is UX v1What is UX v1
What is UX v1
 
Macadamian product camp - uxd for product managers research and design tech...
Macadamian   product camp - uxd for product managers research and design tech...Macadamian   product camp - uxd for product managers research and design tech...
Macadamian product camp - uxd for product managers research and design tech...
 
eXtreme User eXperience (XUX) - How one team melded UX with XP
eXtreme User eXperience (XUX) - How one team melded UX with XPeXtreme User eXperience (XUX) - How one team melded UX with XP
eXtreme User eXperience (XUX) - How one team melded UX with XP
 
What is UX v2
What is UX v2What is UX v2
What is UX v2
 
Beyond Staggered Sprints: Integrating User Experience and Agile
Beyond Staggered Sprints: Integrating User Experience and AgileBeyond Staggered Sprints: Integrating User Experience and Agile
Beyond Staggered Sprints: Integrating User Experience and Agile
 
Advanced titanium for i os
Advanced titanium for i osAdvanced titanium for i os
Advanced titanium for i os
 
Getting into User Experience Design
Getting into User Experience DesignGetting into User Experience Design
Getting into User Experience Design
 
The Evolution of UX
The Evolution of UXThe Evolution of UX
The Evolution of UX
 
Web valley talk - usability, visualization and mobile app development
Web valley talk - usability, visualization and mobile app developmentWeb valley talk - usability, visualization and mobile app development
Web valley talk - usability, visualization and mobile app development
 
Catching up on UX
Catching up on UXCatching up on UX
Catching up on UX
 
What is User Experience?
What is User Experience?What is User Experience?
What is User Experience?
 
UXcellence: The Importance Of Human-Centered Design
UXcellence: The Importance Of Human-Centered DesignUXcellence: The Importance Of Human-Centered Design
UXcellence: The Importance Of Human-Centered Design
 
Fred Spencer: Designing a Great UI
Fred Spencer: Designing a Great UIFred Spencer: Designing a Great UI
Fred Spencer: Designing a Great UI
 
Designing a Great UI
Designing a Great UIDesigning a Great UI
Designing a Great UI
 
Winning Hearts and Minds: Tips for Embedding User Experience in Your Organisa...
Winning Hearts and Minds: Tips for Embedding User Experience in Your Organisa...Winning Hearts and Minds: Tips for Embedding User Experience in Your Organisa...
Winning Hearts and Minds: Tips for Embedding User Experience in Your Organisa...
 
Helping Your Company Adopt a User-Centered Process
Helping Your Company Adopt a User-Centered ProcessHelping Your Company Adopt a User-Centered Process
Helping Your Company Adopt a User-Centered Process
 

Recently uploaded

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
James Anderson
 
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofszkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
Alex Pruden
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
sonjaschweigert1
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Aggregage
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptxSecstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
nkrafacyberclub
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 

Recently uploaded (20)

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
 
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofszkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
 
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptxSecstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 

UX Smackdown! Usability Testing Techniques in the Ring

  • 1. UX SMACKDOWN! Usability Testing Techniques in the Ring #UXSmackdown
  • 2. Aviva Rosenstein Rebecca Sherrill Sr Mgr, User Research UX Lead Salesforce.com Beaconfire.com @uxresearch @beccas Kyle Soucy Michael Summers Founding Principal Global User Experience, Paypal UsableInterface.com UserResearch.com @kylesoucy @ez2use John Whalen UX Lead & Founder Brilliant Experience.com @johnwhalen #uxSmackdown #UXSmackdown
  • 3. ROUND 1 Focus Groups Vs. Fie ld Res earch #UXSmackdown
  • 4. Focus Groups #UXSmackdown
  • 5. ? Why are we proponents of focus groups? #UXSmackdown
  • 6. 1. Focus groups do not have to be this... Photo  copyright  AMC #UXSmackdown
  • 7. 1. Focus groups do not have to be this... Photo  copyright  AMC #UXSmackdown
  • 8. 2. You don’t have to dress up like a $#%^&*@$!! #fieldstudyninja to get good data. #UXSmackdown
  • 9. 3a. Focus groups can be “in situ” #UXSmackdown
  • 10. 3b. Focus groups can be remote #UXSmackdown
  • 11. 4. Focus groups can have individual assignments before group discussion to produce better data. #UXSmackdown
  • 12. Drip Coffee?! No M&M’s? I must make a Totally Starbucks run! unacceptable! I can’t work under these conditions! Diet Pepsi is ...and only 12 no substitute for menu choices?!! Diet Coke! Who has the Zagats! Finally, think of what you’d miss behind the glass... Check out skydeckcartoons.com for more. #UXSmackdown
  • 13. Summary Reasons for conducting Focus Groups: 1. Focus groups can be “small group workshops,” not “focus groups”. 2. You don’t have to dress up like a field studies ninja. 3. Focus groups can be “in situ” or remote. 4. Focus groups can have individual assignments before group discussion to improve data quality. #UXSmackdown
  • 14. Field Research #UXSmackdown
  • 15. “What  people  say,   what  people  do,  and   what  they  say  they  do   are  entirely  different   things.”   —Margaret Mead #UXSmackdown
  • 16. Focus groups aren’t enough: If you want to find out: • What users want & need • What they’re doing now • How they feel about it • What you can improve
  • 17. Immerse yourself in the user’s world #UXSmackdown
  • 18. ROLE: Business Owner TASK: Approve visual design direction CONTEXT: Waterfall dev process. Supervises multiple product managers. Frequently Approve visual design direction TASK: mobile; uses iPhone. CHARACTERISTICS: Short aention span. Under significant time pressure. PM ID Focuses on visuals and metrics. BO CONTENT CRITERIA: Brief, clear presentation in common formats consumable on mobile devices Dev Mgr VzD #UXSmackdown
  • 19. Stakeholder participation in analysis = everyone owns the study findings Photo  by  Jane  Mejdahl,    used  under  CC  BY-­‐SA  2.0.   19
  • 20. Focus group falsehoods: • Human memory fail • Suggestibility • Social norms • Self-enhancement bias • Self serving facilitators #UXSmackdown
  • 22. Oh! How do you Thank like the gift I you! made you? (Ugh, it’s horrible.) #UXSmackdown
  • 23. How do you Oh! like the app Great! we built you? (Yuck, it’s useless.) #UXSmackdown
  • 24. Our study found that focus group participants are more generous, more law abiding, and floss more often. #UXSmackdown
  • 25. Summary Field studies let you: • Understand user needs • Uncover tacit knowledge that isn’t consciously available • Find deltas between how users think they do something and their performance • Develop actionable insights • Identify opportunities for innovation #UXSmackdown
  • 26. ROUND 1 Make Some Noise! #UXSmackdown
  • 27. ROUND 1 Questions #UXSmackdown
  • 28. ROUND 2 Eye Tr acking Vs. mote T esting Un moderated Re #UXSmackdown
  • 29. Eye Tracking #UXSmackdown
  • 34. Benefits #UXSmackdown
  • 35. Test hundreds of people simultaneously while keeping them in their own natural environment. #UXSmackdown
  • 36. Test multiple websites simultaneously. #UXSmackdown
  • 37. Reduced costs... depending on the tool you use. #UXSmackdown
  • 38. Great way to plant the seed of UCD methodologies and introduce usability testing into a company. UX #UXSmackdown
  • 39. Fewer logistics to manage and no need to worry about no-shows and getting last-minute replacements. #UXSmackdown
  • 40. Fast Results #UXSmackdown
  • 41. In-context Feedback “...I’m only offered a search by model number. How ridiculous.” “...I would like to see the products availability and delivery/ shipping information listed.” Image source: Loop11 #UXSmackdown
  • 42. The danger... I wonder how a real customer would actually use this website? Panelist #UXSmackdown
  • 43. The danger... BLOCKED BLOCKED #UXSmackdown
  • 44. There are no shortcuts to conducting good research. #UXSmackdown
  • 45. YO U B E THE JU DG E! Good? Evil? #UXSmackdown Photo  copyright  New  Line  Cinema
  • 46. ROUND 2 Make Some Noise! #UXSmackdown
  • 47. ROUND 2 Questions #UXSmackdown
  • 48. ROUND 3 elity Proto types H igh Fid Vs. Itera tive T esting Rapid #UXSmackdown
  • 49. High Fidelity Prototypes #UXSmackdown
  • 50. Why #HighFidelityRules... ...and #IterativeTestingFails Photo  copyright  Apple #UXSmackdown
  • 51. Weight 2,061 2,954 Drive Train Rear Rear HP 350 320 Torque 292 273 Top Speed 156 171 0-60 5.3 4.9 Why High-Fidelity Prototypes? 1. Everything looks great on paper... #UXSmackdown
  • 52. ...but sometimes the final experience is not the same. #UXSmackdown
  • 53. ...but sometimes the final experience is not the same. #UXSmackdown
  • 54. In today’s swipe and swish world you need the full experience. #UXSmackdown
  • 55. Pause Suppli Perfor HIGHSCORE 0 es mance SCORE 0 2a. Iterative testing can’t tell you if you’re creating flow. #UXSmackdown
  • 56. 2a: Iterative testing can’t tell you if you’re creating flow. #UXSmackdown
  • 57. You should see the front. blue fade yellow fade 2b: Iterative testing can’t tell you if you’re creating persuasive design. #UXSmackdown
  • 58. 2b: Iterative testing can’t tell you if you’re creating persuasive design. #UXSmackdown
  • 59. 3. High fidelity prototypes identify the full implications of “stakeholder design” #UXSmackdown
  • 61. John, couldn’t you iteratively test something simple, like a login without a high fidelity prototype?! #UXSmackdown
  • 62. John, couldn’t you iteratively test something simple, like a login without a high fidelity prototype?! YES #UXSmackdown
  • 63. ...but my argument is that increasingly we need the full experience to understand a design’s success. #UXSmackdown
  • 64. Final note: I too find great value in iterative design in the right circumstance! #UXSmackdown
  • 65. Summary Reasons for High-Fidelity Prototypes: 1. In today’s swipe and swish world you need the full experience. 2. You can’t really “iteratively test”, persuasive or multi-channel designs. 3. High-fidelity prototypes identify the full implications of “stakeholder design”. #UXSmackdown
  • 66. High Fidelity Rapid Iterative Prototype vs. Usability Testing Testing #UXSmackdown
  • 67. Low-Fidelity Prototyping viously Ad d pre rk N etwo ity Activ #UXSmackdown
  • 68. RITE Usability Testing Test! Rev ised prob as s lem oon is id as a enti fied > Revise Revise Prototype! Prototype! Test! #UXSmackdown
  • 69. How do you make design decisions based on only testing 1 person? #UXSmackdown
  • 70. Modified RITE Process ent ! Revise ! Test! Debrief! Pres ngs type (All users) Findi P roto Test! Debrief! (All users) #UXSmackdown
  • 71. Benefits of choosing low-fidelity prototypes #UXSmackdown
  • 72. Getting end user feedback earlier in the design process. #UXSmackdown
  • 73. It’s easier to throw away paper than code. #UXSmackdown
  • 74. Provides the opportunity to think outside the box. #UXSmackdown
  • 75. It gets the entire team involved. #UXSmackdown
  • 76. Participants are less apt to give negative feedback on designs that have a "finished" appearance. Oh, it’s lovely! I wouldn’t change anything. #UXSmackdown
  • 77. i FT W! n Lo w-F RIT E o #UXSmackdown
  • 78. ROUND 3 Make Some Noise! #UXSmackdown
  • 79. ROUND 3 Questions #UXSmackdown
  • 80. Focus G roups Vs. Field R racking esearc Eye T h Vs. ed Remot e Testing U nmoderat High Fidelity Prototypes Vs. Rapid Iterative Testing #UXSmackdown