Closing the feedback loop
with a little help from your
friends



Agile 2009 Jackson Fox August 26, 2009
A few things before we start
1. Feel free to ask questions at any time
2. Can also send questions to @jacksonfox
3. Please tag any posts #feedbackloop




Agile 2009                                  2
A bit about myself...
NAME: Jackson Fox
JOB: UX Designer
COMPANY: Viget Labs


Agile 2009              3
e this q uestion
         Sinc         a lot...
          com   es up


                   vi·get
                                 e

Agile 2009                           4
Previously...
Comp Sci Major C
Academic B+
Web Developer A


Agile 2009         5
Previously...
Comp Sci Major HCI
Academic Cog Psych
Web Developer Interfaces
= User Experience

Agile 2009                 6
Product Team + Users = $


Agile 2009                   7
w do w  e make
             Ho
               this h appen?


  Product Team + Users = $
                           The $300 mil
                              button


Agile 2009                            8
Ethnography




Agile 2009 http://flic.kr/p/3gXSyw   9
Usability Lab




Agile 2009 http://flic.kr/p/Kic8T   10
Most of the time,
             we need to be creative

             What Would
             MacGyver Do?
             (with apologies to m. jackson wilkinson)




Agile 2009                                              11
Level of interaction with
             the customer

   1. Talking to customers
   2. Thinking like a customer
   3. Thinking like yourself




Agile 2009                               12
Level of interaction with
             the customer

         Talking to customers
         Thinking like a customer
         Thinking like yourself

Increasing quality of
feedback & insight


Agile 2009                               13
Is there
  We rarely have              step her  another
  the time                             e?


         Talking to customers
         Thinking like a customer
         Thinking like yourself

                              d too much
                     W e spen
                     time here

Agile 2009                                        14
Everyone Else:
                      Doesn’t Do Usability
     UX Person:
    Does Usability




 Your organizat
                ion
Agile 2009                                   15
Wh y not?     Everyone Else:
                             Doesn’t Do Usability
     UX Person:
    Does Usability




 Your organizat
                ion
Agile 2009                                          16
The big picture


As an agile team, how do you collaborate with
other groups in your organization in order to get
insight into customer needs and get timely
customer feedback on your work?



Agile 2009                                     17
Our Agenda
An example
Step-by-step
    Making the plan
    Finding our allies
    Doing the research
Making it a habit


Agile 2009               18
Our Agenda
An example
Step-by-step
    Making the plan
    Finding our allies
    Doing the research
Making it a habit


Agile 2009               19
Example:
             Choice Hotels EU

Agile 2009                      20
We had a simple plan
1. Build a prototype
2. Get feedback
3. Make awesome site




Agile 2009             21
Just a few small problems
➡   Je ne parle pas français
➡   Ich spreche keine deutsch
➡   No hablo español




Agile 2009                      22
Just a few small problems
➡   No money for travel
➡   Those pesky time zones
➡   Local customs




Agile 2009                   23
We got everyone involved
                     Training          Remote Testing




        Viget Team              Choice Team         Users

Agile 2009                                                  24
The Results
 4 Teams (UK, FR, DE, Australia)
 4 Languages
50 Users
 2 Weeks


Agile 2009                         25
Benefits	
➡   Choice teams felt like we were really listening
➡   Helped Choice teams take an active role in the design process
➡   Teams felt ownership of the design process
➡   Teams felt ownership of the user experience
➡   Helped resolve differences of opinion
➡   Gained skills for the future




Agile 2009                                                          26
Things this presentation
isn’t about:
Usability Testing

Agile 2009                 27
Thing this presentation
isn’t about:
Usability Testing
Voice of the Customer
Analytics
Agile 2009                28
Our Agenda
An example
Step-by-step
    Making the plan
    Finding our allies
    Doing the research
Making it a habit


Agile 2009               29
Making the plan...
What do we want to know?
How are we going to find out?

Agile 2009                     30
“Designch”          “User
               Resear              Researc
                                          h”
                        Empathy
             Context                    Insight

                   Generate
                      Two kinds of
                     research tasks

                   Evaluate
              Perform
                     a            Optimi zation
                            nce
                            Usability
Agile 2009                                        31
The Rumsfeld Scale
 Known Knowns
 Known Unknowns
 Unknown Unknowns
                                           need t o know
                             What  do you          goals?
Where                                 mplish your
       are peo               t o acco
problem        ple havi
       s?               ng


 Agile 2009                                           32
Good Research Questions
Are concrete, with clear answers
Don’t worry (yet) about how you’re going to answer them
Are driven by near-term project needs (1-2 iterations ahead)




Agile 2009                                                     33
Meet TuningFork.com
TuningFork.com is a specialized digital music store (not really), and
you are the product team. TurningFork has been doing relatively well
in their niche, but is facing increasing pressure from PolkaTunez.com.
To fight back, TuningFork is planning to release a greatly improved
shopping experience. In 12 weeks (6 iterations).

Project Goal
Decrease support calls related to purchasing and downloads



Agile 2009                                                          34
Task: Research Questions
Project Goal
Decrease support calls related to music downloads

Our Task
What are 5 research questions we should be asking? [5 min]




Agile 2009                                                   35
Pick the method that fits
the question, not the
question that fits the
method
Agile 2009                 36
Research Methods
Usability Testing            Contextual Interviews
Remote Usability Testing     Surveys
Guerilla Usability Testing   A/B Testing
5-Second Tests               Multivariate Testing
Card Sorting                 Divide the Dollar
Field Research
These are all useful, but don’t get hung-up
on them, try to be c reative in finding ways to
approach your question


Agile 2009                                           37
Task: Research Methods
Project Goal
Decrease support calls related to music downloads

Your Task
What research methods could we use to get answers to our
research question? (Pick one question from our list.) [5 min]




Agile 2009                                                      38
Finding our allies...
Who do we want to talk to?
Who can help us?

Agile 2009                   39
Is there a specific group of
customers we want to talk
to in order to answer our
questions?
Agile 2009                    40
Task: Which Users?
Project Goal
Decrease support calls related to music downloads

Our Task
What are 5 characteristics of the people we should be talking to? [5
min]




Agile 2009                                                             41
Identifying Allies
Access
Who is already in contact with users?
Who is already in contact with your users?

Mutual Interest
Who would be interested in this information?
Who else has a stake in your project goals?

Strengths
Who already has complementary skills?
Agile 2009                                     42
Who are these people?


Agile 2009              43
Task: Which Allies?
Project Goal
Decrease support calls related to music downloads

Our Task
Within your groups, brainstorm a list of roles/people within an
organization that might serve as allies on our project. Think about
groups with access, groups with mutual interest, groups with
complementary strengths, and groups that might have all three. [5
min]


Agile 2009                                                            44
How Do We Collaborate?
➡   We’re looking for collaborators, not subject
➡   Split up the work (planning, recruiting, testing)
➡   Share your experience
➡   Review results as a team




Agile 2009                                              45
Doing the research...
Creating the infrastructure
Getting good results

Agile 2009                    46
Creating The Infrastructure
➡   Clearly define which users we want to talk to
➡   Identify and test out the tools
➡   Document who is responsible for specific tasks
➡   Document the process
➡   Do a practice run




Agile 2009                                           47
Getting Good Results
➡   No, seriously, practice
➡   Identify the information you want to collect
➡   Decide if you’re going to record sessions
➡   Create a form for collecting information




Agile 2009                                         48
Our Agenda
An example
Step-by-step
    Making the plan
    Finding our allies
    Doing the research
Making it a habit


Agile 2009               49
Agile workflow
➡   Focus on information needs 1-2 weeks or iterations ahead
➡   Identify information needs in user stories
➡   Make it a habit




Agile 2009                                                     50
Building a customer-centric
organization


Agile 2009                    51
Vision
             Feedback
             Culture
                Jared Spool
              IA Summit 2008




Agile 2009                     52
Feedback
In the last six weeks, have you
spent more than two hours
watching someone use yours or a
competitors design?


Agile 2009                        53
Our Results
➡   Teams felt like partners in the design process
➡   Teams felt ownership of the user experience
➡   Resolved conflicts based on individual opinions
    and preconceived notions




Agile 2009                                            54
Keeping the channel open
➡   It’s easier to get feedback through open channels than it is to work
    from a cold start every time
➡   You can get user feedback when you need it, not when it’s most
    convenient to get




Agile 2009                                                             55
Recap


Agile 2009   56
Stop, Collaborate, Listen
➡   There are times when we don’t have the time or the opportunity to
    do the research we need to feed the design process
➡   There are other people in your organization who want to know the
    same things you do
➡   You can fill in knowledge gaps by collaborating
➡   Keep research efforts targeted, and focus on project goals
➡   The best results and efficiencies will come when you make this a
    regular part of your process



Agile 2009                                                          57
Good
➡   Can provide insight and feedback in circumstances where it
    otherwise wouldn’t have been possible
➡   Exposes more of the organization to the needs of your users
➡   Helps the rest of your team and your organization become part of
    the design process




Agile 2009                                                             58
Bad/Ugly
➡   May require a good deal of effort and political will to find allies
➡   Never as good as being there in person
➡   High up-front costs depending on methods and practice




Agile 2009                                                                59
Questions?
Agile 2009                60
The End
             You’ve Been Awesome
             http://spkr8.com/talks/1389




Agile 2009                                 61
Get in touch!
NAME: Jackson Fox
EMAIL: jacksonfox@gmail.com
TWITTER: @jacksonfox



Agile 2009                    62

Closing the feedback loop with a little help from your friends

  • 1.
    Closing the feedbackloop with a little help from your friends Agile 2009 Jackson Fox August 26, 2009
  • 2.
    A few thingsbefore we start 1. Feel free to ask questions at any time 2. Can also send questions to @jacksonfox 3. Please tag any posts #feedbackloop Agile 2009 2
  • 3.
    A bit aboutmyself... NAME: Jackson Fox JOB: UX Designer COMPANY: Viget Labs Agile 2009 3
  • 4.
    e this question Sinc a lot... com es up vi·get e Agile 2009 4
  • 5.
    Previously... Comp Sci MajorC Academic B+ Web Developer A Agile 2009 5
  • 6.
    Previously... Comp Sci MajorHCI Academic Cog Psych Web Developer Interfaces = User Experience Agile 2009 6
  • 7.
    Product Team +Users = $ Agile 2009 7
  • 8.
    w do w e make Ho this h appen? Product Team + Users = $ The $300 mil button Agile 2009 8
  • 9.
  • 10.
    Usability Lab Agile 2009http://flic.kr/p/Kic8T 10
  • 11.
    Most of thetime, we need to be creative What Would MacGyver Do? (with apologies to m. jackson wilkinson) Agile 2009 11
  • 12.
    Level of interactionwith the customer 1. Talking to customers 2. Thinking like a customer 3. Thinking like yourself Agile 2009 12
  • 13.
    Level of interactionwith the customer Talking to customers Thinking like a customer Thinking like yourself Increasing quality of feedback & insight Agile 2009 13
  • 14.
    Is there We rarely have step her another the time e? Talking to customers Thinking like a customer Thinking like yourself d too much W e spen time here Agile 2009 14
  • 15.
    Everyone Else: Doesn’t Do Usability UX Person: Does Usability Your organizat ion Agile 2009 15
  • 16.
    Wh y not? Everyone Else: Doesn’t Do Usability UX Person: Does Usability Your organizat ion Agile 2009 16
  • 17.
    The big picture Asan agile team, how do you collaborate with other groups in your organization in order to get insight into customer needs and get timely customer feedback on your work? Agile 2009 17
  • 18.
    Our Agenda An example Step-by-step Making the plan Finding our allies Doing the research Making it a habit Agile 2009 18
  • 19.
    Our Agenda An example Step-by-step Making the plan Finding our allies Doing the research Making it a habit Agile 2009 19
  • 20.
    Example: Choice Hotels EU Agile 2009 20
  • 21.
    We had asimple plan 1. Build a prototype 2. Get feedback 3. Make awesome site Agile 2009 21
  • 22.
    Just a fewsmall problems ➡ Je ne parle pas français ➡ Ich spreche keine deutsch ➡ No hablo español Agile 2009 22
  • 23.
    Just a fewsmall problems ➡ No money for travel ➡ Those pesky time zones ➡ Local customs Agile 2009 23
  • 24.
    We got everyoneinvolved Training Remote Testing Viget Team Choice Team Users Agile 2009 24
  • 25.
    The Results 4Teams (UK, FR, DE, Australia) 4 Languages 50 Users 2 Weeks Agile 2009 25
  • 26.
    Benefits ➡ Choice teams felt like we were really listening ➡ Helped Choice teams take an active role in the design process ➡ Teams felt ownership of the design process ➡ Teams felt ownership of the user experience ➡ Helped resolve differences of opinion ➡ Gained skills for the future Agile 2009 26
  • 27.
    Things this presentation isn’tabout: Usability Testing Agile 2009 27
  • 28.
    Thing this presentation isn’tabout: Usability Testing Voice of the Customer Analytics Agile 2009 28
  • 29.
    Our Agenda An example Step-by-step Making the plan Finding our allies Doing the research Making it a habit Agile 2009 29
  • 30.
    Making the plan... Whatdo we want to know? How are we going to find out? Agile 2009 30
  • 31.
    “Designch” “User Resear Researc h” Empathy Context Insight Generate Two kinds of research tasks Evaluate Perform a Optimi zation nce Usability Agile 2009 31
  • 32.
    The Rumsfeld Scale Known Knowns Known Unknowns Unknown Unknowns need t o know What do you goals? Where mplish your are peo t o acco problem ple havi s? ng Agile 2009 32
  • 33.
    Good Research Questions Areconcrete, with clear answers Don’t worry (yet) about how you’re going to answer them Are driven by near-term project needs (1-2 iterations ahead) Agile 2009 33
  • 34.
    Meet TuningFork.com TuningFork.com isa specialized digital music store (not really), and you are the product team. TurningFork has been doing relatively well in their niche, but is facing increasing pressure from PolkaTunez.com. To fight back, TuningFork is planning to release a greatly improved shopping experience. In 12 weeks (6 iterations). Project Goal Decrease support calls related to purchasing and downloads Agile 2009 34
  • 35.
    Task: Research Questions ProjectGoal Decrease support calls related to music downloads Our Task What are 5 research questions we should be asking? [5 min] Agile 2009 35
  • 36.
    Pick the methodthat fits the question, not the question that fits the method Agile 2009 36
  • 37.
    Research Methods Usability Testing Contextual Interviews Remote Usability Testing Surveys Guerilla Usability Testing A/B Testing 5-Second Tests Multivariate Testing Card Sorting Divide the Dollar Field Research These are all useful, but don’t get hung-up on them, try to be c reative in finding ways to approach your question Agile 2009 37
  • 38.
    Task: Research Methods ProjectGoal Decrease support calls related to music downloads Your Task What research methods could we use to get answers to our research question? (Pick one question from our list.) [5 min] Agile 2009 38
  • 39.
    Finding our allies... Whodo we want to talk to? Who can help us? Agile 2009 39
  • 40.
    Is there aspecific group of customers we want to talk to in order to answer our questions? Agile 2009 40
  • 41.
    Task: Which Users? ProjectGoal Decrease support calls related to music downloads Our Task What are 5 characteristics of the people we should be talking to? [5 min] Agile 2009 41
  • 42.
    Identifying Allies Access Who isalready in contact with users? Who is already in contact with your users? Mutual Interest Who would be interested in this information? Who else has a stake in your project goals? Strengths Who already has complementary skills? Agile 2009 42
  • 43.
    Who are thesepeople? Agile 2009 43
  • 44.
    Task: Which Allies? ProjectGoal Decrease support calls related to music downloads Our Task Within your groups, brainstorm a list of roles/people within an organization that might serve as allies on our project. Think about groups with access, groups with mutual interest, groups with complementary strengths, and groups that might have all three. [5 min] Agile 2009 44
  • 45.
    How Do WeCollaborate? ➡ We’re looking for collaborators, not subject ➡ Split up the work (planning, recruiting, testing) ➡ Share your experience ➡ Review results as a team Agile 2009 45
  • 46.
    Doing the research... Creatingthe infrastructure Getting good results Agile 2009 46
  • 47.
    Creating The Infrastructure ➡ Clearly define which users we want to talk to ➡ Identify and test out the tools ➡ Document who is responsible for specific tasks ➡ Document the process ➡ Do a practice run Agile 2009 47
  • 48.
    Getting Good Results ➡ No, seriously, practice ➡ Identify the information you want to collect ➡ Decide if you’re going to record sessions ➡ Create a form for collecting information Agile 2009 48
  • 49.
    Our Agenda An example Step-by-step Making the plan Finding our allies Doing the research Making it a habit Agile 2009 49
  • 50.
    Agile workflow ➡ Focus on information needs 1-2 weeks or iterations ahead ➡ Identify information needs in user stories ➡ Make it a habit Agile 2009 50
  • 51.
  • 52.
    Vision Feedback Culture Jared Spool IA Summit 2008 Agile 2009 52
  • 53.
    Feedback In the lastsix weeks, have you spent more than two hours watching someone use yours or a competitors design? Agile 2009 53
  • 54.
    Our Results ➡ Teams felt like partners in the design process ➡ Teams felt ownership of the user experience ➡ Resolved conflicts based on individual opinions and preconceived notions Agile 2009 54
  • 55.
    Keeping the channelopen ➡ It’s easier to get feedback through open channels than it is to work from a cold start every time ➡ You can get user feedback when you need it, not when it’s most convenient to get Agile 2009 55
  • 56.
  • 57.
    Stop, Collaborate, Listen ➡ There are times when we don’t have the time or the opportunity to do the research we need to feed the design process ➡ There are other people in your organization who want to know the same things you do ➡ You can fill in knowledge gaps by collaborating ➡ Keep research efforts targeted, and focus on project goals ➡ The best results and efficiencies will come when you make this a regular part of your process Agile 2009 57
  • 58.
    Good ➡ Can provide insight and feedback in circumstances where it otherwise wouldn’t have been possible ➡ Exposes more of the organization to the needs of your users ➡ Helps the rest of your team and your organization become part of the design process Agile 2009 58
  • 59.
    Bad/Ugly ➡ May require a good deal of effort and political will to find allies ➡ Never as good as being there in person ➡ High up-front costs depending on methods and practice Agile 2009 59
  • 60.
  • 61.
    The End You’ve Been Awesome http://spkr8.com/talks/1389 Agile 2009 61
  • 62.
    Get in touch! NAME:Jackson Fox EMAIL: jacksonfox@gmail.com TWITTER: @jacksonfox Agile 2009 62