User requirements interviews often go wrong when designers fail to properly understand user work contexts. Mistakes include not observing users in the field, accepting non-user representatives, and not getting low-level work details. To get useful data, designers must see live or retrospective work, use "magic words" to elicit specifics, and assume a partnership role rather than acting as interrogators. Even one good field interview provides better insights than none.

1. ProductCamp Vancouver 2011
User Requirements Interviewing Mistakes
Recognizing and preventing them
Dave Flotree
Program Manager
dave.flotree@incontextdesign.com
www.incontextdesign.com 206-546-3833
Copyright © 2011 InContext Enterprises Inc. All rights reserved.

2. InContext Design
Developers of Contextual Design
 A repeatable end-to-end design process
 Used to develop business systems, products
web-based applications, consumer goods…
 Used and taught in universities and companies
all over the world
 Supporting any methodology: RUP, Agile, Corporate …
Principals
 Karen Holtzblatt, co-founder and CEO
• Recognized worldwide as an expert on user-centered design
 Hugh Beyer, co-founder and CTO
• The technology backbone to all our designs and processes
InContext
 Founded – 1992
 >20 professionals
 Offices in Boston and Chicago

3. Solving the problem of design
Design the optimal match:
 Products and systems embody work practice
 Design must support and extend user intent
The User’s
Work Model has:
Language
Work Structure
Work Flow
Intention
Result
The Product/System’s
Work Model has:
Language
System Structure
Features
Concepts

4. Users have all the knowledge….
…but they can’t tell us in a way that reliably informs product design
Users do the work, they don’t self-analyze how they do the work
 The more expert they are, the more invisible the work details are to them
To find the work practice, we need to be with them while they work
 Or re-create recently done work
Field interviews done as contextual inquiries reliably:
 Uncover what customers do and need
 See the underlying details
 And as important as the ―whats‖, discover the ―whys‖

5. Principles to guide contextual inquiry interviews
Context
 Understand user needs in the context of their work
 Go to where people work or live and talk to them while they work
Partnership
 The user is the expert and has all the knowledge
 Work with customers as partners in inquiry
 Use the model of the master-apprentice
Interpretation
 It’s not the facts that matter, it’s the interpretation of the facts
 Uncover the meaning and implications of user action and language
Focus
 Listen and probe from a clear intention
 Acknowledge your entering assumptions, and then assume you are wrong

6. Mistake 1: Pretending you are in the field
These places are not the field:
 A focus group
 The usability lab
 A user conference
 A conference room in the workplace
 Even the office or home may not be the field
• If the work is done elsewhere

7. What to do to avoid the mistake
Expect that users won’t understand right away
 Be ready to politely move the user into the work space
 It’s actually rude not to move the user
• Otherwise you are wasting their time
Travel to the field
 Even when you have little time or budget
• It’s where the customer data is, so help your management understand this
– Even a few field interviews can pick up requirements that might otherwise be missed
• Look for local people who do the work to save time and money
 Remote meeting tools are your fallback, but
• Start with in-person interviews first
– And then use a collaboration tool if the work is dominated by web or tool interaction
• Be sure the user is in the real workplace, doing real work

8. Mistake 2: Accepting a representative user
These people are not who you want to interview
 Someone who used to do the job
• Subject matter experts, for example
 Someone who tells others how to do the job but doesn’t do it
• Managers
 People in your company who do the same job as the customers
• Unless you are designing an internal system
What to do to avoid the mistake
 Don’t assume that you have to interview the non-users
 Let them be heard in other ways
• Stakeholder interviews help set project focus
• Do ―courtesy‖ interviews
– Short, non-contextual interviews that you do because you have to
• Just be careful to separate this data from the field data
• Let them be on the team and gather the real data too

9. Mistake 3: The “I can’t see live work” excuse
Sometimes the work you need to see:
 Is intermittent or infrequent
 Takes place over long periods of time
 Is highly sensitive or confidential
 So you assume that you can’t see the work
What to do to avoid the mistake
 Conduct a retrospective account interview
• Re-create the actual work done in the past
• Within the last two weeks is best
• Re-created with actual work artifacts
 Be careful with retrospective accounts
• You only care about the current and past
– Don’t ask the user to make up the future
• The user will tend to skip over parts of the story
– It’s your job to recognize skips and help the user to fill them in

10. Mistake 4: Not getting low level details
Recognize an abstraction when you hear one
 ―Generally, typically, normally, our company’s policy…‖
 Abstractions may tell us there’s a problem or opportunity
• But they do not give us the detail we need for design
What to do to avoid the mistake
 Don’t ask users what they typically, generally, or normally do — you are
asking for a generalization
 Stick with a real instance and play it out live or retrospectively
 And use the ―magic words‖ from Mistake 5…

11. Mistake 5: Assuming users can tell you…
...if you just ask
Don’t expect users to articulate what they do without your help
 The more expert the users…
• …the less able they are to articulate what they do and why
 They do the work, not think about the work
What to do to avoid the mistake
 The user is the expert — let them lead
 Your job is to help the user articulate their work practice, so use the
―magic words‖
• ―When was the last time you…?‖
• ―Can you show me what you did?‖

12. Mistake 6: Asking “Why?” or open-ended questions
Interpretation is the data
Customer
 Create a shared understanding
of what is going on
 Inquire into the meaning of Fact
customer action and words
tune the interpretation
What to do to avoid
the mistake Hypothesis
 Offer interpretations, not just
open-ended questions
Implication
Design
Idea

13. Mistake 7: Not being honest about the reaction
Sometimes a ―no‖ isn’t said as ―no‖
 People want to be helpful and nice!
What to do to avoid the mistake
 Listen for the ―no‖
• Huh?
• Umm... could be
• ―They‖ would like it
• ―Yes‖ comes with elaboration
 Watch for non-verbal cues

14. Mistake 8: Falling into ineffective interview styles
The Traditional Interviewer
 ―I Ask/You Answer‖
The Court Reporter
 Writing down everything without finding underlying intents
The Police Interrogator
 Grilling the customer
The Agenda Follower
 Paying attention to your personal focus, not the user’s work
The ―Polite‖ Interviewer
 Letting the user go on tangents that are not in your focus
 Thinking it is rude to be curious
The Mad Inventor
 Constantly offering design ideas and solutions

15. What to do to avoid the mistake
Knowing the ineffective relationship models creates awareness
 Use these styles as triggers to change your approach
Apprenticeship is the preferred relationship model
 Listen, learn, be humble, don’t judge
 And assume that people do things for a reason
Return to the ongoing work
 It always keeps you in the apprenticeship model

16. Mistake 9: Expecting the interview to run itself
The interview is not casual or informal
 It requires intense focus on capturing a lot of detail in a limited time
 While following the master-apprentice relationship model
How you run the interview is key—it’s a partnership
 The way you conduct the interview ―trains― the user
• Users will follow your lead
 So make sure expectations are clear
Users have no frame of reference for this kind of interview
 So it can take awhile to adapt
 Some users will adapt quickly, others take a long time
• And then it will feel like hard work on your part for the entire interview
• But that's o.k., that’s your job!

17. Mistake 10: Forgetting that one good field interview…
…is much better than no interviews
 And two interviews is twice as good
 And so on….
Success is getting any user data into the project
 And then having a customer-centered process as part of standard
development
 If using contextual data is new, small steps are often needed
• It often involves organizational culture change

18. Get the most out of data after the interview
Don’t write a trip report or debrief by yourself
 Use a process like an interpretation session with at least one other person
 Other perspectives reveal more insights
 Capture the data in a way that useful for organizing it, e.g. virtual Post-It notes
Build an affinity diagram
 Reveals underlying pattern: intent, strategy, structure, and scope
 Shows what matters to the entire population, and keeps variations that matter
 Consider if other work models will be valuable
Brainstorm (vision) product concepts / requirements
 Grounded in the customer data — no more designing in isolation
Work out the details
 Interaction patterns, storyboarding, and system structure ensure a coherent experience
Test with customers
 Mock up the design in rough paper prototypes
 Rough mockups test structure and coherence of the design

19. Contextual Design
Requirements & Solutions
Contextual Inquiry Talk to specific customers in the field
What matters to
Interpret the data as a team to capture key issues
users –
Interpretation Session
characterizing
what they do
Work Models and
Consolidate data across customers for a full market view
Affinity Diagramming
New ideas and
Visioning Redesign people’s work with new technology ideas
direction
Define & Validate Concepts
Storyboarding Work out the details of particular tasks and roles Redesign
activities and
technology to
User Environment provide value
Design system to support this new work
Design
Paper Mock-Up
Mock up the interface using interaction patterns for testing
Interviews Iterate the
system with
Interaction & Visual users
Design and test the final look and user experience
Design

20. Thank you!
Dave Flotree
dave.flotree@incontextdesign.com
www.incontextdesign.com