Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
What your users really think
Incorporating user testing and research into Agile
Phil Barrett • Flow Interactive • Agile Af...
UX design, research and
strategy
London & Cape Town
!
Since 1998
!
Hello.
You check out some new digital product.
You
“Meh.”
You
“Meh.”
Dear sir/madam
!
I recently visited your new website. I must
confess that at first, I found it a bit difficult to...
Who sends you feedback?
No opinion Tentative opinion Strong opinion
Not using
Trying out
Casual user
Evangelist/Beta group...
Who sends you feedback?
Feedback
Silence
No opinion Tentative opinion Strong opinion
Not using
Trying out
Casual user
Evan...
Missing feedback breaks Agile’s awesomeness
Build
Learn
Measure
Agile projects deliver gradual, incremental change
so it’s easy to miss overall user impact.
Flickr: Lars ploughman
Web analytics, tracking
What
do people do with
the product?
!
Hard data.
Split testing
What
will people do if we
try somet...
F2F user testing gives you a blast of reality
I don’t need no stinkin’ feedback
• I know what people want
• My software is clearly good. I mean: look!
• Customers don’t...
I know 

what people want
You’re much better with software 

than most of your users
Website tasks:

the slowest 25%
of users take 

2.4 times

as l...
Econ: Considers, calculates,
makes optimal decisions
Human: Emotions, shortcuts
and irrationality
Real people are unpredic...
My software is good:
look!
Dan Ariely did an experiment.
With origami frogs.
They were hard to make and most people did a bad job.
How much would peo...
Flickr:Nanimo
We think the things we make are expert quality.
• Average bid for expert-made frog: 27¢
• Average bid for ow...
“
The 9x effect
Executives, overvalue their own innovations...
Companies
overweight the new
product’s benefits
by a factor o...
Customers don’t
know what they want
Humans are bad at imagining the future
Understand the jobs to be done
“To design an easy-to-use interface, pay
attention to what users do, not what they say.
!
Self-reported claims are unrelia...
“There is a direct correlation between the
number of hours each team member
is exposed directly to real users and
the impr...
Let’s make it
incredibly easy
Doing a usability test
1. 

Get an interface.

List key tasks.
2. 

Ask someone
new to try doing
the tasks, and
think aloud.
3. 

Don’t interfere.
Just write things

down.
!
Wait for 4 seconds.
4. 

Q. “Is this right?” 

A. “What do you think?”
!
!
Be weird but friendly.
5.
Record everything so
you and your team
can review the issues.
Open questions and storytelling
Do you like this? What do you think?
Do you understand this? What is this for?
Does this a...
Get users from…
• The next desk
• The canteen
• Your forums
• Market research recruiters
Baby steps: Hall testing
You don’t need working code
Do usability tests in every sprint
Just tell the recruiter to get you “5 users
every thursday.”
Evaluate
Implement
Design ...
Test a mix of stuff
Past FuturePresent
Interviews
about past
experiences
Testing working
software
Testing
mockups and
conce...
Oh my goodness, stakeholders love it!
“The next step involved putting users in a room and
watching them use Obox. It was one of the most eye
opening experiences...
§ Get a team mate who likes
talking to people.
§ Get a target user.
§ Ask the user to do the 3
things the software is f...
Thanks!
Phil Barrett • phil@userexperience.co.za • @philbuktoo
Flickr: Lars ploughman
Upcoming SlideShare
Loading in …5
×

What your customers REALLY think: Incorporating usability testing into agile

7,033 views

Published on

I did this talk for Agile Africa 2014

You can’t know whether your agile project is maximising is impact unless you gather customer feedback. But the feedback that comes to you is not always the full story.

This talk looks at why you should actively go an get user feedback with usability testing, and how to go about doing your first usability test.

Published in: Software

What your customers REALLY think: Incorporating usability testing into agile

  1. 1. What your users really think Incorporating user testing and research into Agile Phil Barrett • Flow Interactive • Agile Africa 2014
  2. 2. UX design, research and strategy London & Cape Town ! Since 1998 ! Hello.
  3. 3. You check out some new digital product.
  4. 4. You “Meh.”
  5. 5. You “Meh.” Dear sir/madam ! I recently visited your new website. I must confess that at first, I found it a bit difficult to understand the value you were offering. ! I wasn’t sure if it was because I was not part of your target market, or if you were still working out exactly what was the best thing to offer. ! I persevered and after a while realised that you were actually providing a potential useful service. I think it could be extremely profitable if you simply make the following changes: ! 1.Ensure that I am not required to register before Not you
  6. 6. Who sends you feedback? No opinion Tentative opinion Strong opinion Not using Trying out Casual user Evangelist/Beta group User
  7. 7. Who sends you feedback? Feedback Silence No opinion Tentative opinion Strong opinion Not using Trying out Casual user Evangelist/Beta group User
  8. 8. Missing feedback breaks Agile’s awesomeness Build Learn Measure
  9. 9. Agile projects deliver gradual, incremental change so it’s easy to miss overall user impact. Flickr: Lars ploughman
  10. 10. Web analytics, tracking What do people do with the product? ! Hard data. Split testing What will people do if we try something else? ! Experimentation. The learn stack UX testing sessions Why do people do that? ! Causes, inspiration, direction, prediction outside of known situations. ! Innovation.
  11. 11. F2F user testing gives you a blast of reality
  12. 12. I don’t need no stinkin’ feedback • I know what people want • My software is clearly good. I mean: look! • Customers don’t know what they want Flickr:DanielDionne
  13. 13. I know 
 what people want
  14. 14. You’re much better with software 
 than most of your users Website tasks:
 the slowest 25% of users take 
 2.4 times
 as long
 as the fastest 25% of users Your usersYou http://www.nngroup.com/articles/variability-in-user-performance/
  15. 15. Econ: Considers, calculates, makes optimal decisions Human: Emotions, shortcuts and irrationality Real people are unpredictable
  16. 16. My software is good: look!
  17. 17. Dan Ariely did an experiment. With origami frogs. They were hard to make and most people did a bad job. How much would people bid for their own frogs? And the frogs of others? And expert -made frogs? Flickr:ToddJordan/Tojosan
  18. 18. Flickr:Nanimo We think the things we make are expert quality. • Average bid for expert-made frog: 27¢ • Average bid for own frog: 23¢ • Average bid by someone else for that same frog: ¢5c Even when they are not.
  19. 19. “ The 9x effect Executives, overvalue their own innovations... Companies overweight the new product’s benefits by a factor of 9 
 John T. Gourville 
 Harvard Business School 3 Consumers overweight the incumbent product’s benefits by a factor of 3
  20. 20. Customers don’t know what they want
  21. 21. Humans are bad at imagining the future
  22. 22. Understand the jobs to be done
  23. 23. “To design an easy-to-use interface, pay attention to what users do, not what they say. ! Self-reported claims are unreliable, as are user speculations about future behaviour. Jakob Nielsen 
 NNGroup Use observation
  24. 24. “There is a direct correlation between the number of hours each team member is exposed directly to real users and the improvements we see in the designs. ! It's the closest thing we've found to a silver bullet. Jared Spool
 UIE
  25. 25. Let’s make it incredibly easy
  26. 26. Doing a usability test 1. 
 Get an interface.
 List key tasks.
  27. 27. 2. 
 Ask someone new to try doing the tasks, and think aloud.
  28. 28. 3. 
 Don’t interfere. Just write things
 down. ! Wait for 4 seconds.
  29. 29. 4. 
 Q. “Is this right?” 
 A. “What do you think?” ! ! Be weird but friendly.
  30. 30. 5. Record everything so you and your team can review the issues.
  31. 31. Open questions and storytelling Do you like this? What do you think? Do you understand this? What is this for? Does this annoy you? How does this make you feel? Do you want this? When will you use this? Do you usually do this? Tell me the story of the last time you did something like this…
  32. 32. Get users from… • The next desk • The canteen • Your forums • Market research recruiters
  33. 33. Baby steps: Hall testing
  34. 34. You don’t need working code
  35. 35. Do usability tests in every sprint Just tell the recruiter to get you “5 users every thursday.” Evaluate Implement Design and analysis Design and analysis Implement Design and anal Evaluate Implement Evaluate Evaluate Design and analysis Implement
  36. 36. Test a mix of stuff Past FuturePresent Interviews about past experiences Testing working software Testing mockups and concepts
  37. 37. Oh my goodness, stakeholders love it!
  38. 38. “The next step involved putting users in a room and watching them use Obox. It was one of the most eye opening experiences of our professional careers. Watching a layman use your product will blow your mind. You cannot even begin to imagine how your users interact with it. Obox blogged about their usability testing experience David Perel
 CEO of Obox
  39. 39. § Get a team mate who likes talking to people. § Get a target user. § Ask the user to do the 3 things the software is for. § Record it. Your MVUT Flickr: Lali Masriera/visualpanic
  40. 40. Thanks! Phil Barrett • phil@userexperience.co.za • @philbuktoo Flickr: Lars ploughman

×