Why is it that everyone knows the importance of frequent user testing, yet hardly anyone does it? Because user testing often is time consuming, complex and expensive. It probably doesn’t fit in your development process and thus feels like extra work. To feel reassured you tell yourself to test with users once you have something working, or at the very end of the process. This is strange, because everybody knows that changing your product late in the process will increase costs exponentially. We created a way so that user testing saves time, improves the quality and doesn’t cost a lot of money. Team driven, pragmatic and no extra resources needed. The talk will show how, with only 2 hours every sprint, we focused on creating better products faster. We would love to share our learnings and simple DIY tools that let you start user testing with your current teams tomorrow!

1. UX Research
Agile
Better products faster:
Bring the user into the userstory

2. Chapter title
Anna Witteman
Head of UX Lab
IceMobile
Hello :)

4. Plus
The Netherlands
Appie (AH)
The Netherlands
Azbuka
Russia
Carrefour
France
Big C
Thailand
Alfa Mart
Indonesia
IceMobile: worldwide experience with a strong
focus on Loyalty & Retail

5. • Amsterdam based company (& office in Hong Kong, Toronto)
• 130 employees
• 35 nationalities
• Either a foodie, code king, design nerd, beer brewer or unicorn
IceMobile Amsterdam

6. Body text
• Bullet
Header text
Why?
How?
Customer Insights
Data Analysts & Data Scientist
UX Lab
UX Researchers / Data Storytellers
Who? What? When?
Where?
How many?
How often?
… What next?
RESEARCH TEAM

7. Context ChatPulse UX Testing
Cultural relevant
research
Experience
Mapping

8. THE MARSHMALLOW
CHALLENGE

11. Orient Plan Build Ta-da!
Business
(wo)men
Kindergarten
students
0 18
Minutes
Uh-oh!

12. Agile teams
Product
Backlog
Sprint Backlog
Potentially
Shippable
Product
2-3
weeks
24
hours
Daily Scrum
Meeting
Design
Development
Q/A Testing

13. the problem

14. Ta-da!
Previous
Usability
Testing
Iterative
Usability
Testing
0 Time
Uh-oh!
Involve users

15. Product
Backlog
Sprint Backlog
Potentially
Shippable
Product
2-3
weeks
24
hours
Daily Scrum
Meeting
Design
Development
Q/A Testing
Usability testing
Agile teams

16. findings do not stick

17. mismatch

18. no visible value

19. Transfer information effective
The team should learn most of the user test, not the UX researcher
Fit the user into the process
Do testing so that it is not ‘extra work’
Make results visible
Value should be clear and measurable
We wanted to find a way to:

20. HOW?

21. Which practice proved its value?
Individuals and interactions
Working software
Customer collaboration
Responding to change
over
over
over
over
processes and tools
comprehensive documentation
contract negotiation
following a plan
Agile Manifesto

22. Source: Nielsen, Jakob, and Landauer, Thomas K.: "A mathematical model of the finding of usability problems," Proceedings of ACM INTERCHI'93 Conference (1993), pp. 206-213.
Usability Testing is an iterative process!

23. Iteration is the heartbeat
of agile & user testing
use it!

24. Product = Prototype
• Write a scenario
• Create a prototype
• Recruit users
• Pilot test
• Conduct Tests
• Edit videos
• Analyze video
• Write report
• Present results
User test = Report

27. Our setup is a combination of simple tools

28. Our setup is a combination of simple tools

29. More evaluators = more issues identified
Source: Herttzum, M., Jacobsen, N. (2001). The evaluator’s effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer
Interaction, vol. 13, pp.421-443.

30. Ratio benefits to costs of # evaluators
4 = ideal
#
evaluators
Source: Herttzum, M., Jacobsen, N. (2001). The evaluator’s effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer
Interaction, vol. 13, pp.421-443.

31. Chapter title

33. Chapter title
How to keep track of the findings?
# user that
did this task
# user that had
this problem
time stamps
of this issue
on video
description
of issue

34. Chapter title
How to keep track of the findings?

35. Pulse: iterative User Involvement
•Prioritize: test important part(s) most extensive
•Frequent: reliable results, also test improvements
•Empathize: learn what matters to users
•Optimize: learn to test
Allow responding to change
'I have not failed.
I've just found 10000 ways
that won't work.’
Sir Thomas Edison

36. Pulse UX TestingTraditional UX testing
Iterative UX testing: every 2 weeksUX testing (near) the end of a project
Continuous testingSingle test
Small sessionsBig session
Performed by the teams:
the team members learn most
Performed by researchers:
the researchers learn most
Changes early in the process;
thus less costly
Changes late in the process;
thus very costly
Rise in ‘first time right’ of featuresFew ‘first time right’ of features
120% end-result in 80% of the timeDifficult to measure success

37. What we achieved:
Transfer information effective
The team should learn most of the user test, not the UX researcher
Fit the user into the process
Do testing so that it is not ‘extra work’
Make results visible
Value should be clear and measurable

38. Thank you!
anna@icemobile.com
linkedin.com/in/annawitteman
@Morise
slideshare.net/icemobile

39. We are hiring.
icemobile.com/jobs

40. IceMobileAmsterdam @icemobile@icemobile