We talk a lot about User Experience Design nowadays, but how do you know that what you designed really works? To have a testing mindset is to know whether or not you are designing for the user and not yourself. Understand how Booking.com works with A/B testing when validating ideas, and what are the biggest challenges to be a designer in that environment. In this presentation we’ll see the advantages of obtaining quick feedback with data, so that we can learn from it, iterate and try again!

1. UX + A/B Testing at Booking.com:
Design focused on conversion

2. Maria Lígia
Hi!My name is

5. Maria Lígia
MARIA ELIGIA
MARIA LIDIA
MARIA ELISA
MARIA OLÍVIA

7. Maria

8. USER EXPERIENCE

9. UFSC ChaordicMidiatools CERTI Myreks General Assembly

10. AMSTERDAM

12. BABETTE

13. BABETTE

17. 880+K
properties
174
offices
42
languages
50
currencies
8+K
servers
224
countries &
territories
$64.49 billion
market cap
perl + template HTML
IT languages
3rd largest e-commerce
in the world
10+K
employees

18. Teams from 5 to 10 people
Multidisciplinary teams (PO, Team Leader, Designer, Dev)
REALLY.
Teams are responsible for different areas of the website
Semi-Agile SCRUM methodology
Everything, absolutely everything is tested

19. USER EXPERIENCE

20. DEFINE THE PROBLEM

21. DEFINE PERSONAS

22. DEFINE SCENARIOS
Short story that your persona goes through with your
product related to the problem you defined.

23. USER FLOWS

24. SKETCHING

25. WIREFRAMES

26. DESIGN

27. FRONT END

28. USER EXPERIENCE
Define the problem
Define Personas
Define Scenarios
User flows
Sketching
Wireframes
Design Front End

29. A/B TESTING

30. CONVERSION
What is success to you?
Define your metric

31. Avoids regression
A/B TESTING

32. Keep the customer at the centre of
everything you do

33. A/B TESTING +

34. + 10 years
Highest conversion in the market
Logo
A/B TESTING +

35. VS.

36. Customer Service
Budget for negative experiments
A/B TESTING +
+ 10 years
Highest conversion in the market
Logo

37. TESTING +

38. Usability lab in-house
User Research Team
TESTING +
LOCALIZATION:
Japan

40. The Experiment tool is our user's body language. It
only tells us a percentage of what they are thinking
or wanting.
“
"Erin Weigel
Senior Designer @ booking.com

41. Photos on
the Tablet

42. 150px x 150px 200px 300px 500px
Search Results
Photos on
the Tablet

43. 500px

44. 500px
150px
A
B

45. NEUTRAL

46. 150px x 150px 500px
=
Search Results
Photos on
the Tablet
Increased page load time by 0.5 seconds
=

47. A
B
300px

48. A
B
300px
POSITIVE!

49. Good design at Booking.com isn’t just about making something that
seems to be better. It’s about measuring results, understanding the
impact on the full customer experience, and actually making something
better.
“
"Erin Weigel
Senior Designer @ booking.com
http://blog.booking.com/goldilocks-and-the-three-experiments.html

50. THE HAMBURGUER ICON

51. A
B
?
http://blog.booking.com/hamburger-menu.html

52. A
B
NEUTRAL
http://blog.booking.com/hamburger-menu.html

53. A
B
http://exisweb.net/menu-eats-hamburger

54. A
B
POSITIVE!
http://exisweb.net/menu-eats-hamburger
20% +

55. WHAT NOT TO DO IN A/B TESTS
Test A, then after a while test B
Test more than one idea at the same time
Bet with someone that your change will win
Trust me!

56. HiPPOs !

57. HiPPOs?

58. HiPPOs
Highest Paid Person

59. Design +A/B Test
You = Team Leader = PO = CEO = Founder …

60. Design + A/B Test
Design is hard, but it’s getting harder now.

61. BABETTE

63. TAKE AWAYS

64. UX CHANGES AFFECT
CONVERSION

65. NOT EVERYTHING
IS WHAT IT LOOKS LIKE
Nor how you want it to be, but that’s ok.

66. DON’T RELY
IN GURUS

67. DON’T BELIEVE
IN HiPPOs

68. THE STRONGEST
WILL WIN

69. THE USER IS AT THE CENTER OF
EVERYTHING. AND THEY
GET TO DECIDE.
Even when they have no clue what they want.

70. ITERATE. ALWAYS.
Iteration is essential for a
healthy environment.

71. Experiment fast, fail often and believe in data.
Data never lies.

72. Maria Lígia Klokner
@marialigiak
maria.klokner@booking.com
/marialigiak
http://grnh.se/ef54yh
Dank u wel!
Work with us!