Starting from behat introduction, we have explained some good practices for BDD testing in a symfony software. This presentation should be a quick "guide" for users that starts with Behat and also for those who want to fix their tests problems.

1. Behat from zero to hero
“A practical guide to symfony integration and usage”
Matteo Moretti - Samuele Lilli@SymfonyDayIt - 20 Ottobre 2017 - Milano

2. Matteo Moretti | Samuele Lilli
WHO ARE WE?
Matteo Moretti Samuele Lilli

3. Matteo Moretti | Samuele Lilli
OUR
EXPERIENCE

4. Matteo Moretti | Samuele Lilli
OUR EXPERIENCE
ON nuvola.madisoft.it
https://symfony.com/showcase/191

5. Matteo Moretti | Samuele Lilli
BEHAVIOUR
DRIVEN
DEVELOPMENT

6. Matteo Moretti | Samuele Lilli
BEHAT

7. Matteo Moretti | Samuele Lilli
A LONG TIME
AGO ...

8. Matteo Moretti | Samuele Lilli

9. Matteo Moretti | Samuele Lilli

10. Matteo Moretti | Samuele Lilli
TESTS ARE GREEN!

12. Matteo Moretti | Samuele Lilli
BUT,
SUDDENLY...

13. Matteo Moretti | Samuele Lilli
AS SOON AS
YOU ADD MORE
TESTS...

14. Matteo Moretti | Samuele Lilli
HARD TO MAINTAIN

15. Matteo Moretti | Samuele Lilli
RANDOM
FAILURES

16. Matteo Moretti | Samuele Lilli
SLOW TESTS

18. Matteo Moretti | Samuele Lilli
“WHY THE HELL THIS
TEST IS BROKEN?
I DIDN’T MODIFY THAT
CODE”

19. Matteo Moretti | Samuele Lilli
“MMM, 10 MINUTES TO
ADD A FEATURE. 2
HOURS TO FIX TESTS.”

21. Matteo Moretti | Samuele Lilli

22. Matteo Moretti | Samuele Lilli

23. Matteo Moretti | Samuele Lilli
WHAT IS
BEHAT?

24. Matteo Moretti | Samuele Lilli
IT’S A BDD TOOL!

25. Matteo Moretti | Samuele Lilli
IT’S A
STORY
BDD TOOL!

26. Matteo Moretti | Samuele Lilli
WHAT IS A
STORY?

27. Matteo Moretti | Samuele Lilli
HI! I’M A STORY!

28. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

29. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

30. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

31. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

32. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

33. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

34. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

35. Matteo Moretti | Samuele Lilli
GHERKIN

36. Matteo Moretti | Samuele Lilli
IT’S A LANGUAGE!

37. Matteo Moretti | Samuele Lilli
IT’S A BUSINESS
READABLE
DOMAIN SPECIFIC
LANGUAGE!

39. Matteo Moretti | Samuele Lilli
LIKE YAML
INDENTATION
RULES PARSING

40. Matteo Moretti | Samuele Lilli
STORIES ARE ORGANIZED IN
FEATURES FILES (.feature)

41. Matteo Moretti | Samuele Lilli
GIVEN - WHEN - THEN
(AND, BUT) ARE KEYWORDS
THAT MATCH PHP CODE
AND EXECUTE IT.
STEPS

42. Matteo Moretti | Samuele Lilli

44. Matteo Moretti | Samuele Lilli
REGEX

45. Matteo Moretti | Samuele Lilli

46. Matteo Moretti | Samuele Lilli

47. Matteo Moretti | Samuele Lilli

48. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

49. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

50. Matteo Moretti | Samuele Lilli
HOW A GOOD
STORY LOOKS
LIKE?

51. Matteo Moretti | Samuele Lilli
FOCUS ON OUTPUT NOT INPUT
X

52. Matteo Moretti | Samuele Lilli
√
FOCUS ON OUTPUT NOT INPUT

53. Matteo Moretti | Samuele Lilli
DON’T USE UI REFERENCES
X

54. Matteo Moretti | Samuele Lilli
DON’T USE UI REFERENCES
√

55. Matteo Moretti | Samuele Lilli
USE BACKGROUND

56. Matteo Moretti | Samuele Lilli
USE TRANSFORMERS

57. Matteo Moretti | Samuele Lilli
USE TRANSFORMERS

58. Matteo Moretti | Samuele Lilli
COLLAPSE STEPS AS MUCH AS
POSSIBLE
UNLESS THESE GIVES YOU
BUSINESS/STORY VALUE

59. Matteo Moretti | Samuele Lilli

60. Matteo Moretti | Samuele Lilli
HOW
MUCH
DETAILS?

61. Matteo Moretti | Samuele Lilli
“What many people don’t realize is that
different levels of details are appropriate
for different scenarios in the same system
- sometimes in the same feature -
depending on what it is they’re describing”
(The cucumber book - Matt Wynne, Aslak Hellesoy)

62. Matteo Moretti | Samuele Lilli
BEHAT → GHERKIN → FEATURES →
STORIES (SCENARIOS) → STEPS →
CONTEXTS

63. Matteo Moretti | Samuele Lilli
BEHAT → GHERKIN → FEATURES →
STORIES (SCENARIOS) → STEPS →
CONTEXTS

64. Matteo Moretti | Samuele Lilli
DIVIDE ET IMPERA

65. Matteo Moretti | Samuele Lilli
ONE FEATURE
PER .FEATURE FILE

66. Matteo Moretti | Samuele Lilli
DIVIDE FEATURES
AND CONTEXTS
BY DOMAIN SUBJECT

67. Matteo Moretti | Samuele Lilli
DIVIDE FEATURES
AND CONTEXTS
BY ACTIONS TO BE
PERFORMED ON
DOMAIN SUBJECTS

68. Matteo Moretti | Samuele Lilli

69. Matteo Moretti | Samuele Lilli

70. Matteo Moretti | Samuele Lilli
UI

71. Matteo Moretti | Samuele Lilli
HEADLESS BROWSER EMULATOR
VS
BROWSER CONTROLLER

72. Matteo Moretti | Samuele Lilli
HEADLESS BROWSER EMULATOR
● pure HTTP spec implementations
● very simple to run and configure
● no need of a real browser: speed
● usually no JS/AJAX support

73. Matteo Moretti | Samuele Lilli
BROWSER CONTROLLER
● aim to control a real browser
● complex to configure
● slow
● JS/AJAX support

75. Matteo Moretti | Samuele Lilli
YOU MAY NEED BOTH OF THEM

76. Matteo Moretti | Samuele Lilli
MINK
http://mink.behat.org/en/latest/

77. Matteo Moretti | Samuele Lilli
BEHAT PARSES SCENARIOS
+
MINK BROWSES THE WEB

79. Matteo Moretti | Samuele Lilli

80. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

81. Matteo Moretti | Samuele Lilli
HOW TO USE MINK
FROM CONTEXT
CLASSES?

82. Matteo Moretti | Samuele Lilli
PAGE OBJECT PATTERN
https://martinfowler.com/bliki/PageObject.html

85. Matteo Moretti | Samuele Lilli
https://github.com/sensiolabs/BehatPageObjectExtension
BEHAT PAGE OBJECT EXTENSION

86. Matteo Moretti | Samuele Lilli
ASSERTIONS

88. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

89. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

90. Matteo Moretti | Samuele Lilli
AJAX

91. Matteo Moretti | Samuele Lilli
DON’T PULL YOUR
TESTS WITH SLEEP()
CALLS

93. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

94. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

95. Matteo Moretti | Samuele Lilli
UI END-TO-END TESTS
● BRITTLE
● EXPENSIVE TO WRITE
● TIME CONSUMING TO RUN
● MORE AFFECTED BY NON-DETERMINISM

96. Matteo Moretti | Samuele Lilli

97. Matteo Moretti | Samuele Lilli
https://martinfowler.com/bliki/TestPyramid.html

98. Matteo Moretti | Samuele Lilli
DO YOU REALLY NEED A REAL JS BROWSER?
https://blog.eleven-labs.com/en/behat-structure-functional-tests/

99. Matteo Moretti | Samuele Lilli
STICK TO A BROWSER EMULATOR
USE @javascript ONLY WHEN NEEDED

100. Matteo Moretti | Samuele Lilli
FIXTURES DATA
● NO FIXTURES (INTEGRATION TESTS)
● LOAD FIXTURES AT RUNTIME
● IN MEMORY DBMS (SQLITE)

101. Matteo Moretti | Samuele Lilli
PARALLELISM
● SPEED UP PERFORMANCE
● COMPLEX SETUP AND DEBUG
DO YOU NEED IT?

102. Matteo Moretti | Samuele Lilli
FASTEST
https://github.com/liuggio/fastest
bin/behat --list-scenarios --config
tests/behat/behat.yml | bin/fastest -vv --process=4
"bin/behat --stop-on-failure --config
tests/behat/behat.yml {}"

103. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

104. Matteo Moretti | Samuele Lilli
SAVE OUTPUT FOR
FAILED TESTS

106. Matteo Moretti | Samuele Lilli
DEBUG

107. Matteo Moretti | Samuele Lilli
CLI DEBUG
● BEHAT CODE
● APP CODE IF INTEGRATION TEST (NO UI)
● APP CODE IF UI TESTS WITH BROWSER
EMULATOR (BrowserKit)

108. Matteo Moretti | Samuele Lilli
WEB DEBUG
FOR APP CODE IF BROWSER CONTROLLER
(SELENIUM)

109. Matteo Moretti | Samuele Lilli
RECAP

110. Matteo Moretti | Samuele Lilli
TREAT YOUR TEST
CODE AS YOUR
APP (PROD) CODE

111. Matteo Moretti | Samuele Lilli
WRITE CLEAN,
CONCISE AND
WELL ORGANIZED
FEATURES

112. Matteo Moretti | Samuele Lilli
SPLIT YOUR
CONTEXT CLASSES
AND
COMPOSE / REUSE
THEM

113. Matteo Moretti | Samuele Lilli
REMEMBER TEST PYRAMID
FOCUS ON INTEGRATION LEVEL

114. Matteo Moretti | Samuele Lilli
USE @javascript WISELY
USE PAGE OBJECTS FOR UI

115. Matteo Moretti | Samuele Lilli
THANK YOU

116. Matteo Moretti | Samuele Lilli
WE ARE HIRING!
(wanna join? ask us or visit our website)
labs.madisoft.it

117. Matteo Moretti | Samuele Lilli
https://joind.in/talk/707a1

118. QUESTIONS?