Presentation from meetup "IoT Under Test" organized by Silvair. Tomasz shared with you: ✅ overview of BDD approach and Gherkin syntax ✅ good practices for writing Gherkin scenarios - based on his experience

1. Behave yourself
Gherkin steps that matter

2. Tomasz Lelental
Firmware QA/Test Automation
tomasz.lelental@silvair.com

3. The plan
● BDD/Gherkin theory
● Good and bad practices - Examples
● Recap

4. Behavior Driven Development
● Connect business people with technical people
● Understand product requirements before the implementation
● Small iterations for faster feedback
● Create product documentation that can be automatically checked
against the product’s behaviour
➔ Concrete, real-word examples that illustrate intended behaviour of the system
https://cucumber.io/docs/bdd/

5. concrete examples in BDD
● Collaboration platform (PO, developers and testers)
● Proof of common understanding
● Guidance for implementation
● Test automation (“Living documentation”)

6. Feature file with Examples
● Understandable by all: Client, Developers, Testers
○ Functionality-oriented (no implementation details)
○ Ubiquitous language
○ No room for assumptions
● Resistant to (some) changes in the product

7. Feature files
LightTemperature.feature
LightLightness.feature
Feature: SVM and POD thresholds are configurable
Background:
Given DuT connected to a power source
Scenario: Startup threshold to Shutdown threshold relation - sufficient difference
Given DuT has configured following parameters
| param_name | param_value |
| supply voltage startup threshold | 1.6 |
| supply voltage shutdown threshold | 1.5 |
Then DuT should start at '1.6' V on SVM/POD pin
And DuT should shutdown at '1.5' V on SVM/POD pin
Scenario: Thresholds after Factory Reset
Given DuT has configured following parameters
| param_name | param_value |
| supply voltage startup threshold | 2.0 |
| supply voltage shutdown threshold | 1.0 |
And provisioned DuT
When I factorize DuT
Then DuT should start at '2.0' V on SVM/POD pin
And DuT should shutdown at '1.0' V on SVM/POD pin
CustomizableSvmPod.feature

8. Steps and their roles
● „Given” (Zakładając / Mając):
○ set up the environment (with confirmation)
● „When” (Jeżeli / Kiedy):
○ trigger an action
○ no requirement-related assertions
● „Then” (Wtedy):
○ confirm
○ only assertion (with optional query)
● „And” (Oraz), „But” (Ale):
○ follow the preceding Given/When/Then

9. ● Declarative:
Declarative vs. Imperative
● Imperative:
Open ‘myfancywebsite.com’ page in a web browser
Click the ‘ log in’ link
Enter ‘Mariolka91’ in the username field
Enter ‘mojmisiaczek123’ in the password field
Click the ‘ continue’ button
Log in with valid credentials to My Fancy Website
“User has to be logged in to gain access to specific data.”

10. Gherkin - frameworks
https://cucumber.io/docs/installation/

11. ● BDD/Gherkin theory
● Good and bad practices - Examples
● Recap

12. Our universe
Device
under
Test
(DuT)
Actor
(the “I”)

13. Use step prefixes according to their meaning
Scenario: Update the Lightness value
When DuT has Light Lightness state set to 0x0000
Then I send to DuT: Light Lightness Set with value 0x1234
And DuT should have Light Lightness state equal to 0x1234
Scenario: Update the Lightness value
Given DuT has Light Lightness state set to 0x0000
When I send to DuT: Light Lightness Set with value 0x1234
Then DuT should have Light Lightness state equal to 0x1234

14. Avoid irrelevant implementation details
Scenario: state value after a power-cycle
Given DuT has Light Lightness state set to 0xABCD
And DuT has Generic On Power Up state set to restore
When I restart DuT with the relay on +5V line of the power cable
Then DuT should have Light Lightness state equal to 0xABCD
Scenario: state value after a power-cycle
Given DuT has Light Lightness state set to 0xABCD
And DuT has Generic On Power Up state set to restore
When I power-cycle the DuT
Then DuT should have Light Lightness state equal to 0xABCD

15. Avoid irrelevant information
Given DuT has Light Lightness state set to 0xABCD
And DuT has Generic Level state set to 0x0000
And DuT has Light Lightness Range state set to 0x0000 0xFFFF
And DuT has Generic On Power Up state set to restore
And DuT has Generic Default Transition Time state set to 0x00
When I power-cycle the DuT
Then DuT should have Light Lightness state equal to 0xABCD
Given DuT has Light Lightness state set to 0xABCD
And DuT has Generic On Power Up state set to restore
When I power-cycle the DuT
Then DuT should have Light Lightness state equal to 0xABCD

16. Avoid irrelevant timeouts …
Given I wait for connecting to proxy 20 seconds
Given connected to mesh network proxy
or
Given connection to mesh network proxy is established
@step("I wait for connecting to proxy {timeout} seconds")
def connection_proxy(context, timeout):
proxy_connection = ProxyConnection()
proxy_connection.connect()
… and use the variable if defined

17. Wrap all but one When+Then pair into single Given or Then steps
When I send to DuT: Light Lightness Set with payload 0x1234
Then DuT should send: Light Lightness Status with payload 0x1234
When I send to DuT: Light Lightness Set with payload 0xABCD
And I send to DuT: Light Lightness Get
Then DuT should send: Light Lightness Status with payload 0xABCD
Given DuT has Light Lightness state set to 0x1234
When I send to DuT: Light Lightness Set with payload 0xABCD
Then DuT should have Light Lightness state equal to 0xABCD

18. Aggregate several steps with the same action into one step with data tables
When I navigate to Adding Page
And I am on Add List Page
When I change rssi
When I select device " First"
And I see alert " The device should draw attention. Do you want to add it"
And I check provisioning device " First"
And I click button to add
And I wait until progress bar is visible
When I select device " Second"
And I see alert " The device should draw attention. Do you want to add it"
And I check provisioning device " Second"
And I click button to add
And I wait until progress bar is visible
When I select device " Third"
And I see alert " The device should draw attention. Do you want to add it"
And I check provisioning device " Third"
And I click button to add
And I wait until progress bar is visible
When I select device " Fourth"
And I see alert " The device should draw attention. Do you want to add it"
And I check provisioning device " Fourth"
And I click button to add
And I wait until progress bar is visible
And I check progress for device
Given mesh devices in the project
| device_name |
| First |
| Second |
| Third |
| Fourth |

19. Given scene "A" is configured with parameters:
| device_name | value_set |
| First | 0 |
| Second | 20 |
| Third | 100 |
| Fourth | 80 |
Make use of data tables in step - to provide data sets
Given user navigates to settings tab
And I navigate to first scene configuration page
And I set value " 0" for device " First"
And I set value " 20" for device " Second"
And I set value " 100" for device " Third"
And I set value " 80" for device " Fourth"
When I save value
Then I see scene successful configuration confirmation
And I click close in Scene Alert page

20. Step should express the real-real intended objective
● not „what/how is being done” but „what it is done for”
Scenario: Garbage Collector
Given DuT has memory filled up to 1234 bytes free
When we write some more data into DuT memory
Then DuT should have more than 1234 bytes of memory free
Background:
Given the Garbage Collector threshold is 1234 bytes
Scenario: Garbage Collector
Given DuT has memory filled under Garbage Collector threshold
When we write some more data into DuT memory
Then DuT should trigger Garbage Collector

21. Scenario: Generic Level Get/Set/Status messages
Given DuT has 'Generic Default Transition Time' state set to '0x00'
When I send 'Generic Level' 'GET' message with 'None' value
Then I should receive 'Generic Level' 'STATUS' message with payload ' [0-9A-F]{4}'
When I send 'Generic Level' 'SET' message with ' 0x1100' value, '0x01' TID
Then I should receive 'Generic Level' 'STATUS' message with payload ' 0x1100'
When I send 'Generic Level' 'GET' message with 'None' value
Then I should receive 'Generic Level' 'STATUS' message with payload ' 0x1100'
When I send 'Generic Level' 'SET UNACKNOWLEDGED' message with '0x2200' value, '0x01' TID
Then I should not receive 'Generic Level' 'STATUS' message within mesh response timeout
When I send 'Generic Level' 'GET' message with 'None' value
Then I should receive 'Generic Level' 'STATUS' message with payload '0x2200'
Split a Scenario with chain of verifications into several single-action ones
Background:
Given DuT has 'Generic Default Transition Time' state set to '0x00'
And DuT has 'Generic Level' state set to '0xAAAA'
Scenario: Generic Level - Get
When I send to DuT: 'Generic Level' 'GET' message with 'None' payload
Then DuT should send 'Generic Level' 'STATUS' message with payload '0xAAAA'
Scenario: Generic Level - Set Acknowledged
When I send to DuT: 'Generic Level' 'SET' message with '0x1100 0x01'
payload
Then DuT should send 'Generic Level' 'STATUS' message with payload '0x1100'
And DuT should have 'Generic Level' State equal to '0x1100'
Scenario: Generic Level - Set Unacknowledged
When I send to DuT: 'Generic Level' 'SET UNACKNOWLEDGED' message with
'0x2200 0x02' payload
Then DuT should not send 'Generic Level' 'STATUS' message
But DuT should have 'Generic Level' State equal to '0x2200'

22. Describe intention/test case in the Scenario name
(do not re-phrase steps)
Scenario: Even smallest trigger value should not make the product
unresponsive
Scenario: Performance with saturated mesh message queue

23. Use explicit “given”/”when”/”then” assignment instead of generic “step”
Given DuT is provisioned
Then DuT is provisioned
@step("DuT is provisioned")
def provision_dut(context):
context.dut.provision()
@given("DuT is provisioned")
@when("we provision DuT")
def provision_dut(context):
context.dut.provision()
@then("DuT is provisioned")
def is_dut_provisioned(context):
assert context.dut.is_provisioned()

24. ● BDD/Gherkin theory
● Good and bad practices - Examples
● Recap

25. Most important to remember
● Express your intention: “why”
○ not necessarily “what”
○ avoid “how”
➔ declarative steps (“no matter how …“)
● Avoid assumptions

26. Most important to remember
● Passive voice for preconditions
● Active voice for interactions
● Gherkin is not yet another programming language

27. Benefits
● Useful for both Developers and Clients (business people)
● Reduced maintenance cost and time
○ Single place of change - implementation hidden in step’s automation layer
○ Easier to understand after long time

28. Questions?

29. Thank you!