Recommended
More Related Content
What's hot
Similar to Continuous Quality
Similar to Continuous Quality (20)
Recently uploaded
Recently uploaded (20)
Continuous Quality
- 1. Continuous Quality a proposal integrating BDD+Selenium, and their integration with QA QA in DevOps Methodologies
- 2. Summary Introduction to TDD/BDD model TDD Model BDD Model BDD in action *DD Model within V-Model Tweak V-Model Characteristics of BDD Model How to get standard documentation
- 3. Test Driven Development Behavior Driven Development model Testing our application is a crucial part of the development lifecycle. Being able to simulate what a user encounter while using your application is a challenge. TDD is a software development model that involves the creation of the automatic test before the creation of software. BDD is a test-driven development methodology whose purpose is to ensure that the development team fully understand the end user’s requests
- 4. TDD modelTest-driven development is a software development model that involves the creation of the automatic test before the creation of software that must be tested TDD provides for the repetition of a short development cycle in three phases known as "TDD cycle": Red-Green-Refactoring (or Red-Green-Grey): 1. RED In the first phase, the programmer writes an automatic test for the new function to be developed, which must fail because the function has not yet been realized. 2. GREEN In the second phase, the programmer develops the minimum amount of code needed to pass the test. 3. GREY or Refactoring In the third phase the programmer performs a refactoring of the code to match certain quality standards. ✔ ✘
- 5. BDD model: TDD generalized Behavior-driven development is a test-driven development (TDD) methodology. The purpose is to ensure that the development team fully understands the end user's requests and that the end user's understands what the development team has understood. During the BDD development process, features are described with the user stories. Once a user story is defined, we can focus on the possible scenarios where the feature is executed. The scenario of a user story represents the acceptance test once the feature development is completed.
- 6. BDD in action • We write the Features and scenarios Feature Scenario 1 Scenario 2 Scenario n Unit Tests Fix the errors FailPass Integration Tests • We write the Unit Tests • We launch the Unit Tests (Fail) • We write the code to fix the tests • We write the Integration Tests
- 7. How to use it within V-Model?
- 8. BDD Behavior BDD Feature Scenario Development Unit Test Integration Test Behavior Write Features Write Scenarios Development (Re)Write Tests Write Production Code Refactoring test test Fail Fail Pass
- 9. Unit test (?) Unit test (?) Behavior V-model + BDD Feature Scenario Development Unit Test Integration Test URS Dev FRS OQ UAT/PQ V-MODEL Demo Dry OQ Code and doc freeze DS BDD DSDS
- 10. Behavior V-model + BDD + Automation Feature Scenario Development Unit Test Integration Test BDD URS Dev FRS OQ UAT/PQ V-MODEL Demo Dry OQ Code and doc freeze DS DSDS Unit test (?)
- 11. Unit test (?) Vmodel + BDD + Automation URS Dev FRS OQ UAT/PQ V-MODEL Demo Dry OQ Code and doc freeze DS DSDS Automatic testing Formal testing Translation
- 12. Report of BDD model within QA Dev team attends the meeting that sets the needs of the business URS, FRS are covered by User Stories User Stories (Features + Scenarios) are written by the business and the dev team. User stories are written in natural language following Gherkin syntax. IQ/UT URS FRS DS OQ UAT/PQ Dev User stories
- 13. How does it work ? Gherkin
- 14. Behavior Driven Development • URS and FRS are created together with business, development and validation team User Requirements Specification = Features Functional Requirements Specification = Scenarios • The user stories are written in a natural language with keywords called Gherkin. • The Behave Python library reads the user stories file. User Stories
- 15. User Stories • Features are composed of scenarios and describe clear business goals • Scenarios consist of steps and describe a specific aspect of behavior of the system
- 16. How to write Features ? Features = User Requirements Specifications Clear Title Feature : one line describing the story / feature As a [role] I want [feature] So that [benefit]
- 17. How to write Features ? Example Features = User Requirements Specifications Clear Title Feature : Account Holder withdraws cash As an Account holder I want to withdraw cash from an ATM So that I can get money when the bank is closed
- 18. How to write Scenarios ? Scenarios = Fonctional Requirements Specifications Clear Feature Scenario : some determinable business condition Given [Context] When [event occurs] Then [expected observable outcome]
- 19. How to write Scenarios ? Scenarios = Fonctional Requirements Specifications Clear Feature : Account Holder withdraws cash Scenario : Account has sufficient funds Given the account balance is 100.- When I request 20.- Then the ATM dispenses 20.- Scenario 2 : Account has insuffient funds Scenario 3 : Card has been disabled
- 20. Scenario Outlines Copying and pasting scenarios to use different values can become tedious and repetitive Use scenario outlines
- 21. Scenario Outlines - Example Feature: Adaptative ATM As an account holder I want to be able to choose the language of the ATM So that I understand the messages displayed Scenario Outline: Change language displayed by ATM Given I am in the main page When I click on the <language> button Then the ATM displays <greetings> Examples: | language | greetings | | french | Bonjour | | german | Guten Tag | | italian | Buongiorno | | english | Hello |
- 22. How does it works ? Automation with Behave and Selenium
- 23. Behave: Gherkin language interpreter @given('I am in Google main page') def step_given_in_google(context): go_to_website('www.google.com') @given('I have typed my website name') def step_given_website_typed(context): type_my_website_name(‘Wikipedia')) @when('I click on the " Search" button') def step_when_button_click(context): click_on_search_button() @then('the search result page is displayed') def step_then_result_should_be_displayed(context): assert(getCurrentPageTitle == ‘wikipedia – Google Search') @then('the first result is my website') def step_then_I_am_first(context): assert(getFirstResult()== ‘Wikipedia') Feature: Search engine rank check As a website owner In order to assess the popularity of my site I want to check if my website is the first on a search engine Scenario 1: Get the first website in Google Given I am in Google main page And I have typed my website name When I click on the “Search" button Then a search result page is displayed And the first result is my website
- 24. Selenium: Web Browser automation from selenium import webdriver driver = webdriver.Firefox() def go_to_website(website): driver.get(website) def type_my_website_name(website): elem = driver.find_element_by_id('lst-ib') elem.clear() elem.send_keys(website) driver.save_screenshot('screenshot.png') go_to_website('http://www.google.com') type_my_website_name('wikipedia')
- 25. Big picture … Scenario 1: Get the first website in Google Given I am in Google main page And I have typed my website name When I click on the “Search" button … from selenium import webdriver driver = webdriver.Firefox() def go_to_website(website): driver.get(website) def type_my_website_name(website): elem = driver.find_element_by_id('lst-ib') elem.clear() elem.send_keys(website) driver.save_screenshot('screenshot.png') … … click_on_search_button() @given('I am in Google main page') def step_given_in_google(context): go_to_website('http://www.google.com') @given('I have typed my website name') def step_given_website_typed(context): type_my_website_name(‘Wikipedia')) @when('I click on the " Search" button') …
- 26. How does it work ? “Easy recipe” A practical start-to-finish demo
- 27. Goal Make a culinary recipe website Make it adaptable to a variety of diet D S
- 28. User Stories are described in Gherkin Feature: Recipe management As a cooking website user In order to find what to eat I want query the database for recipes Scenario: Display an existing recipe Given I am in the main page And I have typed a recipe name in the search field And the recipe exists in the database When I click on the "Go to recipe" button Then the selected recipe is displayed Scenario: No recipe in database Given I am in the main page And I have typed an imaginary recipe name in the search field And the recipe does not exists in the database When I click on the "Go to recipe" button Then an error message "The recipe you're looking for can't be found" is displayed Make a culinary recipe website Make it adaptable to a variety of diet D S Gherkin feature file (prose)
- 29. User Stories are described in Gherkin Make a culinary recipe website Make it adaptable to a variety of diet Feature: Adaptive ingredient list As a cooking website user with special dietary needs In order to have a meal compatible with my eating habits I want to easily adapt the recipe to my needs Scenario Outline: Ingredient replacement Given I am in a recipe page And I have a selected the "<dietary_need>" options When I click on the "Adapt" button Then the "<milk>" ingredient is replaced by "<milk_alternative>" And the "<egg>" ingredient is replaced by "<egg_alternative>" And changes from the original recipe should be highlighted Examples: Ingredient list substitution | dietary_need | milk | milk_alternative | egg | egg_alternative | | Original | milk | milk | eggs | eggs | | Lactose intolerant | milk | goat milk | eggs | eggs | | Vegetarian | milk | milk | eggs | eggs | | Vegan | milk | soja milk | eggs | margarine | D S Gherkin feature file (prose)
- 30. Feature: Recipe management As a cooking website user In order to find what to eat I want query the database for recipes Scenario: No recipe in database Given I am in the main page And I have typed an imaginary recipe name in the search field And the recipe does not exists in the database When I click on the "Go to recipe" button Then an error message "The recipe you're looking for can't be found" is displayed Scenario: Display an existing recipe Given I am in the main page And I have typed a recipe name in the search field And the recipe exists in the database When I click on the "Go to recipe" button Then the selected recipe is displayed Automated tests are written with Behave, using Gherkin @given('I am in the main page') def step_impl(context): context.browser.get('file:///C:/PMI_Demo/code/index.html') @given('I have typed an imaginary recipe name in the search field') def step_impl(context): elem = context.browser.find_element_by_id('q') context.recipe = 'Pan galactic gargle blaster' elem.send_keys(context.recipe) @when('I click on the "Go to recipe" button') def step_impl(context): context.browser.find_element_by_id('search-btn').click() D S Gherkin feature file (prose) Behave file (code)
- 31. Let’s begin the development !
- 32. Cycle 1: Tests are run, nothing is working … entire error message omitted for clarity … D S
- 33. Cycle 2: Dev code in order to get as much scenario and features passing as possible D S
- 34. Cycle 2: Demo to business Outcome of the meeting: • “So far so good, don’t forget to display the ingredient list in a tabular fashion” • “The website crashes when I type ‘これは動作していません。’ ” • “We should add ‘halal’” • “We should have a check for the price of ingredients” D S
- 35. This is an additional scenario, so: 1. Modify the Ingredient list substitution table 2. Modify the tests so that it adapts 3. Develop the feature so that the test passes 4. Refactor to a higher standard “We should add ‘halal’” What to do ? Behavior Write Features Write Scenarios Development (Re)Write Tests Write Production Code Refactoring test test Fail Fail Pass D S
- 36. Behavior Write Features Write Scenarios Development (Re)Write Tests Write Production Code Refactoring test test Fail Fail Pass “Weshouldhaveacheckforthepriceof ingredients” Whatto do ? • This is a whole new feature that is out of scope of the current feature set. • Tell the business to wait for the next release D S
- 37. Cycle 3: Dev produce passing but unrefined working code Code in the same state as before, only one scenario has been added: 4 failed scenarios instead of 3 D S
- 38. Cycle 3: (optional) Demo to business D S Outcome of the meeting: • “Everything is fine !”
- 39. Cycle 4: Refactor to higher code standard Regression test: we still perform tests so that we are confident that the refactoring haven’t impacted functionality D S
- 40. Cycle 4: Demo to business D S Outcome of the meeting: • “Everything is still fine !” • → Proceed to code and doc freeze, and formal validation
- 41. Automation can help formal validation • Actions in Behave are modular and executed sequentially to test for behavior → this is nearly equivalent to an OQ test script ! • Steps can be written so that within each step in Behave there is a human instruction associated • This can be used to generate a draft version of an OQ test script @given('I am in the main page') def step_impl(context): context.browser.get('file:///C:/PMI_Demo/code/index.html') # Instructions step_instruction = """ Get to the main page """ context.human_instructions.append(step_instruction) # /Instruction END @when('I click on the "Go to recipe" button') def step_impl(context): context.browser.find_element_by_id('search-btn').click() # Instructions step_instruction = """ Click on the "Go to recipe" button. """ context.human_instructions.append(step_instruction) # /Instruction END D S Behave file (code)
- 42. Automation can help formal validation @given('I am in the main page') def step_impl(context): context.browser.get('file:///C:/PMI_Demo/code/index.html') # Instructions step_instruction = """ Get to the main page """ context.human_instructions.append(step_instruction) # /Instruction END # /Instruction END @when('I click on the "Go to recipe" button') def step_impl(context): context.browser.find_element_by_id('search-btn').click() # Instructions step_instruction = """ Click on the "Go to recipe" button. """ context.human_instructions.append(step_instruction) # /Instruction END D S Behave file (code) OQ Test script (prose) This document still needs to be modified, formally reviewed and validated
- 43. User acceptance test & Deployment into Prod D S
- 44. Outcome of this process • Business, Validation and Development were involved from the start on a common document. → Minimal miscommunication. → Business gets exactly what it wants → Validation can validate faster and more confidently → Developer has a clear goal
- 45. Responsibilities Business responsibilities Provide feedback throughout the process Validation responsibilities Ensure that the user stories are as complete as necessary Developer responsibilities Implement user stories and help validation
- 46. Test script execution Executor Validator <None> Test script Author Validator <None> Automation: link between automated and formal testing OQ Dry OQ Code and doc freeze Translation from machine code to human language Test script Author (with help from ) Validator Test script execution Executor Validator Tester Validator Robot Developer
Editor's Notes
- DevOps methodologies are not adverse to QA and more generally to V-Model. What we intend to do now is to offer you a different working method starting from a tool designed for software development and its use within our current work cycle.
- Importance of tests in dev lifecycle Introducing TDD and BDD models The part concerning the tests within the life cycle of a software in general is very important; in fact, be able to simulate what a user will encounter during the use of the software, predict the steps and store this information are fundamental parts of the DevOps methodologies. EXPLAIN NOW TDD/BDD Starting from the old way of thinking about the software life cycle, or the waterfall, we can realize that we start from the concepts of requirements and design to get to the implementation and to finish with the checks; summarizing we can say that we start from the implementation that allows us to easily create code but that makes the part reserved for testing difficult. With the TDD the fundamental and primary part of the whole cycle are the tests, which allow us to write the code in a simple, effective and fast way. We pass then from Implementation first Easy to code Difficult to test To Test first Easy to test Easy to code
- Model operation Red phase Green phase refactoring
- Evolution of *DD going from Tests to Behavior Introducing terms of user story and scenario User story is a human readable text that define the expected behavior of a software. This will be discussed in much more detail in later slides.
- Introducing all the steps you can find in BDD `Feature and scenario will be explained much more in detail after
- Identify the 2 macro section of BDD methodology starting from a detailed flowchart Moving on to the BDD model we can distinguish two macro phases: Behavior and Development. During the first phase the most important thing is to write the Features and the various Scenarios based on the needs of the business. Starting from this information we can write the tests that will guide the drafting of the code up to the finishing touch (Refactoring) before to move on the next scenario and so on until we’ll complete all the Features.
- Identify possible association between the BDD entities and V-Model Feature -> URS Scenario -> FRS Unit Test and Integration Test -> Dev Seen that Agile methodology imposes a flexibility with possible changes in FRS, the DS are written after the Dev and before the Dry OQ The BDD model and the V-Model can be related. This allows us to redesign the path of the V-Model without affecting the Agile philosophy. The Features correspond to the URS, the Scenarios correspond to the FRS and the writing of the Unit Test/Integration Test with the implementation of the code, corresponds to the development phase. At this point the Design Spech are written immediately after having implemented the code so as not to continuously modify them; Unit Test if not explicitly requested by the complexity of the project can be considered as optional but already present, even before the code. Otherwise everything remains unchanged.
- Explain how Dry OQ is done Introducing the Selenium utility Explaining how automation is done and how it is possible to use this layer for Dry OQ Introducing the Selenium tool (like any other automation tool applicable to the context) we can speed up the process even further. Selenium is a tool that allows us to automate the so-called GUI (Graphical User Interface) tests that aim to simulate human behavior to test the correct functioning of the graphic interface, that is to simulate a human being using our software. Starting from a file written with an human readable language (Gherkin language) it is possible to instruct the tool to automatically perform all the actions that a human being would do. In the traditional V-Model we would have a tester who, according to the FRS, draws up a document with all the actions to be taken and with the results to be obtained for each action, then another tester following the document and generates all reports. All of this can be accelerated using a translation layer that compiles the document and the various reports starting from what Selenium does in automatic.
- Short report of BDD model within QA PAUSE NEXT SLIDE
- PAUSE HERE
- Behaviour-driven development is a highly collaborative process. User requirements specifications and Fonctionnal requirements specifications are written together by business, development and validation. The important thing is not the number of participants but the pluridisciplinarity of their profiles. URS correspond to features and FRS to Scenarios. Together, features and scenarios are called User Stories. User Stories are written in Gherkin, a structured natural language. Then, developpers use Behave which is a python library that provide a translation layer/framework to interpret the Gherkin into workable Python code. The Three Amigos (BA, developer, and QA) discuss the new feature and review the specification. The aim is to create a common understanding and shared vocabulary across these individuals. Requirements and tests need to be visible and understandable to all members of a Scrum team.
- I will describe how to write user stories in Gherkin. User Stories are composed of features, describing clear business goals. Features are composed of scenarios, describing the specific aspect of behavior of the system. This is an example of what the collaborative file in Gherkin langage looks like.
- I will describe how to write a feature, which correspond to user requirement specification When writing a feature, we begin with a line, describing the story, clarifying the business need. Then the feature include an «As a», «I want» and «So that» block, which illustrates the motivation behind the story.
- Example. If you want your system to allow you to withdraws cash, the feature can be written as : …
- Each feature contains a list of scenarios, describing some business condition. In order to be able to describe a scenario, it is important to follow a three steps schema, using Given-When-Then keywords : Given serves to put the system in a known state before the user starts using it when describes the key action performed by the user Then is the observed outcomes
- Example corresponding to the precedent feature : The first scenario can be «Account has sufficient funds» Other possible scenario corresponding to the same features could be «Account has insuffient funds» or «Card has been disabled»
- When writing user stories in Gherkin, in order to avoid copy pasting scenarios with different values, it is possible to use «Scenarios Outlines»
- Scenario outlines allow us to more concisely express these examples through the use of a template with placeholders, using Scenario Outline, Examples with tables, and delimited parameters In this example, to avoid writing one scenario per langage, we use Scenario outline, with “Langage” and “Greetings” being the placeholders. The placeholders indicate that when the Examples row is run, they should be substituted with real values from the Examples table
- PAUSE HERE
- Gherkin is a structured natural language Behave is a python library that provide a translation layer/framework to interpret the Gherkin into workable Python code. Behave does not automatically write tests based on the Gherkin file, but can extract useful data from it. That is why it is very important to have a solid specification file beforehand. In Behave, you write code to check each element/line of the Gherkin file
- Selenium is a multilingual set of tool to automate browser action. In other words Selenium is a robot we can program to perform stuff on a browser. Here “stuff” includes pressing buttons, filling forms, etc… It can also save screenshot. <Demo>
- Taking a higher view: We write a Gherkin file We use Behave to interpret this Gherkin file Within Behave, we use Selenium to interact with the web, in our case Clarity LIMS. Selenium can retrieve file for us (for instance instrument driver file). Parse It is possible with Selenium to get files (for instance driver files) and parse them while still within Python to be sure that they conform to a template. PAUSE NEXT SLIDE
- PAUSE HERE So, so far we have discussed tools and process, but maybe you can have a better view of how those pieces fit together by showing you a more extensive demo. We have created a mini project to showcase you all the different part we have previously shown.
- Our mini project is a website that suggest recipe. Not only that but it can adapt the recipe to a variety of diet <show the final product>
- Step 1 … is to write the user stories User stories are written by a committee of 3 people minimum. Here we have a feature that describe the basic functionality of a recipe website, that is: How to display a recipe What happens when the recipe cannot be found.
- This user story regards the adaptability of the recipes. This user story shows an example of a scenario outline If a group of scenario shares a lot of thing except one or two elements, we can summarize this in a compact way with a table Here we can see that if we have a lactose intolerant user, we have to substitute milk to goat milk
- As explained before, Behave is a Python library that reads the Gherkin file. The developer implement in Python the step necessary to check each statement of the Gherkin file, using Selenium if needed. Behave can then run theses step and report an error if the behavior is unexpected. <Demo with a wait of 2.5s for each step of the Behave code for the completed website.> This is just a demo of the test framework Now step 2
- Step 2 is dev
- Let’s go back in time, we are back in the beginning of the development. Cycle 1 If we run behave, nothings works because there’s nothing to test against. <PMI_Demo - Beginning state> Code is empty Test is failing
- Cycle 2 Developers are supposed to write code in order to pass the tests. <PMI_Demo Early state> This is ugly. Some stuff is broken. Some is working At this point, if there’s a small unit of code that is completed and showable: demo
- There still can be demo when a small unit of code is completed Code and functionality are shown Other needs can be addressed during the demo precision about the implementation bug report. Some other requests are related to the development process I want to focus on the last two. One is about pleasing our muslim friends The other one is about adding a grocery list
- Adding a halal diet is a modification of a scenario. It is covered by the development process: Modify the test Write code Run code against the test
- On the contrary, adding a price management to the website is considered like adding another feature. That’s why it is super important to have at least all the feature at the beginning so we can properly plan.
- So here we show a another round of development. At the end of this cycle, we have worked well and all the test pass. We can demo, but this optional <PMI_Demo - Minimal Working code> This is ugly. Code is unreadable. But working But the website isn’t finished ! We still have to refactor.
- At this point we can have another regular demo to show to the business that everything works well. This demo is optional, depending on the mood and time of everyone.
- During refactoring, we make sure that the code follow a standard of quality: is readable, maintainable and efficient. <PMI_Demo> Code is readable, etc.. We use regression test so that we are more confident that we didn’t break something during the refactoring. (this can also be used while in production to make sure Validated Changes are not breaking functionality)
- We can have a final demo that should represent the state of the system in production. At this point we can go up in the V-cycle and proceed to formal validation. Pause ? What is formal validation ? Maybe I a have a wrong view (please validation lead correct me), but regulated environment means that there was a formal review by human being. This was my work hypothesis so bear a second with me.
- This is not part of BDD proper. During development we used Behave to automate some actions. These actions are organized in a modular fashion These actions are executed sequentially by behave We can capitalize on this to help the tester write the formal OQ test script
- Here it is in a concrete example. On the left a sample Behave file With human instruction embedded On the right what it outputs This document still needs to be formally reviewed, modified and validated. This is a draft. The modularity of the code helps the validator to write OQ test script more quickly. Dev automatic test should be as comprehensive as possible, but it’s up to the validator manually test what he/she wants. It’s only a draft that is produced by the automated system. Validation plan dictate how deep the manual testing should go.
- At this point business make a final test of everything Once everything gives the code to the operation team.
- At the end of this process, what value did that bring to the entities involved ? Since a common document from the start. → Minimal miscommunication. Business get exactly what is needs Validation can validate faster and more confidently Developer has a clear goal Bugs should are found during the unit testing phase, demo and refactoring phase
- Identify all actors Explaining how Test Script are written Explaining how Test Script are executed Short explanation of translator Explaining that after the translation the “normal” V-Model part can starts