Property Based BDD Examples (ETSI UCAAT 2016, Budapest)

•

2 likes•855 views

BDD (Behavior-driven development) is a software development process focusing on active collaboration, which illustrates and automates the requirements using key examples of the problem domain. In BDD the formalized examples use a natural language-based DSL driven by the Given/When/Then keywords. At the same time, property-based testing (PBT) uses abstract (mathematical) formulas to declare expectations for the output values given some constraints on the input. The PBT tools try to disproof that the application fulfills these requirements by taking samples from the valid input value space. The experience shows that for understanding and properly implementing the requirements, the team has to understand the requirements as a set of abstract rules, where collecting key examples can help a lot. BDD is strong at managing and automating these key examples and PBT is strong at defining and automating the rules, so the question should arise: can these two methods somehow support each-other? Can we increase the value of BDD by formalizing and testing the rules? Can we increase the value of PBT by a better communication of the rules and constraints we have? This session shows an experiment of combining BDD and PBT. It provides examples of how the specification could look like in this combined world and what it could be used for. And there is a working, open-source prototype too!

Software

Budapest, 26-28 October 2016
PROPERTY BASED BDD EXAMPLES
Presented by Gaspar Nagy
© All rights reserved
gaspar@specsolutuions.eu
@gasparnagy, http://gasparnagy.com

BDD PBT
• Behavior Driven
Development
• ~Specification by
Example (SbE)
• ~Acceptance Test Driven
Development (ATDD)
• ~Keyword Driven Testing
2 © All rights reserved
• Property Based Testing
• Property Testing
• ~Random Testing
• ~Model-based Testing

Scenario: Correct easy answer scores 10
Given I register a team
When I submit a correct easy answer
Then my score should be 10
Collaboration
Automation with
Cucumber/SpecFlow
What is BDD?

This is an example!
5 © All rights reserved
Scenario: Correct easy answer scores 10
Given I register a team
When I submit a correct easy answer
Then my score should be 10

How many examples you need?
6 © All rights reserved
Scenario: Add two numbers
Given I have entered 1 into the calculator
And I have entered 2 into the calculator
When I press add
Then the result should be 3 on the screen
Scenario: Add two numbers
Given I have entered 5 into the calculator
And I have entered -7 into the calculator
When I press add
Then the result should be -2 on the screen
Scenario: Add two numbers
Given I have entered 2 into the calculator
And I have entered 0 into the calculator
When I press add
Then the result should be 2 on the screen

BDD is for…
understanding & validating
business requirements
through illustrative examples
8 © All rights reserved

PBT is for…
verifying
implementation
through checking statements about the
output for many different possible inputs
9 © All rights reserved
source: http://blog.jessitron.com/2013/04/property-based-testing-what-is-it.html
the “properties”

For example for addition…
• Commutative property: a + b = b + a
• Associative property: (a + b) + c = a + (b + c)
• Identity property: a + 0 = a
• Distributive property: a * (b + c) = a*b + a*c
We would like to verify these for ~ALL input
combinations!
10 © All rights reserved

There is a tool for doing this!
• QuickCheck (Haskell) is the canonical framework, but
there are many different ports of it to other
programming languages
• QuickCheck for Java
• PhpQuickCheck for PHP
• ScalaCheck for Scala
• FsCheck for .NET (F#, C#)
• … (see more at https://en.wikipedia.org/wiki/QuickCheck)
11 © All rights reserved

When the implementation is wrong…
13 © All rights reserved

BDD PBT
• 1 + 2 = 3
• 5 + -7 = -2
• 2 + 0 = 2
14 © All rights reserved
• Commutative
• Associative
• Identity
• Distributive

Examples in SpecFlow
15 © All rights reserved

Identity property BDD style…
16 © All rights reserved

Defining constraints and expectations…
17 © All rights reserved

More real life examples…
18 © All rights reserved

Summary
• BDD turns examples into automated tests
• PBT automates rules with many different input
• The power of this two can be combined to achieve an
executable specification
• The BDD and the PBT tools can work together for this
• See http://github.com/gasparnagy/SpecFlow.FsCheck
19 © All rights reserved

QUESTIONS?
gaspar@specsolutuions.eu
@gasparnagy
http://gasparnagy.com
© All rights reserved
Special thanks to
• Ciaran McNulty
• Konstantin Kudryashov

Viewers also liked

Abstract If you have ever done or tried to do behavior driven development, you know that finding the right scenarios (ie. tests) and writing the Gherkin feature files with the Given/When/Then steps is not so easy. Many teams leave this task entirely for the developers. Others try to use the sprint planning meeting to discuss the exact phrasing. None of these are really perfect. Regarding my experience, for having good scenarios, you have to have a good sprint planning. In this session I am going to talk about what makes a sprint planning good and how you can facilitate your sprint planning meeting in a way that it helps finding the right examples. You will hear about Specification by Examples, the "Three Amigos", example mapping, really good acceptance criteria and also about team motivation, but more importantly you will hear about how to use them efficiently in practice.

Given/When/Then-ready sprint planning with Example Mapping (Agilia Budapest 2...

Gáspár Nagy

BDD is a method that focuses on defining the requirements based on the expected behavior; DDD is a method that uses a ubiquitous language all across the solution; UX helps design user interactions based on user behavior… and Selenium WebDriver (or similar) is a tool that is mostly used to ignore all these and automate the app using clicks on HTML elements… In this session you will explore options how to improve testing efficiency by looking at UI automation from a different angle by analyzing and understanding the UI concepts used in the application, matching them to domain model elements and build an automation solution that focuses on these.

Behavior Driven Web UI Automation with Selenium and Cucumber/SpecFlow (BDDx L...

Gáspár Nagy

Nov 26 Chinese and English Infosession

Alice Lam

With modern software development moving at a rapid pace, automation engineers may find it difficult to keep up with frequent changes. Incorporating parts of the testing process into everyone's workflow, this creates the potential for automated testing to mirror software delivery. Description: With modern software development moving at a rapid pace, automation engineers may find it difficult to keep up with frequent changes. But if parts of the testing process are incorporated into everyone's workflow, this creates the potential for automated testing to mirror software delivery. My goal is to make software testing more accessible to entire product and engineering teams by providing familiar tools that allow individuals to easily contribute to the automation process. For this talk, I will explore how we can meet developers halfway by using the javascript-based testing framework WebdriverIO, demonstrate how product managers can create living documentation and test cases using Cucumber, and talk about sharing page objects and testing libraries across multiple services using NPM. Code Examples: https://github.com/dcypherthis/gtr-test-cucumber https://github.com/dcypherthis/gtr-test-mocha-bdd https://github.com/dcypherthis/gtr-test-xray

Jumpstarting Testing In Your Organization with Selenium, Cucumber, & WebdriverIO

Josh Cypher

The Power of Visuals

Ádám Nagy

Do you remember your childhood when your mom or an older story teller pronounced the most interesting words in the world? “Once upon a time…” Let me be now your story teller, listen to my story about a legacy app, a painful problem, the search for the solution and the lucky outcome. No, no, this is not the story about a new green field project where everything is so clear. However, even in this story there is a master builder, who shows the developers how to scaffold the legacy app with BDD scenarios using SpecFlow or Cucumber and everything will be all right in the end. Let me tell you about different strategies, collaboration and testing patterns that can be followed in such a renovation-like situation. So, are you ready?... “Once upon a time there was a company where…” To be continued at my session

Scaffolding a legacy app with BDD scenarios using SpecFlow/Cucumber (HUSTEF 2...

Gáspár Nagy

Given/When/Then-ready sprint planning (Agile Tour Vienna 2015)

Gáspár Nagy

Folyamatos integráció és kódépítés (ALM Day Budapest, 24/11/2015, Hungarian)

Gáspár Nagy

Abstract: Behavior driven development (BDD) is an agile method that (among others) puts the ideas of agile testing into practice. BDD emphasizes collaboration between testers, developers and the business stakeholders and help preventing bugs by introducing testers and testing concept into the specification phase. This is all fine when you start with a new, green field project. But what about those old legacy applications that have no proper automated test coverage and even if there are automated tests no one is dare to touch them? Is it possible to introduce BDD for these projects? What are the different strategies, collaboration and testing patterns that can be followed? This session gives a brief overview about what BDD is today and addresses these questions. The code examples use .NET and SpecFlow, but can be easily adapted to any other platform too.

Introducing BDD to Legacy Applications with SpecFlow/Cucumber (Agilia Confere...

Gáspár Nagy

Scaffolding a legacy app with BDD scenarios using SpecFlow/Cucumber (BDD Lond...

Gáspár Nagy

Viewers also liked (10)

Given/When/Then-ready sprint planning with Example Mapping (Agilia Budapest 2...

Behavior Driven Web UI Automation with Selenium and Cucumber/SpecFlow (BDDx L...

Nov 26 Chinese and English Infosession

Jumpstarting Testing In Your Organization with Selenium, Cucumber, & WebdriverIO

The Power of Visuals

Scaffolding a legacy app with BDD scenarios using SpecFlow/Cucumber (HUSTEF 2...

Given/When/Then-ready sprint planning (Agile Tour Vienna 2015)

Folyamatos integráció és kódépítés (ALM Day Budapest, 24/11/2015, Hungarian)

Introducing BDD to Legacy Applications with SpecFlow/Cucumber (Agilia Confere...

Scaffolding a legacy app with BDD scenarios using SpecFlow/Cucumber (BDD Lond...

Similar to Property Based BDD Examples (ETSI UCAAT 2016, Budapest)

OSGi Enterprise R6 specs are out! - David Bosschaert & Carsten Ziegeler

mfrancis

How to build Sdk? Best practices

Vitali Pekelis

Metasepi team meeting #13: NetBSD driver using Haskell

Kiwamu Okabe

We are inviting you to join our free webinar to see a case study of a real project developed with Behaviour Driven Development (BDD). Our product, "SpecSync for Azure DevOps" has been developed with BDD. The functionalities are specified as BDD scenarios that can be verified as automated tests. But BDD scenarios alone would not be enough for us to meet our quality expectations, so there are other tests and quality considerations as well -- all these fit into our agile testing and traceability strategy. In this webinar Gaspar Nagy gives you a walkthrough of the quality considerations and tests of the product by focusing on the following questions: • What kind of tests are needed? How to decide what is the right way to specify and test a concrete function? • What kind of feedback can we get from the different tests and how? Do they form a test automation pyramid? • How are the requirements expressed in BDD scenarios and how are they connected to the other development artifacts? • BDD, tests automation and continuous integration tips & tricks The webinar focuses on the general testing and automation challenges, so people with or without coding skills are both welcome. At the end of the webinar there will be a Q&A, so you can also ask your own questions. The webinar recording will be made available for everyone who registered.

BDD Scenarios in a Testing & Traceability Strategy (Webinar 19/02/2021)

Gáspár Nagy

Behavior-Driven Development and Acceptance Testing are heavily intertwined and in many aspects are one and the same. Both focus on starting at the outer layers of your application by concentrating on what matter to users; behavior. In this session/workshop we'll talk about how testing can be used both for specifying your application yet to be develop expected behavior and as accurate, running documentation that can be used to validate your stakeholder's acceptance criteria. We'll talk about the different types of testing and do a few hands-on exercises to flesh out a Rails application with RSpec and Capybara.

Rspec and Capybara Intro Tutorial at RailsConf 2013

Brian Sam-Bodden

Bloomberg’s Machine Learning/Text Analysis team has developed many machine learning libraries for fast real-time sentiment analysis of incoming news stories. These models were developed using smaller training sets, implemented in C++ for minimal latency, and are currently running in production. To facilitate backtesting our production models across our full data set, we needed to be able to parallelize our workloads, while using the actual production code. We also wanted to integrate the C++ code with PySpark and use it to run our models. In this talk, I will discuss some of the challenges we faced, decisions we made, and other options when dealing with integrating existing C++ code into a Spark system. The techniques we developed have been used successfully by our team multiple times and I am sure others will benefit from the gotchas that we were able to identify.

Integrating Existing C++ Libraries into PySpark with Esther Kundin

Databricks

Telegraf is an agent for collecting, processing, aggregating, and writing metrics. With over 200 plugins, Telegraf can fetch metrics from a variety of sources, allowing you to build aggregations and write those metrics to InfluxDB, Prometheus, Kafka, and more. In this talk, we will take a look at some of the lesser known, but awesome, plugins that are often overlooked; as well as how to use Telegraf for monitoring of Cloud Native systems.

The Telegraf Toolbelt: It Can Do That, Really? | David McKay | InfluxData

InfluxData

The Telegraf Toolbelt | David McKay | InfluxData

InfluxData

Get your teeth into Plack

Workhorse Computing

OSGi Community Event 2016 Presentation by David Bosschaert (Adobe) & Carsten Ziegeler (Adobe) The OSGi expert groups are working on the next big release. Learn in this session about the various new specification efforts going on and how they will make your developer life easier. The new specifications range from configuration handling, object conversion, JAX-RS, distributed eventing, to cloud and IoT.

New and cool in OSGi R7 - David Bosschaert & Carsten Ziegeler

mfrancis

The "Holy Grail" of Dev/Ops

Erik Osterman

Modern Web Development with Perl

Dave Cross

High Productivity Web Development Workflow

Vũ Nguyễn

High productivity web development workflow - JavaScript Meetup Saigon 2014

Oliver N

The HPC community is embracing the advantages of mixed-language development environments, presenting challenges for debugging and testing when application execution and data flow cross languages. How can we take advantage of the unique features offered by different languages while minimizing the impact on bug reproduction, root-cause analysis, and solution? This presentation walks through the current mixed-language HPC landscape to describe the problems with testing these types of applications and best-practice solutions using TotalView for HPC. You will learn how these architectures make it easy to “steer” computation between modules of different languages, to accelerate prototyping and development, and how advanced testing techniques provide visibility into the call stack and data for efficient debugging.

Approaches to debugging mixed-language HPC apps

Rogue Wave Software

_Given_ the prominence of API-first development _when_ developing web applications, projects _then_ need to be tested beyond unit and functional tests. Authoring tests using the Gherkin with Behat provides a way to achieve BDD with APIs. We'll dive right into writing Gherkin, how to create your own test scenarios in Gherkin, and how to hook those tests up with custom PHP test methods. See tests in action with live coding and live demos, including writing new custom tests. Leave excited to produce fully-tested APIs!

BDD API Tests with Gherkin and Behat

Mark Niebergall

Not familiar with Neo4j? This workshop provides an introduction and shares how to build a low-code interactive dashboard with tables, graphs, charts, and more. Neodash: https://neo4j.com/labs/neodash/ RecipeNLG Dataset: https://www.kaggle.com/code/paultimothymooney/explore-recipe-nlg-dataset Neo4j Aura: https://dev.neo4j.com/discover-aura Check out https://community.neo4j.com/ for questions and discussions around Neo4j

ROAD TO NODES - Intro to Neo4j + NeoDash.pdf

Neo4j

Application Programming Interfaces allow developers to leverage existing program code in an effort to build additional functionality, automate processes or present existing functionality in a different format. APIs exist for many of Hyperion products; however, clear examples of how to use them are not always easy to find nor are applications for the API readily available. Proper application of the APIs for HFM, Workspace, FDM, Smartview and Shared Services can simplify the daily routines of end users and administrators. This session will provide a high level overview of how each of the APIs work. Additionally, real-world examples for each API will be provided. Fully working code will be available for download from the ODTUG 12 site which attendees can use in their own environments.

Hyperion EPM APIs - Added value from HFM, Workspace, FDM, Smartview, and Shar...

Charles Beyer

BDD Scenarios in a Testing Strategy

Gáspár Nagy

Configuration as Code in Jenkins. What's new? Nov 2016

Oleg Nenashev

Similar to Property Based BDD Examples (ETSI UCAAT 2016, Budapest) (20)

OSGi Enterprise R6 specs are out! - David Bosschaert & Carsten Ziegeler

How to build Sdk? Best practices

Metasepi team meeting #13: NetBSD driver using Haskell

BDD Scenarios in a Testing & Traceability Strategy (Webinar 19/02/2021)

Rspec and Capybara Intro Tutorial at RailsConf 2013

Integrating Existing C++ Libraries into PySpark with Esther Kundin

The Telegraf Toolbelt: It Can Do That, Really? | David McKay | InfluxData

The Telegraf Toolbelt | David McKay | InfluxData

Get your teeth into Plack

New and cool in OSGi R7 - David Bosschaert & Carsten Ziegeler

The "Holy Grail" of Dev/Ops

Modern Web Development with Perl

High Productivity Web Development Workflow

High productivity web development workflow - JavaScript Meetup Saigon 2014

Approaches to debugging mixed-language HPC apps

BDD API Tests with Gherkin and Behat

ROAD TO NODES - Intro to Neo4j + NeoDash.pdf

Hyperion EPM APIs - Added value from HFM, Workspace, FDM, Smartview, and Shar...

BDD Scenarios in a Testing Strategy

Configuration as Code in Jenkins. What's new? Nov 2016

More from Gáspár Nagy

Ramp up your testing solution, ExpoQA 2023

Gáspár Nagy

This talk is about the experience I have gathered on how to handle and avoid technical debt. I affiliated three projects in the last years that had to deal with technical debt. I could see how different methods can help (or how they cannot). In my context, we needed a combination of BDD, DevOps and a better focus on sustainability. You will hear about ideas how these techniques can work together that you can also consider for your project.

Fighting against technical debt (CukenFest 2020)

Gáspár Nagy

A teszt automatizálási piramist Mike Cohn vezette több évvel ezelőtt. Bár sokan hallottak már róla, mégis kevesen vannak akiknek az eredeti formájában alkalmazni is tudják. Ennek egyik oka az lehet, hogy a piramis segít leképezni a teszteket az őket megvalósító technológiára (unit, integrációs, UI), de abban, hogy mire szolgálnak ezek a tesztek (tesztelési célok) nem adnak segítséget. A tesztelési jéghegy (testing iceberg) ezt az aspektust próbálja meg megjeleníteni a piramisban. Az előadásban a tesztelési jéghegy ötletét és részleteit vizsgáljuk meg. Megnézzük, hogy a "víz alatti" tesztek hogy tudják támogatni a jéghegy csúcsán álló, az üzleti követelményekre fokuszáló teszteket. Azt is megnézzük, hogy hogyan lehet a teszteket a piramis szintjei között mozgatni és végül, egy-két ötletet adnunk arra, hogy hogyan lehet a teszt automatizálási piramis hasznunkra a saját projektjeinkben.

Süllyedünk! Ütközés a tesztelési jégheggyel (Teszt & Tea Meeup Budapest, 2018...

Gáspár Nagy

Abstract: This session is about my experience with BDD in a Continuous Delivery model. Many teams use Behavior Driven Development (BDD) for automated UI testing. Although UI testing might be an important element of your verification pipeline, it is slow, brittle and costly. But in the era of modern software development we would like to have fast feedback and quick reactions. A continuous delivery / deployment (CD) model can support this well. But what does it look like with BDD then? Can BDD help implementing CD? This session is an attempt to define the role of BDD in CD. What works, what does not? Continuous delivery should mean continuous quality for the delivery team. What kind of quality criteria do we want to target with CD and how is it supported by BDD?

Continuous Behavior - BDD in Continuous Delivery (CoDers Who Test, Gothenburg...

Gáspár Nagy

We are sinking: Hitting the testing iceberg (CukenFest London, 2018)

Gáspár Nagy

BDD is a method that focuses on defining the requirements based on the expected behavior; DDD is a method that uses a ubiquitous language all across the solution; UX helps design user interactions based on user behavior… and Selenium WebDriver (or similar) is a tool that is mostly used to ignore all these and automate the app using clicks on HTML elements… In this session, we are exploring options how to improve testing efficiency by looking at UI automation from a different angle: by analyzing and understanding the UI concepts used in the application, matching them to domain model elements and build an automation solution that focuses on these.

Behavior Driven UI Automation (Agile Testing Days 2017, Potsdam)

Gáspár Nagy

Testing is Difficult (Agile in the City Bristol 2017, Lightening talk)

Gáspár Nagy

Do you remember your childhood when your mom or an older story teller pronounced the most interesting words in the world? “Once upon a time…” Let me be now your story teller, listen to my story about a legacy app, a painful problem, the search for the solution and the lucky outcome. No, no, this is not the story about a new green field project where everything is so clear. However, even in this story there is a master builder, who shows the developers how to scaffold the legacy app with BDD scenarios using SpecFlow or Cucumber and everything will be all right in the end. Let me tell you about different strategies, collaboration and testing patterns that can be followed in such a renovation-like situation. So, are you ready...? “Once upon a time there was a company where…” To be continued at my session...

Scaffolding a legacy app with BDD scenario (Agile in the City Bristol 2017)

Gáspár Nagy

Behavior Driven Web UI Automation with Selenium and Cucumber/SpecFlow (Qualit...

Gáspár Nagy

A tesztelés szerepe folyamatos kihelyezést használó projektekben (Microsoft, ...

Gáspár Nagy

More from Gáspár Nagy (10)