Mastering BDD - Eran Kinsbruner Workshop Quest 2018

Mastering Behavior Driven Development
Workshop & Tutorial
© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Eran Kinsbruner
@ek121268, http://continuoustesting.blog

About Me
• Lead Technical Evangelist at Perfecto
• Blogger and Speaker
• http://mobiletestingblog.com
• https://www.infoworld.com/author/Eran-
Kinsbruner/
• 18+ Years in Development & Testing
• Author of The Digital Quality Handbook
To get your copy of the book, come by the Perfecto Booth

Agenda
Learning Objectives:
• What is BDD and its key advantages and disadvantages.
• What are the material differences in test automation between Mobile, Web and Responsive apps.
• How to build a BDD project that is based on a page object model that runs in parallel on various platforms.
Tutorial Outline:
• An Intro to BDD
• What is driving a shift to such software development practice
• Who are the stakeholders that make BDD process work
• What are the differences between BDD and formal software development life cycle
• BDD 101
• What are feature files, step definition (glue code), scenarios, background, DDT, Tagging, and a BDD project structure
(Gherkin)
• Working with mobile and web objects and page object model within BDD
• Available tools and frameworks in the market that supports BDD (e.g. Serenity BDD, Cucumber, Protractor, and more)
• BDD in the Cloud (quantum project reference)
• Configure a BDD project for responsive web and mobile apps to run in parallel in the cloud
• Leverage reporting to maximize quality productivity
• BDD and Visual Analysis
• Quantum & Applitools how to
• BDD Markdown Framework – Gauge.org

There’s a Paradigm Shift in Software Development
5/22/2018 4© 2015, Perfecto Mobile Ltd. All Rights Reserved.
+

Quality Is Everyone's ‘Problem’

DevOps == Continuous Everything
Credit: TestingWhiz

1
2
3 4
Approaches to DevOps Testing
Coverage
DailySprint
Smoke
Full
Coverage
before
Robustness
Robustness
Before
Coverage
Most of the market
• Sprint level execution
• Half-way to coverage
1. Create some scripts
2. Create more scripts
3. Even more scripts
4. (Try to) run them daily
1
2
3
4
1. Create few robust scripts
2. Make sure they run daily
3. Slowly add more scripts
1. Maintain stability & visibility
4. Add more scripts
• Too many failures
• Too much time spent on analysis

Test Coverage Principles

A Day in A Life – Covering Real Environments Should be Part of Your Coverage Planning
12/3/2016
System/Network Conditions
• Battery not full
• Location Services Active
• 4G Network/No Wi-Fi
• System level is important
Background Notifications
• Messages
• News app notifications
• Sport apps notifications
• Emails alerts
• Social (Facebook) alerts

The Coverage Cube

Mobile Platform Share – Quality is a Moment in Time
3 OS Families Required to Support
3 - 4 OS Families to Support

Some Real Life Example

Worldwide Distribution

Assuring UX Has Never Been More Important

Mobile & Web- Test Coverage Principles

Platform Specific Coverage Recommendations

Coverage Planning Tools

Reality Check: The Broken Promise of Automation
(Key Challenges)
Plethora of
Test
Frameworks
But
Parallel testing
across
platform is a
hard to achieve
goal
Flakiness of
tests slows
down release
trains (test
issues, lab
issues)
Automation
coverage is
limited due to
test framework
supported
features
Timely support for
new
features/platform
s is an issue
Execution
performance
/speed is
inconsistent
across
frameworksThere is no
support for
non-functional
testing within
the key
frameworks
Green build
isn’t a normal
- The level of
trust in
testing by dev
is low
Feedback
loop and
reporting -
hard to
manage
Still a lot of
manual
testing
No
guidelines
around
writing
“Good” test
code
Quantity
over
Quality is
still a valid
and !right
metric

TDD & BDD – Path to Advanced Agile Testing
Source: Use DevOps And Supply Chain Principles To
Automate Application Delivery Governance
Christopher Condo and Diego Lo Giudice (Forrester)
December 8, 2017

BDD Definition – By Wikipedia
• In software engineering, behavior-driven development (BDD) is a software development
process that emerged from test-driven development (TDD).[1][2][3][vague]Behavior-driven development
combines the general techniques and principles of TDD with ideas from domain-driven
design and object-oriented analysis and design to provide software development and management
teams with shared tools and a shared process to collaborate on software development.[1][4]
• BDD is largely facilitated through the use of a simple domain-specific language (DSL) using natural
language constructs (e.g., English-like sentences) that can express the behavior and the expected
outcomes. Test scripts have long been a popular application of DSLs with varying degrees of
sophistication. BDD is considered an effective technical practice especially when the "problem space"
of the business problem to solve is complex.

BDD method has crossed the chasm
Source: InfoQ

ATDD

BDD, The 3 Amigos and TDD
Definition
• Three amigos refers to the primary perspectives to examine an increment
of work before, during, and after development. Those perspectives are:
• Business - What problem are we trying to solve?
• Development - How might we build a solution to solve that problem?
• Testing - What about this, what could possibly happen?

Differences Between TDD, ATDD, and BDD
• TDD is rather a paradigm than a process. It describes the
cycle of writing a test first, and application code afterwards –
followed by an optional refactoring.
• TDD doesn’t make any statements about: Where do I begin
to develop? What exactly should I test? How should tests be
structured and named?
• BDD always start with the piece of functionality that’s most
important to your user.
• TDD and BDD have language differences, BDD tests are
written in an English-like language.
• BDD is customer-focused while ATDD leans towards the
developer-focused side of things like [Unit]TDD does. This
allows much easier collaboration with non-techie
stakeholders, than TDD.
• BDD gives a clearer understanding as to what the system
should do from the perspective of the developer and the
customer.
• BDD focuses on the behavioral aspect of the system unlike
TDD that focuses on the implementation aspect of the
system.
• ATDD focuses on capturing requirements in acceptance
tests and uses them to drive the development. (Does the
system do what it is required to do?)
• TDD tools and techniques are usually much more techie in
nature, requiring that you become familiar with the detailed
object model (or in fact create the object model in the
process, if doing true test-first canonical TDD).
• TDD allows a good and robust design, still, your tests can be
very far away of the users requirements. BDD is a way to
ensure consistency between requirements and the
developer tests.
• TDD allows a good and robust design, still, your tests can be
very far away of the users requirements. BDD is a way to
ensure consistency between requirements and the
developer tests.

BDD Traditional Workflow (John Ferguson Smart)
Source: John Ferguson Smart Post

Advantages & Disadvantages of Using BDD
Expected Benefits
• Builds a shared understanding about the intent of an increment
of work.
• Easy ramp up from a skillset perspective
• Multi language & IDE support (Java, Ruby)
• Various mobile and web supporting frameworks
• Identifies misunderstandings and confusion early and allows
learning to happen sooner in the delivery of an increment of
work.
• Provides a reasonable guard rail for the number of people who
should be involved in discussions about any given increment of
work.

Advantages & Disadvantages of Using BDD
Disadvantages & Common Pitfalls
• While syncing various personas, Doesn’t employ traditional SW
development processes
• Dependent on available coding skillset within the team (step definition
development)
• Maintenance and cross platform robust test coding is often a challenge
• Limiting three amigo discussions to only three people. If there are other
stakeholders who are relevant to a particular increment of work, include
them in the discussion.
• Three amigo discussions become regularly scheduled meetings and are
treated as another ceremony for the team to do instead of as a handy
guide of what perspectives should be included in a discussion about a
particular increment of work.

Here is a sample Gherkin document:
Feature: Refund item
Scenario: Jeff returns a faulty microwave
Given Jeff has bought a microwave for $100 And he has a receipt
When he returns the microwave
Then Jeff should be refunded $100
In Gherkin, each line that isn't blank has to start with a Gherkin keyword, followed by any text you like.
The main keywords are:
• Feature
• Scenario
• Given, When, Then, And, But (Steps)
• Background
• Scenario Outline
• Examples
There are a few extra keywords as well:
• """ (Doc Strings)
• | (Data Tables)
• @ (Tags)
• # (Comments)
Gherkin Syntax
https://cucumber.io/docs/reference

Gherkin Syntax – Features, Scenario’s & Steps
• Features describe in an high level a single feature of the system
e.g. “Login to the app” and to group to it set of related scenario’s
• Scenario’s are specific business rules related to the feature
• Describe an initial context
• Describe an event
• Describe an expected outcome
• Steps are usually defined using Given, When and Then and
construct the body of the scenario.
• There are some recommended practices around the # of Given and When
within a single scenario. To many == split your scenario
• Then usually would describe the expected outcome followed by an Assertion
or validation (Object based or Visual based).

Diving Into Gherkin – Use of Background’s
Instead of repeating Given steps across multiple scenario’s,
Include the repetitive Given/s in the Background section

Diving Into Gherkin – Use of Runners (Junit/TestNG)

Diving Into Gherkin – Scenario’s vs. Scenario Outline
A container for a table

Doc Strings and Data Tables
Doc strings (“””) are used to pass large string of text to a step definition
Data tables are used to pass a list of values to a step definition

Tagging and RTDD
Use of tags (@) above Features, Scenarios allow classifying and
grouping what’s above the tag. E.g. – a tag placed above a
Feature means that all scenarios below that feature get the tag.

Cucumber Support Multiple Languages (Step Definition)

POM - Benefits
One Place
Change
Model Pages in
Code
Reduce Code
Duplication
Simulate User
Actions
Snippets of
Page

POM - Example
Scenario: Sign in and sign out
When I open automationpractice website
And I sign in
Then I sign out

Cucumber.IO

BDD Test Management
Teams can chose commercial tools (TestRail e.g.) or embed BDD
into their Build/CI tool chain
• Build Tools
• Gradle, Maven
• CI Tools
• Jenkins, Team City, Bamboo

BDD for iOS – XCFit & Cucumberish
XCFit is a pure Swift implementation to support BDD inside XCode projects. It also gives
XCode templates to get started with BDD, Page Object Pattern and Protocol Oriented
pattern. XCFit has some pre-defined steps inside Swift frameworks but the quick benefit
comes from XCode templates. More information on Protocol Oriented BDD here. Again
major drawback is we cannot write or execute .feature files and lacks many core
cucumber features like tags, filtering features.
Cucumberish is a test framework that can be used to write features in Gherkin format and
can be executed from XCode. Cucumberish is written Objective-C and uses Objective-C
Gherkin parser from Cucumber repository. Its great to be able to execute Gherkin feature files
from XCode but Cucumberish has some bugs and needs workarounds for most cases. Also
it’s a bit tricky to introduce Objective-C dependency in the fully Swift Project. Cucumberish
can be imported to UI Test target using CocoaPods or Carthage.

The XCFit predefined Steps are grouped by
• Application Launcher Steps
• User Actions
• Waiter Actions
• Assertions
Follow to learn more:
https://github.com/Shashikant86

Architecture
Perfecto Report
Object Spy
Repository
file
Live dashboard
Cucumber BDD test
Test Java code
testNG
Perfecto utils
Pre define steps
User
advance
code
Script creation Execute Report
Access to devices
Remote debugging
Real devices and browsers
Manual services

BDD – Cucumber
Test -Feature files
Pre define steps
by Perfecto
Logical steps
created by
customers

BDD – Steps definition
Test -Feature files
Pre define steps
by Perfecto
Logical steps
created by
customers

BDD – Object Repository
objectID = locatorID = locator

BDD – components
Test -Feature files
Pre define steps
by Perfecto
(Low Level)
Logical steps
created by
customers
• Standard BDD – cucumber
• Auto complete using standard Cucumber
plugin
Requirements from user:
• Identify elements and add to
repository
• Build flow by using low level and
logical functions

code– components
Test -Feature files
Pre define steps
by Perfecto
Logical steps
created by
customers
• Customer code
• Standard java code
• Focused only on the test actions and
validations
• QAF manage
• Driver init
• getDriver() – provide the exiting driver
• Object repository
• Predefine functions (sendKey, assertText,…)
• Integration with DigitalZoom™ execution and
steps (no code need to be added)
• Perfecto
• Utils code with specific commands like (WT,
swipe)

Project structure
BDD
Logical commands
Java tests for java coder testers
Perfecto extensions
TestNG execution
Data repository
Cloud / app configuration file

Project config file – driver
Appium
Selenium

Execution – Threads and
devices
Each test = new thread with the requested device
Thread 1
iOS
Thread 2
Android

Quantum
Area Components Quantum
Management
Reporting
Execution Manager
Test
Development
Test Authoring
(IDE+Language)
Object Repository
Object Locator/Spy
Appium/Selenium
✓OSS
=
✓OSS
✓OSS
http://projectQuantum.io

Sample Project Demo (RWD)
Sample Visual Testing Demo

BDD Tools Landscape
http://fitnesse.org/FrontPage
https://cucumber.io/ http://jbehave.org/
http://www.thucydides.info/#/
http://specflow.org/
https://mochajs.org/
https://gauge.org/

Gauge Intro
• Installation
• Install Chocolatey (CMD.exe, Admin mode)
• “@"%SystemRoot%System32WindowsPowerShellv1.0powershell.exe" -NoProfile -InputFormat
None -ExecutionPolicy Bypass -Command "iex ((New-Object
System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET
"PATH=%PATH%;%ALLUSERSPROFILE%chocolateybin”
• From windows CMD.exe run
• “choco install gauge”
• Choose the language of choice (Java, C#, Ruby)
• "gauge init java” or “gauge init java_maven_selenium”
• Write your Specs & Run them
• Markdown Syntax that includes: Specifications, Scenarios, Steps and Tags
• “gauge run specs”

Gauge Intro – Continue

gauge run specs/Example.spec:[scenario in line XX]

Gauge Intro – Connect to The Cloud
• Configure in IntelliJ Env. Variables
• PERFECTO_CLOUD
• PERFECTO_USERNAME
• PERFECTO_SECURITY_TOKEN
• Properly define the target platform under test in the
DriverFactory.java
• Implement your SPEC test and its required methods (Selenium)
• Execute!

JS Acceptance Testing with Codecept.JS
Source: Codecept.JS Git

Mocha.JS Intro

Jbehave Intro & Composite Steps
Combine 2 step definitions with parameters
Using @Composite annotation

Suggested BDD Tools Selection Criteria

Tool Selection Case Study (Model) (USAA)
• Define needed capabilities
Selection Criteria
End to End Testing
BDD/ATDD Friendly
Tool Documentation
Visual Navigation Testing

USAA Tool Selection
• Identify importance (weight capabilities)
Weight Key
1 – Low importance
3 – Medium importance
5 – High importance
Weight Selection Criteria
5 (High importance) End to End Testing
3 (Medium importance) BDD/ATDD Friendly
5 (High importance) Tool Documentation
1 (Low importance) Visual Navigation Testing

USAA Tool Selection
• Define scoring key
Scoring Key
0 – Did not meet expectations
2 – Met expectations
3 – Exceeded expectations
Weight Selection Criteria Tool X Tool Y Tool Z
5 (High importance) End to End Testing 3 3 3
3 (Medium importance) BDD/ATDD Friendly 3 2 3
5 (High importance) Tool Documentation 0 2 2
1 (Low importance) Visual Navigation Testing 3 3 2

USAA Tool Selection
• Define scoring key
Weight Selection Criteria Tool X (weighted) Tool Y (weighted) Tool Z (weighted)
5 (High importance) End to End Testing 3 5 x 3 = 15 3 5 x 3 = 15 3 5 x 3 = 15
3 (Medium importance) BDD/ATDD Friendly 3 3 x 3 = 9 2 3 x 2 = 6 3 3 x 3 = 9
5 (High importance) Tool Documentation 0 5 x 0 = 0 2 5 x 2 = 10 2 5 x 2 = 10
1 (Low importance) Visual Navigation Testing 3 1 x 3 = 3 3 1 x3 = 3 2 1 x 2 = 2
Total 27 34 36
Scoring Key
0 – Did not meet expectations
2 – Met expectations
3 – Exceeded expectations

Open Discussion
• Challenges with BDD
• R&D dependency?
• Reports?
• Integrations to tool suites?
• Requirements and backlog alignment?
• Cross-team efficiency and sync?
• Is BDD a transitional phase in your organization or end-goal?
• Metrics used for BDD success?
• What is the future of BDD?
• Other?

Thank You
Eran Kinsbruner
@ek121268
http://continuoustesting.blog

Mastering BDD - Eran Kinsbruner Workshop Quest 2018

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Mastering BDD - Eran Kinsbruner Workshop Quest 2018

Similar to Mastering BDD - Eran Kinsbruner Workshop Quest 2018 (20)

More from Perfecto Mobile

More from Perfecto Mobile (20)

Recently uploaded

Recently uploaded (20)

Mastering BDD - Eran Kinsbruner Workshop Quest 2018