SlideShare a Scribd company logo
How to get Automated Testing
"Done"
Hans Buwalda
LogiGear
www.LogiGear.com
hans@logigear.com
@hansbuwalda
www.TestArchitect.com
www.HappyTester.com
©2016 LogiGear Corporationwww.LogiGear.com
Who is your speaker
• Software testing company, around since 1994
– Foster City, California
– consultancy, training
– test development and automation services
– "test integrated" development services
• Products:
– TestArchitect™, TestArchitect for Visual Studio™
– integrating test development with test management and automation
– based on modularized keyword-driven testing
• LogiGear Magazine:
– themed issues, non-commercial
• Dutch guy, in California with LogiGear since 2001
• Background in math, computer science, management
• Since 1994 focusing on testing and automation
– keywords, agile testing, big testing
Hans Buwalda
LogiGear Corporation
www.LogiGear.com
hans@logigear.com
@hansbuwalda
www.TestArchitect.com
www.HappyTester.com
©2016 LogiGear Corporationwww.LogiGear.com
Relation to code Quality / depth Automation Scalability
Unit Testing
Close relationship
with the code
Singular test
scope, but deep
into the code
Fully automated
by nature
Scalable, grows
with the code,
easy to repeat
Functional
Testing
Usually does not
have a one-on-one
relation with code
Quality and scope
depends on test
design
In particular UI
based automation
can be a challenge
Often a bottle-
neck in scalability
Exploratory
Testing
Human driven, not
seeking a relation
with code
Usually deep and
thorough, good at
finding problems
May or may not
be automated
afterwards
Not meant to be
repeatable. Rather
do a new session
Some test kinds and their scalability (simplified)
©2016 LogiGear Corporationwww.LogiGear.com
• High degree of automation coverage
– under Agile and DevOps automation went from nice-to-have to
must-have
• Scrum teams are well suited to, collectively, own testing
and automation
• QA members (test development and automation) need to
stay in the same sprint with the rest of the team
– developers and product owners are a great help for testing, and
vice versa
– otherwise the task gets harder, and will delay even more
• However, it is not always practical to have fully
automated tests as part of "done" for a sprint . . .
Success factors in Agile (Scrum)
©2016 LogiGear Corporationwww.LogiGear.com
• More test cases needed than is time for
• Shooting a moving target, changes are common
• Lacking technology
• Since QA benefits greatly from close cooperation with PO's and Devs, matters might
get worse once out of sync
• Not always practical to let development wait for QA to finish (true meaning of "done")
QA role may have a hard time keeping
up with the rest of the team
Sprint Sprint Sprint
PO
Dev
QA
PO
Dev
QA
PO
Dev
QA
R
O
L
E
S
PO
Dev
©2016 LogiGear Corporationwww.LogiGear.com
• Design tests to be automation-friendly
– successful automation is not as much a technical challenge as it is a
test design challenge
• Make tests easy to understand for all involved
– so, not too technical please
– note: "easy to understand" does not necessarily mean "easy"
• Invest in testability of the system under test
– one of the main benefits of having a team
– works two ways: good tests facilitate "test driven" development
• Organize the activities well
– I like to consider tests, and their automation, as "products"
– let the testing and automation activities fit the sprint
– support the teams well with infrastructure, technology, services, etc
Approaches
©2016 LogiGear Corporationwww.LogiGear.com
Easy to understand ? ? ?
WebElement element = driver.findElement(By.name(name));
element.sendKeys("mystery magic");
element.submit();
(new WebDriverWait(driver, 10)).until(
new ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver d) {
return d.getTitle()
.toLowerCase().startsWith("mystery");
}
}
);
System.out.println(driver.getTitle());
©2016 LogiGear Corporationwww.LogiGear.com
More easy to understand: actions
4 actions, each with
an action keyword
and arguments
read from top
to bottom
fragment from a test with actions
acc nr first last
open account 123123 John Doe
acc nr amount
deposit 123123 10.11
deposit 123123 20.22
acc nr expected
check balance 123123 30.33
• The test developer creates tests using actions with keywords and arguments
in a easy to understand spreadsheet format
• Checks are, as much as possible, explicit (specified expected values)
• The automation task focuses on automating the keywords, each keyword is
automated only once
• In addition to keywords you can also use "given-then-when" scenarios (BDD)
for good readability
©2016 LogiGear Corporationwww.LogiGear.com
Supporting BDD with actions
Given a customer previously bought a black sweater from me
And I currently have three black sweaters left in stock
When he returns the sweater for a refund
Then I should have four black sweaters in stock
customer buys, article, color
a customer previously bought a {color} {article} from me
set stock, article, color, amount
I currently have {amount} {color} {article} left in stock
my stock of {color} {sweater} is {amount}
return article, article, color
he returns the {color} {article} for a refund
check stock, article, color
I should have four {color} {article} in stock
article color
customer buys sweater black
article color amount
set stock sweater black 3
article color
return article sweater black
article color amount
check stock sweater black 4
email me for this tool:
hans@logigear.com
©2016 LogiGear Corporationwww.LogiGear.com
• Modularized and keyword-driven method for automated
testing
• Particularly suited for large and complex projects
• High degree of agility, even when used in traditional
projects
• High level of automation, 95% or more is the norm
• Puts the focus on test design, less on technology
"Action Based Testing"
High Level Test Design - Test Development Plan
Objectives
Test Module 1
Test Cases
Test Module 2 Test Module N
Actions
. . .
AUTOMATION
Objectives Objectives
interaction test business test
define the "chapters"
create the "chapters"
create the "words"
make the words work
Test Cases Test Cases
window control value
enter log in user name jdoe
enter log in password car guy
window control property expected
check property log in ok button enabled true
user password
log in jdoe car guy
first last brand model
enter rental Mary Renter Ford Escape
last total
check bill Renter 140.42
Action Based Testing
Test Module Sections
Test Module
Objectives
Initial - setup
Test cases
Final - cleanup
©2016 LogiGear Corporationwww.LogiGear.com
Variables and expressions with keywords
• This test does not need an absolute number for the
available cars, just wants to see if a stock is updated
• As a convention we denote an assignment with ">>"
• The "#" indicates an expression
TEST CASE TC 02 Rent some more cars
car available
get quantity Chevy Volt >> volt
first name last name car
rent car John Green Chevy Volt
rent car Jane White Chevy Volt
car expected
check quantity Chevy Volt # volt - 2
©2016 LogiGear Corporationwww.LogiGear.com
ACTION DEFINITION check balance
user
argument customer
argument amount
window control value
enter balance inquiry last name # customer
window control
click balance inquiry view balance
window control expected
check balance inquiry balance # amount
Re-use actions to make new actions
• In the below example we make a new action
• Existing actions are strung together to create new ones with a
broader scope
• Often steps in low level tests are re-used to create these action
definitions
:
customer amount
check balance Smith 223.45
check balance Jones 0.00
check balance James -330.45
:
use many times in tests:
define in one place:
©2016 LogiGear Corporationwww.LogiGear.com
Example action implementation in Python
# get table object, column number and column count
windowName = LIBRARY.NamedArgument("window")
tableName = LIBRARY.NamedArgument("table")
columnName = LIBRARY.NamedArgument("column")
table = ABT.OpenElement(windowName, tableName)
column = table.GetColumnIndex(columnName)
rowCount = table.GetRowCount()
# check the sort order, row by row
previous = table.GetCellText(0, column)
for i in range(1, rowCount):
current = table.GetCellText(i, column)
if current < previous :
LIBRARY.AdministerCheck("order", "sorted", "fails " + str(i+1), 0)
return
previous = current
LIBRARY.AdministerCheck("order", "sorted", "all rows in order", 1)
The following action script (in Python) will verify whether the rows in a table are sorted:
find the table in the UI
if a value is smaller than before, fail the test
if all rows are ascending, pass the test
get arguments from the test line
def action_checkSortOrder():
©2016 LogiGear Corporationwww.LogiGear.com
Using the new action
• By keeping an action generic it can be applied for a
variety of situations
• Some examples of using "check sort order":
window table column
check sort order view orders orders table ID
window table column
check sort order annual results regions revenue
window table column
check sort order inventory cars price
window table column
check sort order registration students last name
©2016 LogiGear Corporationwww.LogiGear.com
"Lead Deputy" Testing
• For "multi station" testing, when multiple machines have to
participate in a test in real-time
• For example if a supervisor needs to approve a withdrawal in a bank
teller system
• Can be "sync" and "parallel" (with a rendezvous point)
acc nr amount tx id
withdraw 123123 10,000 >> tx
name
use deputy supervisor
tx id amount
approve # tx 10,000
use lead
"bank teller"
"supervisor"
"bank teller"
Multiple System Access
System
Under Test
Action Automation
API
access
services
access
(REST etc)
UI
access
database
access
Test Modules
©2016 LogiGear Corporationwww.LogiGear.com
Detailed and unfocused tests are hard to automate
Step Description Expected
step 16 Open http://www.bigstore.com The "BIG Store" main page is displayed, with a "sign in" link
step 17 Click on "Sign In", upper right corner A sign in dialog shows, the "Sign in" button is disabled
step 18 Enter "johnd" in the user name field The "Sign In" button is still disabled
step 19 Enter "bigtester" in the password field Now the "Sign In" button is enabled
step 20 Click on the "Sign in" button The page now shows "Hello John" in the upper right corner
step 21 Enter "acme watch" in the search field The "Search" button is enabled
step 22 Click on the "Search" button 5 watches of Acme Corporation are displayed
step 23 Double click on "Acme Super Watch 2" The details page of the Acme Super Watch 2 is displayed
step 24 Verify the picture of the watch The picture should show a black Acme Super Watch 2
step 25 Select "red" in the "Color" dropdown list The picture now shows a black Acme Super Watch 2
step 26 Type 2 in the "Order quantity" textbox The price in the right shows "$79.00 + Free Shipping"
step 27 Click on "Add to cart" The status panel shows "Acme Super Watch 2" added
step 28 Click on "Check out" The Cart Check Out open, with the 2 Acme Super Watches
The above test is hard to automate in a maintainable way. Not even the smartest
engineer can "fix" a test design that is not suitable for automation.
©2016 LogiGear Corporationwww.LogiGear.com
First step: breakdown in test modules
• Straightforward Criteria
– Functionality (customers, finances, management information, UI, ...)
– Architecture of the system under test (client, server, protocol, sub systems,
components, modules, ...)
– Kind of test (navigation flow, negative tests, response time, ...)
• Additional Criteria
– Stakeholders (like "Accounting", "Compliance", "HR", ...)
– Complexity of the test (put complex tests in separate modules)
– Technical aspects of execution (picture checks, special hardware, multi-station, ...)
– Overall project planning (availability of information, timelines, sprints, ...)
– Risks involved (extra test modules for high risk areas)
– Ambition level (smoke test, regression, aggressive, …)
• Business or interaction
– Business tests: does the application reflect the business well
– Interaction tests: can a user (or another system) interact with the application
– mixing business and interaction is one of the most common, and one of the most
fatal, "anti patterns" in automated testing
©2016 LogiGear Corporationwww.LogiGear.com
• Business objects
– examples: cars, invoices, locations, etc
– tests focus on life cycles
– a majority of tests tend to fit in this category
• Business flows
– end-to-end workflow style test,
– example: create, fulfill, pay and close an order
• "Other tests"
– functions and features, like premium calculation or PDF output
– administration, users, security, authorizations
– graphics
– technologies, protocols, ...
– customization, extensibility
– . . .
Example of breakdown considerations
©2016 LogiGear Corporationwww.LogiGear.com
Example Top Level Structure
Project create, update, delete/close
copy, move
categorize, enumerate, identify
convert, serialize, export/import, ...
UI, dialogs, forms, pages
input (validation, defaulting, dependencies)
flows (primary paths, alternate paths)
keyboard shortcuts, keyboard controls, ...
. . .
"Cars"
Lifecycles, data operations
Interaction
Functions and Features
Technologies, protocols, controls
Data (handling, quality, ETL, ...)
Security, authorization, admin
Graphics, multi-media, charts, ...
Interoperability
Customizing, extensibility
Business Flows
Concurrency, race conditions, ...
Business Objects
processes, transactions, end-to-end, day in the life,
combinations of flows, ...
calculations, analyses, PDF output, ...
"Customers"
. . .
©2016 LogiGear Corporationwww.LogiGear.com
What about existing tests?
• Compare to moving house:
– some effort can't be avoided
– be selective, edit your stuff,
• look at the future, not the past
– first decide where to put what, then put it there
– moving is an opportunity, you may not get such chance again soon
• Follow the module approach
– define the modules and their scope as if from scratch
– use the existing test cases in two ways:
• verify completeness
• harvest and re-use them for tests and for actions
– avoid porting over "step by step", in particular avoid over-checking
©2016 LogiGear Corporationwww.LogiGear.com
Scope of a test module
• Always know the scope of the test module
• The scope should be unambiguous
• The scope determines many things:
– what the test objectives are
– which test cases to expect
– what level of actions to use
– what the checks are about and which events should generate a
warning or error (if a “lower” functionality is wrong)
• In other words: keep your eye on the ball . . .
©2016 LogiGear Corporationwww.LogiGear.com
Tip: Use right level actions
project task
check task in project Drill Assembly Plan of Approach
 Better to have a business level action and hide the details in an action
definition:
 But do show details when it matters. The example below is too high
level, requires drill down into action definition to understand what is
tested.
window tree tree item path
click tree item main projects /Projects/Drill Assembly
window list item
check list item exists main tasks Plan of Approach
©2016 LogiGear Corporationwww.LogiGear.com
Stable and Robust Automation
• Test your automation
– develop separate tests for the actions and interface mappings themselves
• Identifying properties of UI elements
– a lot of time is spent on identifying and re-identifying UI elements
• "active" timing
– wait based on conditions in the system under test, avoid fixed “sleeps”
– if needed request developers to provide hooks
• Testing and automation friendly system designs
– with well-defined components, services, tiers, etc
• Keep an eye on the test design
– unsuccessful automation virtually always can be traced back to poor test design
©2016 LogiGear Corporationwww.LogiGear.com
Mapping an interface
• An interface mapping (common in test tools) will map windows and controls to names
• When the interface of an application changes, you only have to update this in one place
• The interface mapping is a key step in your automation success, allocate time to design
it well, in particular naming and choosing identifying properties
INTERFACE ENTITY library
interface entity setting title {.*Music Library}
name class label
interface element title text Title:
interface element artist text Artist:
interface element file size text File size (Kb):
name class position
interface element playing time text textbox 4
interface element file type text textbox 5
interface element bitrate text textbox 6
name class position
interface element music treeview treeview 1
©2016 LogiGear Corporationwww.LogiGear.com
Hidden Identifying Property
©2016 LogiGear Corporationwww.LogiGear.com
Mapping the interface
(style we use in our product)
• An interface mapping will map windows and controls to names
• When the interface of an application changes, you only have to update this in one place
• The interface mapping is a key step in your automation success, allocate time to design it well
©2016 LogiGear Corporationwww.LogiGear.com
• Testability should be understood as a "must have" requirement
– first question in a development project: "how do we test this?"
• Define (assign) identifying properties for any interface element (control,
HTML element, etc)
• Have hooks for timing
– avoid "hard sleeps" at all costs
• Have white-box access to anything interesting:
– input data (ability to emulate)
– output data (what is underlying data being displayed)
– random generators (can I set a seed?)
– states (like in a game)
– objects displayed (like monsters in a game)
– pay attention to security, don't leave hacking doors open
• A well-designed system, with tiers, components, services, is also very
helpful for test design and automation
Testability, some key items
©2016 LogiGear Corporationwww.LogiGear.com
Test design and automation in agile
High Level Test
Design (organization
of test modules)
Test Module Development
Interface Definition
Action Automation
Test Execution
Sprint Products
Product
Backlog
Test re-use
Automation re-use
product
owner
team
prod owner
& team
User stories
Documentation
Domain understanding
Acceptance Criteria
PO Questions
Situations
Relations
Agile life cycle
Test development
Main Level Test Modules
Interaction Test Modules
Cross over Test Modules
©2016 LogiGear Corporationwww.LogiGear.com
Pre-conditions for Sprints
• Agreement on the approach:
– is testability a requirement for the software?
• Understanding of subject matter, testing, automation,
method, best practices, etc
• Automation technology available and working
– UI, non-UI, custom controls, graphics, embedded, etc
• A plan on how you to deal with data and environments
To discuss an approach, consider daily "sit down" meetings with some or
all members to coach and evaluate
 an end-of-day counterpart to the early-morning "stand up" meetings
 short and friendly, not about progress and impediments, but about practices and
experiences with them (like "what actions did you use?")
 a few meetings may suffice
©2016 LogiGear Corporationwww.LogiGear.com
Facilitation outside sprints, example
Sprint
Have a facility (internal or outsourced) to help the agile life cycle
• keep up, catch up, continue beyond "done"
• one outsourced facility can support multiple teams, smooth out peaks and valleys
• allow sprint teams to focus on development priorities
• team is responsible, and decides if, what and when to outsource
Sprint
Test Definition & Development
T
E
A
M
Additional Test Development
Additional Automation
F
A
C
I
L
I
T
Y
Test Development
Automation
Automation
Test Definition & Development
Automation
©2016 LogiGear Corporationwww.LogiGear.com
• QA needs to keep up with the rest of the team
• Test design is a key driver for automation success/failure
• Testability, as a team responsibility, is a core requirement
• Organizing the testing well in the sprints gets more of it
"done"
• Having an agile facilitation model can support the QA's
to stay in sync
Summary

More Related Content

What's hot

Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
TechWell
 
Agile tour ncr test360_degree - agile testing on steroids
Agile tour ncr test360_degree - agile testing on steroidsAgile tour ncr test360_degree - agile testing on steroids
Agile tour ncr test360_degree - agile testing on steroidsVipul Gupta
 
Test pyramid
Test pyramidTest pyramid
Test pyramid
Poornima Durairaj
 
A journey to a Full Stack Tester
A journey to a Full Stack Tester A journey to a Full Stack Tester
A journey to a Full Stack Tester
KMS Technology
 
Software Testing with Agile Requirements Practices
Software Testing with Agile Requirements Practices Software Testing with Agile Requirements Practices
Software Testing with Agile Requirements Practices A B M Moniruzzaman
 
From Gatekeeper to Partner by Kelsey Shannahan
From Gatekeeper to Partner by Kelsey ShannahanFrom Gatekeeper to Partner by Kelsey Shannahan
From Gatekeeper to Partner by Kelsey Shannahan
QA or the Highway
 
Why Automated Testing Matters To DevOps
Why Automated Testing Matters To DevOpsWhy Automated Testing Matters To DevOps
Why Automated Testing Matters To DevOps
dpaulmerrill
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
TechWell
 
10 signs your testing is not enough
10 signs your testing is not enough10 signs your testing is not enough
10 signs your testing is not enough
SQALab
 
QA Process Overview for Firefox OS 2014
QA Process Overview for Firefox OS 2014QA Process Overview for Firefox OS 2014
QA Process Overview for Firefox OS 2014
Anthony Chung
 
Agile Testing Analytics
Agile Testing AnalyticsAgile Testing Analytics
Agile Testing Analytics
QASymphony
 
Agile testing principles and practices - Anil Karade
Agile testing principles and practices - Anil KaradeAgile testing principles and practices - Anil Karade
Agile testing principles and practices - Anil Karade
IndicThreads
 
Test team dynamics, Антон Мужайло
Test team dynamics, Антон МужайлоTest team dynamics, Антон Мужайло
Test team dynamics, Антон Мужайло
Sigma Software
 
Relieveing the Testing Bottle Neck - Webinar
Relieveing the Testing Bottle Neck - WebinarRelieveing the Testing Bottle Neck - Webinar
Relieveing the Testing Bottle Neck - Webinar
Cprime
 
Continuous Deployment and Testing Workshop from Better Software West
Continuous Deployment and Testing Workshop from Better Software WestContinuous Deployment and Testing Workshop from Better Software West
Continuous Deployment and Testing Workshop from Better Software West
Cory Foy
 
Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...
Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...
Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...
Codemotion
 
Agile testingandautomation
Agile testingandautomationAgile testingandautomation
Agile testingandautomation
jeisner
 
Test Automation Strategies for the Agile World
Test Automation Strategies for the Agile WorldTest Automation Strategies for the Agile World
Test Automation Strategies for the Agile World
TechWell
 
Agile Testing - presentation for Agile User Group
Agile Testing - presentation for Agile User GroupAgile Testing - presentation for Agile User Group
Agile Testing - presentation for Agile User Group
suwalki24.pl
 
Qa process 2012
Qa process 2012Qa process 2012
Qa process 2012
ashokack
 

What's hot (20)

Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
 
Agile tour ncr test360_degree - agile testing on steroids
Agile tour ncr test360_degree - agile testing on steroidsAgile tour ncr test360_degree - agile testing on steroids
Agile tour ncr test360_degree - agile testing on steroids
 
Test pyramid
Test pyramidTest pyramid
Test pyramid
 
A journey to a Full Stack Tester
A journey to a Full Stack Tester A journey to a Full Stack Tester
A journey to a Full Stack Tester
 
Software Testing with Agile Requirements Practices
Software Testing with Agile Requirements Practices Software Testing with Agile Requirements Practices
Software Testing with Agile Requirements Practices
 
From Gatekeeper to Partner by Kelsey Shannahan
From Gatekeeper to Partner by Kelsey ShannahanFrom Gatekeeper to Partner by Kelsey Shannahan
From Gatekeeper to Partner by Kelsey Shannahan
 
Why Automated Testing Matters To DevOps
Why Automated Testing Matters To DevOpsWhy Automated Testing Matters To DevOps
Why Automated Testing Matters To DevOps
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
 
10 signs your testing is not enough
10 signs your testing is not enough10 signs your testing is not enough
10 signs your testing is not enough
 
QA Process Overview for Firefox OS 2014
QA Process Overview for Firefox OS 2014QA Process Overview for Firefox OS 2014
QA Process Overview for Firefox OS 2014
 
Agile Testing Analytics
Agile Testing AnalyticsAgile Testing Analytics
Agile Testing Analytics
 
Agile testing principles and practices - Anil Karade
Agile testing principles and practices - Anil KaradeAgile testing principles and practices - Anil Karade
Agile testing principles and practices - Anil Karade
 
Test team dynamics, Антон Мужайло
Test team dynamics, Антон МужайлоTest team dynamics, Антон Мужайло
Test team dynamics, Антон Мужайло
 
Relieveing the Testing Bottle Neck - Webinar
Relieveing the Testing Bottle Neck - WebinarRelieveing the Testing Bottle Neck - Webinar
Relieveing the Testing Bottle Neck - Webinar
 
Continuous Deployment and Testing Workshop from Better Software West
Continuous Deployment and Testing Workshop from Better Software WestContinuous Deployment and Testing Workshop from Better Software West
Continuous Deployment and Testing Workshop from Better Software West
 
Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...
Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...
Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...
 
Agile testingandautomation
Agile testingandautomationAgile testingandautomation
Agile testingandautomation
 
Test Automation Strategies for the Agile World
Test Automation Strategies for the Agile WorldTest Automation Strategies for the Agile World
Test Automation Strategies for the Agile World
 
Agile Testing - presentation for Agile User Group
Agile Testing - presentation for Agile User GroupAgile Testing - presentation for Agile User Group
Agile Testing - presentation for Agile User Group
 
Qa process 2012
Qa process 2012Qa process 2012
Qa process 2012
 

Similar to How to get Automated Testing "Done"

Integration Testing with Selenium
Integration Testing with SeleniumIntegration Testing with Selenium
Integration Testing with Selenium
All Things Open
 
You have Selenium... Now what?
You have Selenium... Now what?You have Selenium... Now what?
You have Selenium... Now what?
Great Wide Open
 
Build the Right Regression Suite with Behavior-Driven Testing
Build the Right Regression Suite with Behavior-Driven TestingBuild the Right Regression Suite with Behavior-Driven Testing
Build the Right Regression Suite with Behavior-Driven Testing
TechWell
 
'BIG Testing' with Hans Buwalda
'BIG Testing' with Hans Buwalda 'BIG Testing' with Hans Buwalda
'BIG Testing' with Hans Buwalda
TEST Huddle
 
Qtp - Introduction values
Qtp - Introduction valuesQtp - Introduction values
Qtp - Introduction values
Vibrant Technologies & Computers
 
QAustral Testing
QAustral   TestingQAustral   Testing
QAustral Testing
cusmaim
 
Machine learning in action at Pipedrive
Machine learning in action at PipedriveMachine learning in action at Pipedrive
Machine learning in action at Pipedrive
André Karpištšenko
 
Continuous Performance Testing and Monitoring in Agile Development
Continuous Performance Testing and Monitoring in Agile DevelopmentContinuous Performance Testing and Monitoring in Agile Development
Continuous Performance Testing and Monitoring in Agile Development
Dynatrace
 
Agile testing - Testing From Day 1
Agile testing - Testing From Day 1Agile testing - Testing From Day 1
Agile testing - Testing From Day 1
Kaizenko
 
Backward thinking design qa system for quality goals
Backward thinking   design qa system for quality goalsBackward thinking   design qa system for quality goals
Backward thinking design qa system for quality goals
gaoliang641
 
Continuous Performance Testing: The New Standard
Continuous Performance Testing: The New StandardContinuous Performance Testing: The New Standard
Continuous Performance Testing: The New Standard
TechWell
 
Agile Testing Framework - The Art of Automated Testing
Agile Testing Framework - The Art of Automated TestingAgile Testing Framework - The Art of Automated Testing
Agile Testing Framework - The Art of Automated Testing
Dimitri Ponomareff
 
Automation in the world of project
Automation  in the world of projectAutomation  in the world of project
Automation in the world of project
Zbyszek Mockun
 
QA Role in Agile Teams
QA Role in Agile Teams QA Role in Agile Teams
QA Role in Agile Teams
Synerzip
 
In memory OLAP engine
In memory OLAP engineIn memory OLAP engine
In memory OLAP engineWO Community
 
Advanced structures india introduction
Advanced structures india introductionAdvanced structures india introduction
Advanced structures india introduction
Advanced Structures India Private Limited
 
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity Software Ireland
 
Agile Gurugram 2016 | Conference | Change Vector Tracking in emergent Archite...
Agile Gurugram 2016 | Conference | Change Vector Tracking in emergent Archite...Agile Gurugram 2016 | Conference | Change Vector Tracking in emergent Archite...
Agile Gurugram 2016 | Conference | Change Vector Tracking in emergent Archite...
AgileNetwork
 
Specification-by-Example: A Cucumber Implementation
Specification-by-Example: A Cucumber ImplementationSpecification-by-Example: A Cucumber Implementation
Specification-by-Example: A Cucumber Implementation
TechWell
 
Beginners overview of automated testing with Rspec
Beginners overview of automated testing with RspecBeginners overview of automated testing with Rspec
Beginners overview of automated testing with Rspec
jeffrey1ross
 

Similar to How to get Automated Testing "Done" (20)

Integration Testing with Selenium
Integration Testing with SeleniumIntegration Testing with Selenium
Integration Testing with Selenium
 
You have Selenium... Now what?
You have Selenium... Now what?You have Selenium... Now what?
You have Selenium... Now what?
 
Build the Right Regression Suite with Behavior-Driven Testing
Build the Right Regression Suite with Behavior-Driven TestingBuild the Right Regression Suite with Behavior-Driven Testing
Build the Right Regression Suite with Behavior-Driven Testing
 
'BIG Testing' with Hans Buwalda
'BIG Testing' with Hans Buwalda 'BIG Testing' with Hans Buwalda
'BIG Testing' with Hans Buwalda
 
Qtp - Introduction values
Qtp - Introduction valuesQtp - Introduction values
Qtp - Introduction values
 
QAustral Testing
QAustral   TestingQAustral   Testing
QAustral Testing
 
Machine learning in action at Pipedrive
Machine learning in action at PipedriveMachine learning in action at Pipedrive
Machine learning in action at Pipedrive
 
Continuous Performance Testing and Monitoring in Agile Development
Continuous Performance Testing and Monitoring in Agile DevelopmentContinuous Performance Testing and Monitoring in Agile Development
Continuous Performance Testing and Monitoring in Agile Development
 
Agile testing - Testing From Day 1
Agile testing - Testing From Day 1Agile testing - Testing From Day 1
Agile testing - Testing From Day 1
 
Backward thinking design qa system for quality goals
Backward thinking   design qa system for quality goalsBackward thinking   design qa system for quality goals
Backward thinking design qa system for quality goals
 
Continuous Performance Testing: The New Standard
Continuous Performance Testing: The New StandardContinuous Performance Testing: The New Standard
Continuous Performance Testing: The New Standard
 
Agile Testing Framework - The Art of Automated Testing
Agile Testing Framework - The Art of Automated TestingAgile Testing Framework - The Art of Automated Testing
Agile Testing Framework - The Art of Automated Testing
 
Automation in the world of project
Automation  in the world of projectAutomation  in the world of project
Automation in the world of project
 
QA Role in Agile Teams
QA Role in Agile Teams QA Role in Agile Teams
QA Role in Agile Teams
 
In memory OLAP engine
In memory OLAP engineIn memory OLAP engine
In memory OLAP engine
 
Advanced structures india introduction
Advanced structures india introductionAdvanced structures india introduction
Advanced structures india introduction
 
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
 
Agile Gurugram 2016 | Conference | Change Vector Tracking in emergent Archite...
Agile Gurugram 2016 | Conference | Change Vector Tracking in emergent Archite...Agile Gurugram 2016 | Conference | Change Vector Tracking in emergent Archite...
Agile Gurugram 2016 | Conference | Change Vector Tracking in emergent Archite...
 
Specification-by-Example: A Cucumber Implementation
Specification-by-Example: A Cucumber ImplementationSpecification-by-Example: A Cucumber Implementation
Specification-by-Example: A Cucumber Implementation
 
Beginners overview of automated testing with Rspec
Beginners overview of automated testing with RspecBeginners overview of automated testing with Rspec
Beginners overview of automated testing with Rspec
 

More from TEST Huddle

Why We Need Diversity in Testing- Accenture
Why We Need Diversity in Testing- AccentureWhy We Need Diversity in Testing- Accenture
Why We Need Diversity in Testing- Accenture
TEST Huddle
 
Keys to continuous testing for faster delivery euro star webinar
Keys to continuous testing for faster delivery euro star webinar Keys to continuous testing for faster delivery euro star webinar
Keys to continuous testing for faster delivery euro star webinar
TEST Huddle
 
Why you Shouldnt Automated But You Will Anyway
Why you Shouldnt Automated But You Will Anyway Why you Shouldnt Automated But You Will Anyway
Why you Shouldnt Automated But You Will Anyway
TEST Huddle
 
Being a Tester in Scrum
Being a Tester in ScrumBeing a Tester in Scrum
Being a Tester in Scrum
TEST Huddle
 
Leveraging Visual Testing with Your Functional Tests
Leveraging Visual Testing with Your Functional TestsLeveraging Visual Testing with Your Functional Tests
Leveraging Visual Testing with Your Functional Tests
TEST Huddle
 
Using Test Trees to get an Overview of Test Work
Using Test Trees to get an Overview of Test WorkUsing Test Trees to get an Overview of Test Work
Using Test Trees to get an Overview of Test Work
TEST Huddle
 
Big Data: The Magic to Attain New Heights
Big Data:  The Magic to Attain New HeightsBig Data:  The Magic to Attain New Heights
Big Data: The Magic to Attain New Heights
TEST Huddle
 
Will Robots Replace Testers?
Will Robots Replace Testers?Will Robots Replace Testers?
Will Robots Replace Testers?
TEST Huddle
 
TDD For The Rest Of Us
TDD For The Rest Of UsTDD For The Rest Of Us
TDD For The Rest Of Us
TEST Huddle
 
Scaling Agile with LeSS (Large Scale Scrum)
Scaling Agile with LeSS (Large Scale Scrum)Scaling Agile with LeSS (Large Scale Scrum)
Scaling Agile with LeSS (Large Scale Scrum)
TEST Huddle
 
Creating Agile Test Strategies for Larger Enterprises
Creating Agile Test Strategies for Larger EnterprisesCreating Agile Test Strategies for Larger Enterprises
Creating Agile Test Strategies for Larger Enterprises
TEST Huddle
 
Is There A Risk?
Is There A Risk?Is There A Risk?
Is There A Risk?
TEST Huddle
 
Are Your Tests Well-Travelled? Thoughts About Test Coverage
Are Your Tests Well-Travelled? Thoughts About Test CoverageAre Your Tests Well-Travelled? Thoughts About Test Coverage
Are Your Tests Well-Travelled? Thoughts About Test Coverage
TEST Huddle
 
Do we need testers on agile teams?
Do we need testers on agile teams?Do we need testers on agile teams?
Do we need testers on agile teams?
TEST Huddle
 
Testers & Teams on the Agile Fluency™ Journey
Testers & Teams on the Agile Fluency™ Journey Testers & Teams on the Agile Fluency™ Journey
Testers & Teams on the Agile Fluency™ Journey
TEST Huddle
 
Practical Test Strategy Using Heuristics
Practical Test Strategy Using HeuristicsPractical Test Strategy Using Heuristics
Practical Test Strategy Using Heuristics
TEST Huddle
 
Thinking Through Your Role
Thinking Through Your RoleThinking Through Your Role
Thinking Through Your Role
TEST Huddle
 
Using Selenium 3 0
Using Selenium 3 0Using Selenium 3 0
Using Selenium 3 0
TEST Huddle
 
New Model Testing: A New Test Process and Tool
New Model Testing:  A New Test Process and ToolNew Model Testing:  A New Test Process and Tool
New Model Testing: A New Test Process and Tool
TEST Huddle
 
Five Digital Age Trends That Will Dramatically Impact Testing And Quality Sk...
 Five Digital Age Trends That Will Dramatically Impact Testing And Quality Sk... Five Digital Age Trends That Will Dramatically Impact Testing And Quality Sk...
Five Digital Age Trends That Will Dramatically Impact Testing And Quality Sk...
TEST Huddle
 

More from TEST Huddle (20)

Why We Need Diversity in Testing- Accenture
Why We Need Diversity in Testing- AccentureWhy We Need Diversity in Testing- Accenture
Why We Need Diversity in Testing- Accenture
 
Keys to continuous testing for faster delivery euro star webinar
Keys to continuous testing for faster delivery euro star webinar Keys to continuous testing for faster delivery euro star webinar
Keys to continuous testing for faster delivery euro star webinar
 
Why you Shouldnt Automated But You Will Anyway
Why you Shouldnt Automated But You Will Anyway Why you Shouldnt Automated But You Will Anyway
Why you Shouldnt Automated But You Will Anyway
 
Being a Tester in Scrum
Being a Tester in ScrumBeing a Tester in Scrum
Being a Tester in Scrum
 
Leveraging Visual Testing with Your Functional Tests
Leveraging Visual Testing with Your Functional TestsLeveraging Visual Testing with Your Functional Tests
Leveraging Visual Testing with Your Functional Tests
 
Using Test Trees to get an Overview of Test Work
Using Test Trees to get an Overview of Test WorkUsing Test Trees to get an Overview of Test Work
Using Test Trees to get an Overview of Test Work
 
Big Data: The Magic to Attain New Heights
Big Data:  The Magic to Attain New HeightsBig Data:  The Magic to Attain New Heights
Big Data: The Magic to Attain New Heights
 
Will Robots Replace Testers?
Will Robots Replace Testers?Will Robots Replace Testers?
Will Robots Replace Testers?
 
TDD For The Rest Of Us
TDD For The Rest Of UsTDD For The Rest Of Us
TDD For The Rest Of Us
 
Scaling Agile with LeSS (Large Scale Scrum)
Scaling Agile with LeSS (Large Scale Scrum)Scaling Agile with LeSS (Large Scale Scrum)
Scaling Agile with LeSS (Large Scale Scrum)
 
Creating Agile Test Strategies for Larger Enterprises
Creating Agile Test Strategies for Larger EnterprisesCreating Agile Test Strategies for Larger Enterprises
Creating Agile Test Strategies for Larger Enterprises
 
Is There A Risk?
Is There A Risk?Is There A Risk?
Is There A Risk?
 
Are Your Tests Well-Travelled? Thoughts About Test Coverage
Are Your Tests Well-Travelled? Thoughts About Test CoverageAre Your Tests Well-Travelled? Thoughts About Test Coverage
Are Your Tests Well-Travelled? Thoughts About Test Coverage
 
Do we need testers on agile teams?
Do we need testers on agile teams?Do we need testers on agile teams?
Do we need testers on agile teams?
 
Testers & Teams on the Agile Fluency™ Journey
Testers & Teams on the Agile Fluency™ Journey Testers & Teams on the Agile Fluency™ Journey
Testers & Teams on the Agile Fluency™ Journey
 
Practical Test Strategy Using Heuristics
Practical Test Strategy Using HeuristicsPractical Test Strategy Using Heuristics
Practical Test Strategy Using Heuristics
 
Thinking Through Your Role
Thinking Through Your RoleThinking Through Your Role
Thinking Through Your Role
 
Using Selenium 3 0
Using Selenium 3 0Using Selenium 3 0
Using Selenium 3 0
 
New Model Testing: A New Test Process and Tool
New Model Testing:  A New Test Process and ToolNew Model Testing:  A New Test Process and Tool
New Model Testing: A New Test Process and Tool
 
Five Digital Age Trends That Will Dramatically Impact Testing And Quality Sk...
 Five Digital Age Trends That Will Dramatically Impact Testing And Quality Sk... Five Digital Age Trends That Will Dramatically Impact Testing And Quality Sk...
Five Digital Age Trends That Will Dramatically Impact Testing And Quality Sk...
 

Recently uploaded

Into the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdfInto the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdf
Ortus Solutions, Corp
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
Globus
 
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptx
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxTop Features to Include in Your Winzo Clone App for Business Growth (4).pptx
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptx
rickgrimesss22
 
Graphic Design Crash Course for beginners
Graphic Design Crash Course for beginnersGraphic Design Crash Course for beginners
Graphic Design Crash Course for beginners
e20449
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Natan Silnitsky
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New Zealand
IES VE
 
Understanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSageUnderstanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSage
Globus
 
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfEnhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Jay Das
 
Accelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with PlatformlessAccelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with Platformless
WSO2
 
2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx
Georgi Kodinov
 
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Shahin Sheidaei
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
AMB-Review
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
Paco van Beckhoven
 
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
Tier1 app
 
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of PassageA Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
Philip Schwarz
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Globus
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
informapgpstrackings
 
May Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdfMay Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdf
Adele Miller
 
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
Globus
 

Recently uploaded (20)

Into the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdfInto the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdf
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
 
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptx
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxTop Features to Include in Your Winzo Clone App for Business Growth (4).pptx
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptx
 
Graphic Design Crash Course for beginners
Graphic Design Crash Course for beginnersGraphic Design Crash Course for beginners
Graphic Design Crash Course for beginners
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New Zealand
 
Understanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSageUnderstanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSage
 
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfEnhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
 
Accelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with PlatformlessAccelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with Platformless
 
2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx
 
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
 
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
 
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of PassageA Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
 
May Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdfMay Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdf
 
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
 

How to get Automated Testing "Done"

  • 1. How to get Automated Testing "Done" Hans Buwalda LogiGear www.LogiGear.com hans@logigear.com @hansbuwalda www.TestArchitect.com www.HappyTester.com
  • 2. ©2016 LogiGear Corporationwww.LogiGear.com Who is your speaker • Software testing company, around since 1994 – Foster City, California – consultancy, training – test development and automation services – "test integrated" development services • Products: – TestArchitect™, TestArchitect for Visual Studio™ – integrating test development with test management and automation – based on modularized keyword-driven testing • LogiGear Magazine: – themed issues, non-commercial • Dutch guy, in California with LogiGear since 2001 • Background in math, computer science, management • Since 1994 focusing on testing and automation – keywords, agile testing, big testing Hans Buwalda LogiGear Corporation www.LogiGear.com hans@logigear.com @hansbuwalda www.TestArchitect.com www.HappyTester.com
  • 3. ©2016 LogiGear Corporationwww.LogiGear.com Relation to code Quality / depth Automation Scalability Unit Testing Close relationship with the code Singular test scope, but deep into the code Fully automated by nature Scalable, grows with the code, easy to repeat Functional Testing Usually does not have a one-on-one relation with code Quality and scope depends on test design In particular UI based automation can be a challenge Often a bottle- neck in scalability Exploratory Testing Human driven, not seeking a relation with code Usually deep and thorough, good at finding problems May or may not be automated afterwards Not meant to be repeatable. Rather do a new session Some test kinds and their scalability (simplified)
  • 4. ©2016 LogiGear Corporationwww.LogiGear.com • High degree of automation coverage – under Agile and DevOps automation went from nice-to-have to must-have • Scrum teams are well suited to, collectively, own testing and automation • QA members (test development and automation) need to stay in the same sprint with the rest of the team – developers and product owners are a great help for testing, and vice versa – otherwise the task gets harder, and will delay even more • However, it is not always practical to have fully automated tests as part of "done" for a sprint . . . Success factors in Agile (Scrum)
  • 5. ©2016 LogiGear Corporationwww.LogiGear.com • More test cases needed than is time for • Shooting a moving target, changes are common • Lacking technology • Since QA benefits greatly from close cooperation with PO's and Devs, matters might get worse once out of sync • Not always practical to let development wait for QA to finish (true meaning of "done") QA role may have a hard time keeping up with the rest of the team Sprint Sprint Sprint PO Dev QA PO Dev QA PO Dev QA R O L E S PO Dev
  • 6. ©2016 LogiGear Corporationwww.LogiGear.com • Design tests to be automation-friendly – successful automation is not as much a technical challenge as it is a test design challenge • Make tests easy to understand for all involved – so, not too technical please – note: "easy to understand" does not necessarily mean "easy" • Invest in testability of the system under test – one of the main benefits of having a team – works two ways: good tests facilitate "test driven" development • Organize the activities well – I like to consider tests, and their automation, as "products" – let the testing and automation activities fit the sprint – support the teams well with infrastructure, technology, services, etc Approaches
  • 7. ©2016 LogiGear Corporationwww.LogiGear.com Easy to understand ? ? ? WebElement element = driver.findElement(By.name(name)); element.sendKeys("mystery magic"); element.submit(); (new WebDriverWait(driver, 10)).until( new ExpectedCondition<Boolean>() { public Boolean apply(WebDriver d) { return d.getTitle() .toLowerCase().startsWith("mystery"); } } ); System.out.println(driver.getTitle());
  • 8. ©2016 LogiGear Corporationwww.LogiGear.com More easy to understand: actions 4 actions, each with an action keyword and arguments read from top to bottom fragment from a test with actions acc nr first last open account 123123 John Doe acc nr amount deposit 123123 10.11 deposit 123123 20.22 acc nr expected check balance 123123 30.33 • The test developer creates tests using actions with keywords and arguments in a easy to understand spreadsheet format • Checks are, as much as possible, explicit (specified expected values) • The automation task focuses on automating the keywords, each keyword is automated only once • In addition to keywords you can also use "given-then-when" scenarios (BDD) for good readability
  • 9. ©2016 LogiGear Corporationwww.LogiGear.com Supporting BDD with actions Given a customer previously bought a black sweater from me And I currently have three black sweaters left in stock When he returns the sweater for a refund Then I should have four black sweaters in stock customer buys, article, color a customer previously bought a {color} {article} from me set stock, article, color, amount I currently have {amount} {color} {article} left in stock my stock of {color} {sweater} is {amount} return article, article, color he returns the {color} {article} for a refund check stock, article, color I should have four {color} {article} in stock article color customer buys sweater black article color amount set stock sweater black 3 article color return article sweater black article color amount check stock sweater black 4 email me for this tool: hans@logigear.com
  • 10. ©2016 LogiGear Corporationwww.LogiGear.com • Modularized and keyword-driven method for automated testing • Particularly suited for large and complex projects • High degree of agility, even when used in traditional projects • High level of automation, 95% or more is the norm • Puts the focus on test design, less on technology "Action Based Testing"
  • 11. High Level Test Design - Test Development Plan Objectives Test Module 1 Test Cases Test Module 2 Test Module N Actions . . . AUTOMATION Objectives Objectives interaction test business test define the "chapters" create the "chapters" create the "words" make the words work Test Cases Test Cases window control value enter log in user name jdoe enter log in password car guy window control property expected check property log in ok button enabled true user password log in jdoe car guy first last brand model enter rental Mary Renter Ford Escape last total check bill Renter 140.42 Action Based Testing
  • 12. Test Module Sections Test Module Objectives Initial - setup Test cases Final - cleanup
  • 13. ©2016 LogiGear Corporationwww.LogiGear.com Variables and expressions with keywords • This test does not need an absolute number for the available cars, just wants to see if a stock is updated • As a convention we denote an assignment with ">>" • The "#" indicates an expression TEST CASE TC 02 Rent some more cars car available get quantity Chevy Volt >> volt first name last name car rent car John Green Chevy Volt rent car Jane White Chevy Volt car expected check quantity Chevy Volt # volt - 2
  • 14. ©2016 LogiGear Corporationwww.LogiGear.com ACTION DEFINITION check balance user argument customer argument amount window control value enter balance inquiry last name # customer window control click balance inquiry view balance window control expected check balance inquiry balance # amount Re-use actions to make new actions • In the below example we make a new action • Existing actions are strung together to create new ones with a broader scope • Often steps in low level tests are re-used to create these action definitions : customer amount check balance Smith 223.45 check balance Jones 0.00 check balance James -330.45 : use many times in tests: define in one place:
  • 15. ©2016 LogiGear Corporationwww.LogiGear.com Example action implementation in Python # get table object, column number and column count windowName = LIBRARY.NamedArgument("window") tableName = LIBRARY.NamedArgument("table") columnName = LIBRARY.NamedArgument("column") table = ABT.OpenElement(windowName, tableName) column = table.GetColumnIndex(columnName) rowCount = table.GetRowCount() # check the sort order, row by row previous = table.GetCellText(0, column) for i in range(1, rowCount): current = table.GetCellText(i, column) if current < previous : LIBRARY.AdministerCheck("order", "sorted", "fails " + str(i+1), 0) return previous = current LIBRARY.AdministerCheck("order", "sorted", "all rows in order", 1) The following action script (in Python) will verify whether the rows in a table are sorted: find the table in the UI if a value is smaller than before, fail the test if all rows are ascending, pass the test get arguments from the test line def action_checkSortOrder():
  • 16. ©2016 LogiGear Corporationwww.LogiGear.com Using the new action • By keeping an action generic it can be applied for a variety of situations • Some examples of using "check sort order": window table column check sort order view orders orders table ID window table column check sort order annual results regions revenue window table column check sort order inventory cars price window table column check sort order registration students last name
  • 17. ©2016 LogiGear Corporationwww.LogiGear.com "Lead Deputy" Testing • For "multi station" testing, when multiple machines have to participate in a test in real-time • For example if a supervisor needs to approve a withdrawal in a bank teller system • Can be "sync" and "parallel" (with a rendezvous point) acc nr amount tx id withdraw 123123 10,000 >> tx name use deputy supervisor tx id amount approve # tx 10,000 use lead "bank teller" "supervisor" "bank teller"
  • 18. Multiple System Access System Under Test Action Automation API access services access (REST etc) UI access database access Test Modules
  • 19. ©2016 LogiGear Corporationwww.LogiGear.com Detailed and unfocused tests are hard to automate Step Description Expected step 16 Open http://www.bigstore.com The "BIG Store" main page is displayed, with a "sign in" link step 17 Click on "Sign In", upper right corner A sign in dialog shows, the "Sign in" button is disabled step 18 Enter "johnd" in the user name field The "Sign In" button is still disabled step 19 Enter "bigtester" in the password field Now the "Sign In" button is enabled step 20 Click on the "Sign in" button The page now shows "Hello John" in the upper right corner step 21 Enter "acme watch" in the search field The "Search" button is enabled step 22 Click on the "Search" button 5 watches of Acme Corporation are displayed step 23 Double click on "Acme Super Watch 2" The details page of the Acme Super Watch 2 is displayed step 24 Verify the picture of the watch The picture should show a black Acme Super Watch 2 step 25 Select "red" in the "Color" dropdown list The picture now shows a black Acme Super Watch 2 step 26 Type 2 in the "Order quantity" textbox The price in the right shows "$79.00 + Free Shipping" step 27 Click on "Add to cart" The status panel shows "Acme Super Watch 2" added step 28 Click on "Check out" The Cart Check Out open, with the 2 Acme Super Watches The above test is hard to automate in a maintainable way. Not even the smartest engineer can "fix" a test design that is not suitable for automation.
  • 20. ©2016 LogiGear Corporationwww.LogiGear.com First step: breakdown in test modules • Straightforward Criteria – Functionality (customers, finances, management information, UI, ...) – Architecture of the system under test (client, server, protocol, sub systems, components, modules, ...) – Kind of test (navigation flow, negative tests, response time, ...) • Additional Criteria – Stakeholders (like "Accounting", "Compliance", "HR", ...) – Complexity of the test (put complex tests in separate modules) – Technical aspects of execution (picture checks, special hardware, multi-station, ...) – Overall project planning (availability of information, timelines, sprints, ...) – Risks involved (extra test modules for high risk areas) – Ambition level (smoke test, regression, aggressive, …) • Business or interaction – Business tests: does the application reflect the business well – Interaction tests: can a user (or another system) interact with the application – mixing business and interaction is one of the most common, and one of the most fatal, "anti patterns" in automated testing
  • 21. ©2016 LogiGear Corporationwww.LogiGear.com • Business objects – examples: cars, invoices, locations, etc – tests focus on life cycles – a majority of tests tend to fit in this category • Business flows – end-to-end workflow style test, – example: create, fulfill, pay and close an order • "Other tests" – functions and features, like premium calculation or PDF output – administration, users, security, authorizations – graphics – technologies, protocols, ... – customization, extensibility – . . . Example of breakdown considerations
  • 22. ©2016 LogiGear Corporationwww.LogiGear.com Example Top Level Structure Project create, update, delete/close copy, move categorize, enumerate, identify convert, serialize, export/import, ... UI, dialogs, forms, pages input (validation, defaulting, dependencies) flows (primary paths, alternate paths) keyboard shortcuts, keyboard controls, ... . . . "Cars" Lifecycles, data operations Interaction Functions and Features Technologies, protocols, controls Data (handling, quality, ETL, ...) Security, authorization, admin Graphics, multi-media, charts, ... Interoperability Customizing, extensibility Business Flows Concurrency, race conditions, ... Business Objects processes, transactions, end-to-end, day in the life, combinations of flows, ... calculations, analyses, PDF output, ... "Customers" . . .
  • 23. ©2016 LogiGear Corporationwww.LogiGear.com What about existing tests? • Compare to moving house: – some effort can't be avoided – be selective, edit your stuff, • look at the future, not the past – first decide where to put what, then put it there – moving is an opportunity, you may not get such chance again soon • Follow the module approach – define the modules and their scope as if from scratch – use the existing test cases in two ways: • verify completeness • harvest and re-use them for tests and for actions – avoid porting over "step by step", in particular avoid over-checking
  • 24. ©2016 LogiGear Corporationwww.LogiGear.com Scope of a test module • Always know the scope of the test module • The scope should be unambiguous • The scope determines many things: – what the test objectives are – which test cases to expect – what level of actions to use – what the checks are about and which events should generate a warning or error (if a “lower” functionality is wrong) • In other words: keep your eye on the ball . . .
  • 25. ©2016 LogiGear Corporationwww.LogiGear.com Tip: Use right level actions project task check task in project Drill Assembly Plan of Approach  Better to have a business level action and hide the details in an action definition:  But do show details when it matters. The example below is too high level, requires drill down into action definition to understand what is tested. window tree tree item path click tree item main projects /Projects/Drill Assembly window list item check list item exists main tasks Plan of Approach
  • 26. ©2016 LogiGear Corporationwww.LogiGear.com Stable and Robust Automation • Test your automation – develop separate tests for the actions and interface mappings themselves • Identifying properties of UI elements – a lot of time is spent on identifying and re-identifying UI elements • "active" timing – wait based on conditions in the system under test, avoid fixed “sleeps” – if needed request developers to provide hooks • Testing and automation friendly system designs – with well-defined components, services, tiers, etc • Keep an eye on the test design – unsuccessful automation virtually always can be traced back to poor test design
  • 27. ©2016 LogiGear Corporationwww.LogiGear.com Mapping an interface • An interface mapping (common in test tools) will map windows and controls to names • When the interface of an application changes, you only have to update this in one place • The interface mapping is a key step in your automation success, allocate time to design it well, in particular naming and choosing identifying properties INTERFACE ENTITY library interface entity setting title {.*Music Library} name class label interface element title text Title: interface element artist text Artist: interface element file size text File size (Kb): name class position interface element playing time text textbox 4 interface element file type text textbox 5 interface element bitrate text textbox 6 name class position interface element music treeview treeview 1
  • 29. ©2016 LogiGear Corporationwww.LogiGear.com Mapping the interface (style we use in our product) • An interface mapping will map windows and controls to names • When the interface of an application changes, you only have to update this in one place • The interface mapping is a key step in your automation success, allocate time to design it well
  • 30. ©2016 LogiGear Corporationwww.LogiGear.com • Testability should be understood as a "must have" requirement – first question in a development project: "how do we test this?" • Define (assign) identifying properties for any interface element (control, HTML element, etc) • Have hooks for timing – avoid "hard sleeps" at all costs • Have white-box access to anything interesting: – input data (ability to emulate) – output data (what is underlying data being displayed) – random generators (can I set a seed?) – states (like in a game) – objects displayed (like monsters in a game) – pay attention to security, don't leave hacking doors open • A well-designed system, with tiers, components, services, is also very helpful for test design and automation Testability, some key items
  • 31. ©2016 LogiGear Corporationwww.LogiGear.com Test design and automation in agile High Level Test Design (organization of test modules) Test Module Development Interface Definition Action Automation Test Execution Sprint Products Product Backlog Test re-use Automation re-use product owner team prod owner & team User stories Documentation Domain understanding Acceptance Criteria PO Questions Situations Relations Agile life cycle Test development Main Level Test Modules Interaction Test Modules Cross over Test Modules
  • 32. ©2016 LogiGear Corporationwww.LogiGear.com Pre-conditions for Sprints • Agreement on the approach: – is testability a requirement for the software? • Understanding of subject matter, testing, automation, method, best practices, etc • Automation technology available and working – UI, non-UI, custom controls, graphics, embedded, etc • A plan on how you to deal with data and environments To discuss an approach, consider daily "sit down" meetings with some or all members to coach and evaluate  an end-of-day counterpart to the early-morning "stand up" meetings  short and friendly, not about progress and impediments, but about practices and experiences with them (like "what actions did you use?")  a few meetings may suffice
  • 33. ©2016 LogiGear Corporationwww.LogiGear.com Facilitation outside sprints, example Sprint Have a facility (internal or outsourced) to help the agile life cycle • keep up, catch up, continue beyond "done" • one outsourced facility can support multiple teams, smooth out peaks and valleys • allow sprint teams to focus on development priorities • team is responsible, and decides if, what and when to outsource Sprint Test Definition & Development T E A M Additional Test Development Additional Automation F A C I L I T Y Test Development Automation Automation Test Definition & Development Automation
  • 34. ©2016 LogiGear Corporationwww.LogiGear.com • QA needs to keep up with the rest of the team • Test design is a key driver for automation success/failure • Testability, as a team responsibility, is a core requirement • Organizing the testing well in the sprints gets more of it "done" • Having an agile facilitation model can support the QA's to stay in sync Summary