SlideShare a Scribd company logo
AvoidingTest Hell
Yun Ki Lee
yunki.lee@gmail.com
1
Overview
• WhyTest?
• Sample Application
• BDD and FitNesse
• What would you consider hell to be like?
• How to get into hell
• And how to make it less painful
2
Sample Price Reporting App
• We are a financial service provider
• Loads security prices from a client
• Produces internally calculated security
prices
• Compares the two prices and selects a
price based on a rule
• Produces a valuation report
Client Reporting Module
Client Prices
Price
Comparator Valuations
File
Valuation
Generator
Internal
Price
Calculator
Client Price
Loader
3
Testing Basics
• Why
• To prove what you’ve done works and is what they asked for
• Reduce risk when doing deployment to production
• How
• Manual vs Automated
• Different types of tests, unit tests, component tests, integration tests and so on
• When
• After development is complete? (Waterfall)
• Before you do any development? (TDD)
• Who
• Dedicated team of testers?
• Developers, BAs, users?
4
BDD: Behaviour Driven Development
• Focus on the behaviour
• As a … I want … so that
• Given … when … then …
5
FitNesse
Wiki Fixtures
System
Under
Test
6
Tables
Java Code (System
UnderTest)
How to know whether you’re inTest Hell
• Your tests costs you more than they save you
• How long until you can release the next release into production?
• If you don’t do any manual testing, are you good to go?
• If a test fails, do you believe that it has failed?
• If a test passes, do you trust it?
• How long until a new joiner can get up and running writing tests?
• A function needs to be changed, how long until the tests can be updated?
7
Holding up Releases due toTesting
• How long does it take to arrive at a go / no go decision?
• Manual testing
• Release process not tested and automated
• Solution: automate environment deployments
• Reduce manual steps
• Co-locate services for faster deployments
8
When a test fails
• Did it really fail?
• How long does it take to verify it’s a genuine break?
• Was the failure due to system state?
• Have you got some wait condition so that an action can complete?
• Are you running tests in parallel but have some shared data?
• Solutions:
• Do not hide testing data (next slide)
• Isolate tests that exhibit random behavior (next slide)
• Retry?
9
10
11
Testing Boundaries
PriceComparisonEngine
Thread
Price Loader
extractPrices(source)
Price Rules Persister
loadRules(rule)
comparePrices()
persistPrices()
• Do you want to test the whole engine?
• Or just a couple of features?
12
When a test passes
• Have you got enough coverage to be confident enough to say the
functionality works
• Vertical vs horizontal coverage
• Coverage tools
13
How easy is it to maintain tests?
• Adding new tests
• How easy is it for these people?
• Experienced? New? Haven’t touched the system in a while?
• Modifying tests for new functionality?
• How many tests do they have to update?
• Solution
• Documentation and libraries
• DRY principle
14
Technical Debt inTests
• Packaging of test code isn’t maintained as well
as code
• Less time spent refactoring and reducing
duplication
• Often you will find similar methods that differ
by one or two parameters
• Solution
• Refactor and remove duplicate methods
• Create libraries or APIs that create test objects
15
Mocking
• Simulate behavior of components and
interfaces so that we can isolate that one
component we want to test
• Faster than connecting to a file system or a
database
• Can be brittle and a nightmare to maintain
• Can alleviate some issues if you create a test
library that hides some of the unnecessary
details
PriceComparisonEngine
Thread
Price Loader
extractPrices(source)
Price Rules Persister
loadRules(rule)
comparePrices()
persistPrices()
16
Test Objects
17
Vendor Software and External Data Providers
• Can not control when API and interface
changes
• Complex data sets
• Potentially quite difficult to mock out their
interfaces
• Pitfalls of the following approaches
• Capture and Replay: changes in message
format
• Table dumps: what if table structure changes
due to version upgrade?
• Solution
• Use theVendor API where possible
18
People Issues
• The largest contributor toTest Hell: the people in your team
• What are their motivations?What do they get from doing more frequent releases?
• Why do they want to do anything different?Why replace their manual tasks with automation?
• Structure of your team
• Who is good at what role? Not everyone is the same
• Personality types such as: MBTI
• What if they don’t believe in testing?
• Gamification?
• Who are management focusing their attention on?The poor performers at the cost of good
performers? 19
Things to Google …
• UnitTesting
• BDD
• TDD
• Jmock
• EasyMock
• PowerMock
• Junit
• Continuous Integration
• Object Mothers vsTest Builders
• Test Coverage
• DevOps
• FitNesse
• Concordion
• Cucumber
• JBehave
20

More Related Content

What's hot

Functional Tests. PHP Unconf 2016
Functional Tests. PHP Unconf 2016Functional Tests. PHP Unconf 2016
Functional Tests. PHP Unconf 2016
Vladislav Fedorischev
 
CNUG TDD June 2014
CNUG TDD June 2014CNUG TDD June 2014
CNUG TDD June 2014
Mayank Srivastava
 
TDD Basics with Angular.js and Jasmine
TDD Basics with Angular.js and JasmineTDD Basics with Angular.js and Jasmine
TDD Basics with Angular.js and Jasmine
Luis Sánchez Castellanos
 
Test Driven Development - a Practitioner’s Perspective
Test Driven Development - a Practitioner’s PerspectiveTest Driven Development - a Practitioner’s Perspective
Test Driven Development - a Practitioner’s Perspective
Malinda Kapuruge
 
Approval Tests in Action: A LEGO Exercise and an Experience Report
Approval Tests in Action: A LEGO Exercise and an Experience ReportApproval Tests in Action: A LEGO Exercise and an Experience Report
Approval Tests in Action: A LEGO Exercise and an Experience Report
houseofyin
 
Getting Ahead of Delivery Issues with Deep SDLC Analysis by Donald Belcham
Getting Ahead of Delivery Issues with Deep SDLC Analysis by Donald BelchamGetting Ahead of Delivery Issues with Deep SDLC Analysis by Donald Belcham
Getting Ahead of Delivery Issues with Deep SDLC Analysis by Donald Belcham
.NET Conf UY
 
Benefits from AATs
Benefits from AATsBenefits from AATs
Benefits from AATs
Wyn B. Van Devanter
 
Practical TDD Demonstrated
Practical TDD DemonstratedPractical TDD Demonstrated
Practical TDD Demonstrated
Alan Christensen
 
Iasi code camp 20 april 2013 marian chicu - database unit tests in the sql se...
Iasi code camp 20 april 2013 marian chicu - database unit tests in the sql se...Iasi code camp 20 april 2013 marian chicu - database unit tests in the sql se...
Iasi code camp 20 april 2013 marian chicu - database unit tests in the sql se...
Codecamp Romania
 
Agile Mumbai 2020 Conference | How to get the best ROI on Your Test Automati...
Agile Mumbai 2020 Conference |  How to get the best ROI on Your Test Automati...Agile Mumbai 2020 Conference |  How to get the best ROI on Your Test Automati...
Agile Mumbai 2020 Conference | How to get the best ROI on Your Test Automati...
AgileNetwork
 
Jonathon Wright - Intelligent Performance Cognitive Learning (AIOps)
Jonathon Wright - Intelligent Performance Cognitive Learning (AIOps)Jonathon Wright - Intelligent Performance Cognitive Learning (AIOps)
Jonathon Wright - Intelligent Performance Cognitive Learning (AIOps)
Neotys_Partner
 
#ESPC18 How to do #devops with the #SharePoint Framework and why it matters?
#ESPC18 How to do #devops with the #SharePoint Framework and why it matters?#ESPC18 How to do #devops with the #SharePoint Framework and why it matters?
#ESPC18 How to do #devops with the #SharePoint Framework and why it matters?
Vincent Biret
 
Unit tests benefits
Unit tests benefitsUnit tests benefits
Unit tests benefits
Kate Semizhon
 
#1 unit testing
#1 unit testing#1 unit testing
#1 unit testing
eleksdev
 
Addressing Security Regression Through Unit Testing
Addressing Security Regression Through Unit TestingAddressing Security Regression Through Unit Testing
Addressing Security Regression Through Unit Testing
Christopher Grayson
 
An Introduction to Unit Testing
An Introduction to Unit TestingAn Introduction to Unit Testing
An Introduction to Unit Testing
Sahar Nofal
 
Testing regression
Testing regressionTesting regression
Testing regression
Richie Lee
 
2020-Feb: Testing: Cables and Chains
2020-Feb: Testing: Cables and Chains2020-Feb: Testing: Cables and Chains
2020-Feb: Testing: Cables and Chains
Mark Windholtz
 
Grant Fritchey Justin Caldicott - Best practices for database deployments
Grant Fritchey Justin Caldicott - Best practices for database deploymentsGrant Fritchey Justin Caldicott - Best practices for database deployments
Grant Fritchey Justin Caldicott - Best practices for database deployments
Red Gate Software
 
Performance Tuning in the Trenches
Performance Tuning in the TrenchesPerformance Tuning in the Trenches
Performance Tuning in the Trenches
Donald Belcham
 

What's hot (20)

Functional Tests. PHP Unconf 2016
Functional Tests. PHP Unconf 2016Functional Tests. PHP Unconf 2016
Functional Tests. PHP Unconf 2016
 
CNUG TDD June 2014
CNUG TDD June 2014CNUG TDD June 2014
CNUG TDD June 2014
 
TDD Basics with Angular.js and Jasmine
TDD Basics with Angular.js and JasmineTDD Basics with Angular.js and Jasmine
TDD Basics with Angular.js and Jasmine
 
Test Driven Development - a Practitioner’s Perspective
Test Driven Development - a Practitioner’s PerspectiveTest Driven Development - a Practitioner’s Perspective
Test Driven Development - a Practitioner’s Perspective
 
Approval Tests in Action: A LEGO Exercise and an Experience Report
Approval Tests in Action: A LEGO Exercise and an Experience ReportApproval Tests in Action: A LEGO Exercise and an Experience Report
Approval Tests in Action: A LEGO Exercise and an Experience Report
 
Getting Ahead of Delivery Issues with Deep SDLC Analysis by Donald Belcham
Getting Ahead of Delivery Issues with Deep SDLC Analysis by Donald BelchamGetting Ahead of Delivery Issues with Deep SDLC Analysis by Donald Belcham
Getting Ahead of Delivery Issues with Deep SDLC Analysis by Donald Belcham
 
Benefits from AATs
Benefits from AATsBenefits from AATs
Benefits from AATs
 
Practical TDD Demonstrated
Practical TDD DemonstratedPractical TDD Demonstrated
Practical TDD Demonstrated
 
Iasi code camp 20 april 2013 marian chicu - database unit tests in the sql se...
Iasi code camp 20 april 2013 marian chicu - database unit tests in the sql se...Iasi code camp 20 april 2013 marian chicu - database unit tests in the sql se...
Iasi code camp 20 april 2013 marian chicu - database unit tests in the sql se...
 
Agile Mumbai 2020 Conference | How to get the best ROI on Your Test Automati...
Agile Mumbai 2020 Conference |  How to get the best ROI on Your Test Automati...Agile Mumbai 2020 Conference |  How to get the best ROI on Your Test Automati...
Agile Mumbai 2020 Conference | How to get the best ROI on Your Test Automati...
 
Jonathon Wright - Intelligent Performance Cognitive Learning (AIOps)
Jonathon Wright - Intelligent Performance Cognitive Learning (AIOps)Jonathon Wright - Intelligent Performance Cognitive Learning (AIOps)
Jonathon Wright - Intelligent Performance Cognitive Learning (AIOps)
 
#ESPC18 How to do #devops with the #SharePoint Framework and why it matters?
#ESPC18 How to do #devops with the #SharePoint Framework and why it matters?#ESPC18 How to do #devops with the #SharePoint Framework and why it matters?
#ESPC18 How to do #devops with the #SharePoint Framework and why it matters?
 
Unit tests benefits
Unit tests benefitsUnit tests benefits
Unit tests benefits
 
#1 unit testing
#1 unit testing#1 unit testing
#1 unit testing
 
Addressing Security Regression Through Unit Testing
Addressing Security Regression Through Unit TestingAddressing Security Regression Through Unit Testing
Addressing Security Regression Through Unit Testing
 
An Introduction to Unit Testing
An Introduction to Unit TestingAn Introduction to Unit Testing
An Introduction to Unit Testing
 
Testing regression
Testing regressionTesting regression
Testing regression
 
2020-Feb: Testing: Cables and Chains
2020-Feb: Testing: Cables and Chains2020-Feb: Testing: Cables and Chains
2020-Feb: Testing: Cables and Chains
 
Grant Fritchey Justin Caldicott - Best practices for database deployments
Grant Fritchey Justin Caldicott - Best practices for database deploymentsGrant Fritchey Justin Caldicott - Best practices for database deployments
Grant Fritchey Justin Caldicott - Best practices for database deployments
 
Performance Tuning in the Trenches
Performance Tuning in the TrenchesPerformance Tuning in the Trenches
Performance Tuning in the Trenches
 

Viewers also liked

Análisis DAFO de una experiencia de fomento de competencias STEM
Análisis DAFO de una experiencia de fomento de competencias STEMAnálisis DAFO de una experiencia de fomento de competencias STEM
Análisis DAFO de una experiencia de fomento de competencias STEM
Aitor Lizarazu
 
Drug dispensing contact lenses a reality
Drug dispensing contact lenses a reality Drug dispensing contact lenses a reality
Drug dispensing contact lenses a reality
Dominick Maino
 
Uso educativo del twitter
Uso educativo del twitterUso educativo del twitter
Uso educativo del twitter
Miren Iriarte Arteaga
 
CV TemplateREV [1]
CV TemplateREV [1]CV TemplateREV [1]
CV TemplateREV [1]
Brian Roberts
 
Mitarbeit an Relaunches
Mitarbeit an RelaunchesMitarbeit an Relaunches
Mitarbeit an Relaunches
Thomas Matterne
 
Hydrogels and silicone hydrogels
Hydrogels and silicone hydrogelsHydrogels and silicone hydrogels
Hydrogels and silicone hydrogels
CHURAWAN LAL SAHU
 
Scientific measurement protocols 1
Scientific measurement protocols 1Scientific measurement protocols 1
Scientific measurement protocols 1
CHURAWAN LAL SAHU
 
Sports injuries & its prevention
Sports injuries & its preventionSports injuries & its prevention
Sports injuries & its prevention
akash mitra
 
REYLEIGH’S METHOD,BUCKINGHAM π-THEOREM
REYLEIGH’S METHOD,BUCKINGHAM  π-THEOREMREYLEIGH’S METHOD,BUCKINGHAM  π-THEOREM
REYLEIGH’S METHOD,BUCKINGHAM π-THEOREM
Amiraj College Of Engineering And Technology
 
Factors behind emergence of resistance
Factors behind emergence of resistanceFactors behind emergence of resistance
Factors behind emergence of resistance
katefranklyn
 
Why it is important
Why it is importantWhy it is important
Why it is important
MikylaJRata
 
printing press startup
printing press startupprinting press startup
printing press startup
manimp manimp
 
0 th review
0 th review0 th review
0 th review
Naga Pradeep Deepu
 
Orlando Agrippa, Draper & Dash at "Journeys of Health-Tech Innovation" Nov 3...
Orlando Agrippa, Draper & Dash  at "Journeys of Health-Tech Innovation" Nov 3...Orlando Agrippa, Draper & Dash  at "Journeys of Health-Tech Innovation" Nov 3...
Orlando Agrippa, Draper & Dash at "Journeys of Health-Tech Innovation" Nov 3...
Health-Tech Innovation LABS
 
Cambios de escenarios, nuevos repartos y guiones
Cambios de escenarios, nuevos repartos y guionesCambios de escenarios, nuevos repartos y guiones
Cambios de escenarios, nuevos repartos y guiones
Centro Internacional de Tecnologías Avanzadas. FGSR
 

Viewers also liked (15)

Análisis DAFO de una experiencia de fomento de competencias STEM
Análisis DAFO de una experiencia de fomento de competencias STEMAnálisis DAFO de una experiencia de fomento de competencias STEM
Análisis DAFO de una experiencia de fomento de competencias STEM
 
Drug dispensing contact lenses a reality
Drug dispensing contact lenses a reality Drug dispensing contact lenses a reality
Drug dispensing contact lenses a reality
 
Uso educativo del twitter
Uso educativo del twitterUso educativo del twitter
Uso educativo del twitter
 
CV TemplateREV [1]
CV TemplateREV [1]CV TemplateREV [1]
CV TemplateREV [1]
 
Mitarbeit an Relaunches
Mitarbeit an RelaunchesMitarbeit an Relaunches
Mitarbeit an Relaunches
 
Hydrogels and silicone hydrogels
Hydrogels and silicone hydrogelsHydrogels and silicone hydrogels
Hydrogels and silicone hydrogels
 
Scientific measurement protocols 1
Scientific measurement protocols 1Scientific measurement protocols 1
Scientific measurement protocols 1
 
Sports injuries & its prevention
Sports injuries & its preventionSports injuries & its prevention
Sports injuries & its prevention
 
REYLEIGH’S METHOD,BUCKINGHAM π-THEOREM
REYLEIGH’S METHOD,BUCKINGHAM  π-THEOREMREYLEIGH’S METHOD,BUCKINGHAM  π-THEOREM
REYLEIGH’S METHOD,BUCKINGHAM π-THEOREM
 
Factors behind emergence of resistance
Factors behind emergence of resistanceFactors behind emergence of resistance
Factors behind emergence of resistance
 
Why it is important
Why it is importantWhy it is important
Why it is important
 
printing press startup
printing press startupprinting press startup
printing press startup
 
0 th review
0 th review0 th review
0 th review
 
Orlando Agrippa, Draper & Dash at "Journeys of Health-Tech Innovation" Nov 3...
Orlando Agrippa, Draper & Dash  at "Journeys of Health-Tech Innovation" Nov 3...Orlando Agrippa, Draper & Dash  at "Journeys of Health-Tech Innovation" Nov 3...
Orlando Agrippa, Draper & Dash at "Journeys of Health-Tech Innovation" Nov 3...
 
Cambios de escenarios, nuevos repartos y guiones
Cambios de escenarios, nuevos repartos y guionesCambios de escenarios, nuevos repartos y guiones
Cambios de escenarios, nuevos repartos y guiones
 

Similar to Avoiding test hell

DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
Andrew Williams
 
Bugday bkk-2014 nitisak-auto_perf
Bugday bkk-2014 nitisak-auto_perfBugday bkk-2014 nitisak-auto_perf
Bugday bkk-2014 nitisak-auto_perf
Nitisak Mooltreesri
 
Cloud-based Test Microservices JavaOne 2014
Cloud-based Test Microservices JavaOne 2014Cloud-based Test Microservices JavaOne 2014
Cloud-based Test Microservices JavaOne 2014
Shelley Lambert
 
Chapter 13 software testing strategies
Chapter 13 software testing strategiesChapter 13 software testing strategies
Chapter 13 software testing strategies
SHREEHARI WADAWADAGI
 
Testing for Logic App Solutions | Integration Monday
Testing for Logic App Solutions | Integration MondayTesting for Logic App Solutions | Integration Monday
Testing for Logic App Solutions | Integration Monday
BizTalk360
 
Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Into...
Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Into...Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Into...
Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Into...
Ortus Solutions, Corp
 
Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Adob...
Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Adob...Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Adob...
Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Adob...
Uma Ghotikar
 
5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation
Sauce Labs
 
An Introduction To Software Development - Final Review
An Introduction To Software Development - Final ReviewAn Introduction To Software Development - Final Review
An Introduction To Software Development - Final Review
Blue Elephant Consulting
 
Tools for Software Testing
Tools for Software TestingTools for Software Testing
Tools for Software Testing
Mohammed Moishin
 
July webinar l How to Handle the Holiday Retail Rush with Agile Performance T...
July webinar l How to Handle the Holiday Retail Rush with Agile Performance T...July webinar l How to Handle the Holiday Retail Rush with Agile Performance T...
July webinar l How to Handle the Holiday Retail Rush with Agile Performance T...
Apica
 
QA Team Goes to Agile and Continuous integration
QA Team Goes to Agile and Continuous integrationQA Team Goes to Agile and Continuous integration
QA Team Goes to Agile and Continuous integration
Sujit Ghosh
 
Testing - How Vital and How Easy to use
Testing - How Vital and How Easy to useTesting - How Vital and How Easy to use
Testing - How Vital and How Easy to use
Uma Ghotikar
 
Agile Software Testing the Agilogy Way
Agile Software Testing the Agilogy WayAgile Software Testing the Agilogy Way
Agile Software Testing the Agilogy Way
Jordi Pradel
 
Load testing with Visual Studio and Azure - Andrew Siemer
Load testing with Visual Studio and Azure - Andrew SiemerLoad testing with Visual Studio and Azure - Andrew Siemer
Load testing with Visual Studio and Azure - Andrew Siemer
Andrew Siemer
 
InteropWG Intro & Vertical Programs (May. 2017)
InteropWG Intro & Vertical Programs (May. 2017)InteropWG Intro & Vertical Programs (May. 2017)
InteropWG Intro & Vertical Programs (May. 2017)
Mark Voelker
 
DBmaestro's State of the Database Continuous Delivery Survey- Findings Revealed
DBmaestro's State of the Database Continuous Delivery Survey- Findings RevealedDBmaestro's State of the Database Continuous Delivery Survey- Findings Revealed
DBmaestro's State of the Database Continuous Delivery Survey- Findings Revealed
DBmaestro - Database DevOps
 
Testing Angular
Testing AngularTesting Angular
Testing Angular
Lilia Sfaxi
 
Agile Acceptance testing with Fitnesse
Agile Acceptance testing with FitnesseAgile Acceptance testing with Fitnesse
Agile Acceptance testing with Fitnesse
ClareMcLennan
 
Alexander Podelko - Context-Driven Performance Testing
Alexander Podelko - Context-Driven Performance TestingAlexander Podelko - Context-Driven Performance Testing
Alexander Podelko - Context-Driven Performance Testing
Neotys_Partner
 

Similar to Avoiding test hell (20)

DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
 
Bugday bkk-2014 nitisak-auto_perf
Bugday bkk-2014 nitisak-auto_perfBugday bkk-2014 nitisak-auto_perf
Bugday bkk-2014 nitisak-auto_perf
 
Cloud-based Test Microservices JavaOne 2014
Cloud-based Test Microservices JavaOne 2014Cloud-based Test Microservices JavaOne 2014
Cloud-based Test Microservices JavaOne 2014
 
Chapter 13 software testing strategies
Chapter 13 software testing strategiesChapter 13 software testing strategies
Chapter 13 software testing strategies
 
Testing for Logic App Solutions | Integration Monday
Testing for Logic App Solutions | Integration MondayTesting for Logic App Solutions | Integration Monday
Testing for Logic App Solutions | Integration Monday
 
Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Into...
Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Into...Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Into...
Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Into...
 
Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Adob...
Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Adob...Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Adob...
Introduction to Unit Testing, BDD and Mocking using TestBox & MockBox at Adob...
 
5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation
 
An Introduction To Software Development - Final Review
An Introduction To Software Development - Final ReviewAn Introduction To Software Development - Final Review
An Introduction To Software Development - Final Review
 
Tools for Software Testing
Tools for Software TestingTools for Software Testing
Tools for Software Testing
 
July webinar l How to Handle the Holiday Retail Rush with Agile Performance T...
July webinar l How to Handle the Holiday Retail Rush with Agile Performance T...July webinar l How to Handle the Holiday Retail Rush with Agile Performance T...
July webinar l How to Handle the Holiday Retail Rush with Agile Performance T...
 
QA Team Goes to Agile and Continuous integration
QA Team Goes to Agile and Continuous integrationQA Team Goes to Agile and Continuous integration
QA Team Goes to Agile and Continuous integration
 
Testing - How Vital and How Easy to use
Testing - How Vital and How Easy to useTesting - How Vital and How Easy to use
Testing - How Vital and How Easy to use
 
Agile Software Testing the Agilogy Way
Agile Software Testing the Agilogy WayAgile Software Testing the Agilogy Way
Agile Software Testing the Agilogy Way
 
Load testing with Visual Studio and Azure - Andrew Siemer
Load testing with Visual Studio and Azure - Andrew SiemerLoad testing with Visual Studio and Azure - Andrew Siemer
Load testing with Visual Studio and Azure - Andrew Siemer
 
InteropWG Intro & Vertical Programs (May. 2017)
InteropWG Intro & Vertical Programs (May. 2017)InteropWG Intro & Vertical Programs (May. 2017)
InteropWG Intro & Vertical Programs (May. 2017)
 
DBmaestro's State of the Database Continuous Delivery Survey- Findings Revealed
DBmaestro's State of the Database Continuous Delivery Survey- Findings RevealedDBmaestro's State of the Database Continuous Delivery Survey- Findings Revealed
DBmaestro's State of the Database Continuous Delivery Survey- Findings Revealed
 
Testing Angular
Testing AngularTesting Angular
Testing Angular
 
Agile Acceptance testing with Fitnesse
Agile Acceptance testing with FitnesseAgile Acceptance testing with Fitnesse
Agile Acceptance testing with Fitnesse
 
Alexander Podelko - Context-Driven Performance Testing
Alexander Podelko - Context-Driven Performance TestingAlexander Podelko - Context-Driven Performance Testing
Alexander Podelko - Context-Driven Performance Testing
 

Recently uploaded

Predicting Test Results without Execution (FSE 2024)
Predicting Test Results without Execution (FSE 2024)Predicting Test Results without Execution (FSE 2024)
Predicting Test Results without Execution (FSE 2024)
andrehoraa
 
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
Srinivas Dukka
 
ERP Software Solutions Provider in Coimbatore
ERP Software Solutions Provider in CoimbatoreERP Software Solutions Provider in Coimbatore
ERP Software Solutions Provider in Coimbatore
Nextskill Technologies
 
Tour and travel website management in odoo,
Tour and travel website management in odoo,Tour and travel website management in odoo,
Tour and travel website management in odoo,
Axis Technolabs
 
Empowering Businesses with Intelligent Software Solutions - Grawlix
Empowering Businesses with Intelligent Software Solutions - GrawlixEmpowering Businesses with Intelligent Software Solutions - Grawlix
Empowering Businesses with Intelligent Software Solutions - Grawlix
Aarisha Shaikh
 
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
45unexpected
 
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdfA Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
kalichargn70th171
 
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
singhlata50dh
 
Crafting highly scalable and performant Modern Data Platforms
Crafting highly scalable and performant Modern Data PlatformsCrafting highly scalable and performant Modern Data Platforms
Crafting highly scalable and performant Modern Data Platforms
Sameer Paradkar
 
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
rachitkumar09887
 
Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...
Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...
Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...
OnePlan Solutions
 
Authentication Review-June -2024 AP & TS.pptx
Authentication Review-June -2024 AP & TS.pptxAuthentication Review-June -2024 AP & TS.pptx
Authentication Review-June -2024 AP & TS.pptx
DEMONDUOS
 
Il Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazioneIl Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazione
confluent
 
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
3610stuck
 
To Avoid Mistakes When Using Online Attendance Sheets
To Avoid Mistakes When Using Online Attendance SheetsTo Avoid Mistakes When Using Online Attendance Sheets
To Avoid Mistakes When Using Online Attendance Sheets
Task Tracker
 
Amazon Music Spelling Correction - SIGIR 2024
Amazon Music Spelling Correction - SIGIR 2024Amazon Music Spelling Correction - SIGIR 2024
Amazon Music Spelling Correction - SIGIR 2024
siddharth1729
 
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
dream girl
 
Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...
Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...
Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...
bhumivarma35300
 
Fantastic Design Patterns and Where to use them No Notes.pdf
Fantastic Design Patterns and Where to use them No Notes.pdfFantastic Design Patterns and Where to use them No Notes.pdf
Fantastic Design Patterns and Where to use them No Notes.pdf
6m9p7qnjj8
 
How To Fill Timesheet in TaskSprint: Quick Guide 2024
How To Fill Timesheet in TaskSprint: Quick Guide 2024How To Fill Timesheet in TaskSprint: Quick Guide 2024
How To Fill Timesheet in TaskSprint: Quick Guide 2024
TaskSprint | Employee Efficiency Software
 

Recently uploaded (20)

Predicting Test Results without Execution (FSE 2024)
Predicting Test Results without Execution (FSE 2024)Predicting Test Results without Execution (FSE 2024)
Predicting Test Results without Execution (FSE 2024)
 
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
 
ERP Software Solutions Provider in Coimbatore
ERP Software Solutions Provider in CoimbatoreERP Software Solutions Provider in Coimbatore
ERP Software Solutions Provider in Coimbatore
 
Tour and travel website management in odoo,
Tour and travel website management in odoo,Tour and travel website management in odoo,
Tour and travel website management in odoo,
 
Empowering Businesses with Intelligent Software Solutions - Grawlix
Empowering Businesses with Intelligent Software Solutions - GrawlixEmpowering Businesses with Intelligent Software Solutions - Grawlix
Empowering Businesses with Intelligent Software Solutions - Grawlix
 
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
 
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdfA Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
 
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
 
Crafting highly scalable and performant Modern Data Platforms
Crafting highly scalable and performant Modern Data PlatformsCrafting highly scalable and performant Modern Data Platforms
Crafting highly scalable and performant Modern Data Platforms
 
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
 
Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...
Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...
Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...
 
Authentication Review-June -2024 AP & TS.pptx
Authentication Review-June -2024 AP & TS.pptxAuthentication Review-June -2024 AP & TS.pptx
Authentication Review-June -2024 AP & TS.pptx
 
Il Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazioneIl Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazione
 
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
 
To Avoid Mistakes When Using Online Attendance Sheets
To Avoid Mistakes When Using Online Attendance SheetsTo Avoid Mistakes When Using Online Attendance Sheets
To Avoid Mistakes When Using Online Attendance Sheets
 
Amazon Music Spelling Correction - SIGIR 2024
Amazon Music Spelling Correction - SIGIR 2024Amazon Music Spelling Correction - SIGIR 2024
Amazon Music Spelling Correction - SIGIR 2024
 
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
 
Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...
Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...
Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...
 
Fantastic Design Patterns and Where to use them No Notes.pdf
Fantastic Design Patterns and Where to use them No Notes.pdfFantastic Design Patterns and Where to use them No Notes.pdf
Fantastic Design Patterns and Where to use them No Notes.pdf
 
How To Fill Timesheet in TaskSprint: Quick Guide 2024
How To Fill Timesheet in TaskSprint: Quick Guide 2024How To Fill Timesheet in TaskSprint: Quick Guide 2024
How To Fill Timesheet in TaskSprint: Quick Guide 2024
 

Avoiding test hell

  • 1. AvoidingTest Hell Yun Ki Lee yunki.lee@gmail.com 1
  • 2. Overview • WhyTest? • Sample Application • BDD and FitNesse • What would you consider hell to be like? • How to get into hell • And how to make it less painful 2
  • 3. Sample Price Reporting App • We are a financial service provider • Loads security prices from a client • Produces internally calculated security prices • Compares the two prices and selects a price based on a rule • Produces a valuation report Client Reporting Module Client Prices Price Comparator Valuations File Valuation Generator Internal Price Calculator Client Price Loader 3
  • 4. Testing Basics • Why • To prove what you’ve done works and is what they asked for • Reduce risk when doing deployment to production • How • Manual vs Automated • Different types of tests, unit tests, component tests, integration tests and so on • When • After development is complete? (Waterfall) • Before you do any development? (TDD) • Who • Dedicated team of testers? • Developers, BAs, users? 4
  • 5. BDD: Behaviour Driven Development • Focus on the behaviour • As a … I want … so that • Given … when … then … 5
  • 7. How to know whether you’re inTest Hell • Your tests costs you more than they save you • How long until you can release the next release into production? • If you don’t do any manual testing, are you good to go? • If a test fails, do you believe that it has failed? • If a test passes, do you trust it? • How long until a new joiner can get up and running writing tests? • A function needs to be changed, how long until the tests can be updated? 7
  • 8. Holding up Releases due toTesting • How long does it take to arrive at a go / no go decision? • Manual testing • Release process not tested and automated • Solution: automate environment deployments • Reduce manual steps • Co-locate services for faster deployments 8
  • 9. When a test fails • Did it really fail? • How long does it take to verify it’s a genuine break? • Was the failure due to system state? • Have you got some wait condition so that an action can complete? • Are you running tests in parallel but have some shared data? • Solutions: • Do not hide testing data (next slide) • Isolate tests that exhibit random behavior (next slide) • Retry? 9
  • 10. 10
  • 11. 11
  • 12. Testing Boundaries PriceComparisonEngine Thread Price Loader extractPrices(source) Price Rules Persister loadRules(rule) comparePrices() persistPrices() • Do you want to test the whole engine? • Or just a couple of features? 12
  • 13. When a test passes • Have you got enough coverage to be confident enough to say the functionality works • Vertical vs horizontal coverage • Coverage tools 13
  • 14. How easy is it to maintain tests? • Adding new tests • How easy is it for these people? • Experienced? New? Haven’t touched the system in a while? • Modifying tests for new functionality? • How many tests do they have to update? • Solution • Documentation and libraries • DRY principle 14
  • 15. Technical Debt inTests • Packaging of test code isn’t maintained as well as code • Less time spent refactoring and reducing duplication • Often you will find similar methods that differ by one or two parameters • Solution • Refactor and remove duplicate methods • Create libraries or APIs that create test objects 15
  • 16. Mocking • Simulate behavior of components and interfaces so that we can isolate that one component we want to test • Faster than connecting to a file system or a database • Can be brittle and a nightmare to maintain • Can alleviate some issues if you create a test library that hides some of the unnecessary details PriceComparisonEngine Thread Price Loader extractPrices(source) Price Rules Persister loadRules(rule) comparePrices() persistPrices() 16
  • 18. Vendor Software and External Data Providers • Can not control when API and interface changes • Complex data sets • Potentially quite difficult to mock out their interfaces • Pitfalls of the following approaches • Capture and Replay: changes in message format • Table dumps: what if table structure changes due to version upgrade? • Solution • Use theVendor API where possible 18
  • 19. People Issues • The largest contributor toTest Hell: the people in your team • What are their motivations?What do they get from doing more frequent releases? • Why do they want to do anything different?Why replace their manual tasks with automation? • Structure of your team • Who is good at what role? Not everyone is the same • Personality types such as: MBTI • What if they don’t believe in testing? • Gamification? • Who are management focusing their attention on?The poor performers at the cost of good performers? 19
  • 20. Things to Google … • UnitTesting • BDD • TDD • Jmock • EasyMock • PowerMock • Junit • Continuous Integration • Object Mothers vsTest Builders • Test Coverage • DevOps • FitNesse • Concordion • Cucumber • JBehave 20

Editor's Notes

  1. I don’t have a silver bullet for everything The advice I am to give is more practical rather than theoretical. Its easy to talk about ideal world solutions but unfortunately, the projects I worked on weren’t perfect. We had things called deadlines and constraints on budgets which meant we had to make compromises.
  2. A Junit could be written as what you see here. The developer probably only had a design document on how the class should behave and what methods they had to write. So then they would write a test like that. So what would happen if this junit would fail? What would you know if you looked at a junit report that said SelectPrice failed? Problems – if the test failed, how do you have to spend time investigating what particular functionality failed. You would have to go into the test and see what it really tested. Whether you absolutely have to fix it depends on which requirement it reflects. How its not so easy to determine which requirement it is for. The wording does not reflect the requirements or the desired behaviour of the system So BDD attempts to address this focusing on the behaviour of the system. We went from having requirements the system will do something to something like As a user, I want to be able to find out the highest price for a security so that I charge the most for my portfolio. The tests then we organised in the given some precondition, when some action is called, then I expect something to happen and the method name is something a bit more meaningful so that if you looked at a junit report, when it fails, I know have a better indicator what failed.
  3. FitNesse is an Acceptance Testing Framework with a wiki as the front end, some fixtures or glue code that invokes your system / functions or what we call the System Under Test. There are a couple of other good testing frameworks such as Cucumber, Jbehave, Concordion and so on. We choose FitNesse because it was the easiest for our BAs to use. But I have to admit, some of the other tools might be closer to BDD than FitNesse. Given some prices, when we compare prices and generate the report, then we expect no differences with a predefined report.
  4. Your tests are there to save you money, but they aren’t free. Tests come at a cost. If the cost is greater than what they save you, are they worth it? It costs you money to write tests, maintain them, debug and fix them. Even looking at them costs you money. What if your client whats a piece of functionality in production next week? Can you do it? What about tomorrow? What can you confidently say you’ve tested everything? Are your tests holding you back? If you have a test that fails, do you believe it has failed immediately or do you rerun it first? That rerun is wasted effort. If a test passes, do you believe that functionality works or do you have a poke around at the GUI first? Then there is the people side of things. You ask someone to investigate a test failure, do they groan? Can a new joiner into the team start investigating and writing their own tests relatively quickly?
  5. Is it cheaper to fix the problem before or after its released into production? What if you hold up production? Costs of breaking contractual agreements. You should also ask why didn’t your tests pick up the problem earlier. If a bug is found so late in the cycle, will it costs more to fix before releasing to production due to costs? Time to fix it? How long to retest? If you have not got a release process clearly defined, then it will take you a while to release from DEV to SIT and then UAT. Are the creation of your release packages for different test environments standardized? You need to ensure that what you deploy to each environment, from your build servers to SIT and UAT is the same. So you need to ensure everything is scripted and theres no manual copying of files anywhere. There is a whole lot of things you can do around DevOps to help you here but is beyond the topic today. The project I’m on used to take 2 to 4 hours to deploy to an environment. Anything could go wrong, locking out accounts to copying the wrong set of files and manual copying and unzipping files. We were copying files across continents – from UK to Hong Kong. Now we have it all automated to deploy to 2 Linux servers and 4 Window servers across Hong Kong in under 5 minutes.
  6. One indicator that your tests are costing you is what do you after they’ve failed. Do you trust your reporting well enough to verify it’s a real break? If it is a real break, how long does it take you to get started to look into it? Are there sufficient debug statements and logs? We will see some more examples of things that take up time investigating broken tests And then, was the test broken because of some environment issue? Is it connecting to a database or MQ which happened to be down? Then the test is just creating noise as you don’t know whether the actual function being tested broken or not. A lot of computation heavy methods take time. Did you introduce a wait or delay into your test? What if one day, the environment is a little bit slower than normal? Most of the time, you don’t plan for tests to be ran in parallel. Whoever wrote the first couple of Junits in the system just wrote tests. They probably didn’t think about running tests in parallel or other performance issues. So its possible that some data might be reused across tests. If you’re running tests in parallel, would you have the problem of over writing each others data?
  7. So now lets look at examples of what I just talked about. From the test, how long does it take for you to just look at where the failures are? If its hidden in a file, you have to go the file system and check the reports. In Fitnesse, you can reuse pages so you can have a page with just the test data so that if the data needs to change, you can just change that one test page.
  8. Discuss the use of valuation_date and the wait for 1 minute What if other pages also use 1st march? How are you going to run them in parallel? Are you going to setup VMs? Separate folders per process? The simplest thing you could at least do is agree with others on what date you will use so that you don’t over lap. Changes are, that’s going to be a real quick way to solve some of your issues. The wait for 1 minute, what if the compare prices takes longer than 1 minute? What are things you can try? 90 seconds? 2 minutes? What if the operation really did fail? Then you have to wait 2 minutes. What if you poll? Say 3 seconds for 30 rounds? If it passes, then you’ve sped up the test. But what if it fails? Then you are still waiting a long time. Can you change to event driven architecture? That is most ideal solution, but probably not feasible.
  9. Continuing from the previous example, most of the time, large processes and engines are often put into threads which makes it asynchronous. How many times do you need to test the actual engine or can you extract the logic inside the thread. In the ideal situation, you will have unit tests covering each function and perhaps each class and then a small number of FitNesse tests that represent integration tests.
  10. How well do you trust the coverage %? What does it mean? If you only cover unit tests, then you are missing out on the integration. Still need ways to unsure that testing covers everything? Can have Excel or tools like QC that people put in test cases. If an automatic test suite passes, is that sufficient or do you still have a whole lot of manual testing to go?
  11. If this is a problem, then you have a problem. Is it easy for people to add new tests? Often the first thing most people do is look for a working example and then copy and paste it and then change the values they need. But how do you know whether they are copying a test that is up to date with the methods being called? What usually happens is that people write some tests that passes. Then some people add some new utilities or things that make the tests work better but the old tests don’t get updated. You need to come up with a system that ensures people are copying the most up to date tests. I have not got a silver bullet for this issue. In an ideal world, all tests are up to date. But in reality, we have code reviews and owners for test suites who know which test are up to date. Not only do we need to keep the code up to date, the documentation needs to be there and kept up to date. I always follow the DRY – Don’t Repeat Yourself principle. Ideally everyone will remove duplication but in reality, it doesn’t always happen. You always build up on debt.
  12. The solution is easy – invest time to clean it up. Lets say you package the structure a little better and created a report package and put the report related classes inside. Then inside the documentation for those classes, you don’t need to explicitly say those are reporting classes. Simply by being in the package, you know it’s a reporting related class. You’ve made life easier for yourself. And then same if you refactor methods as well. You’ve reduced the number of things that need documentation.
  13. Mocking allows you to simulate behaviour. Rarely does your class work alone (unless it’s a really bad class) or it’s a toy. But when you are testing just one class or component, you really need to be able to simulate it communicating with other classes and interfaces. You don’t want to connect to the database or file system everytime you test because its slow and you are introducing more variables into the test environment. And you also need to be able to repeat the behaviour over and over again and make sure everytime the inputs into the test are the same. But Mocking itself is not the answer to everything. Frameworks can easily be abused and you will end up with a mess. Often testing frameworks require you to mock out each method call and potentially there could be nested mock objects.
  14. First example – create the objects required for testing inside the test. You can see it can get quite complicated, what if your objects have dependencies you aren’t testing? Your tests can get quite long, full of code that isn’t actually needed for the test but required for the setup Second example – an object mother – huge improvement by using factories. Hides all the details. But if the test fails, you have to dig into the object mother to see what the object looks like (wasting time on figuring out what is wrong). What if you have a slightly different object? Then you need a new method Third example – test object builders. The test is more verbose but more elegant. You can hide dependencies that you don’t care about inside the .with() methods or upon initialisation of the builder object, set a whole lot of default fields.
  15. (skip this slide if not enough time) Chances are the vendor software is doing something complex, if it was simple, you may not have purchased it. And therefore it will have complex data requirements and messages flowing in and out of the system. If you are programming in a language that compiles like Java, then using the Vendor’s API will give you some clue that changed in a version upgrade.
  16. Everything I have talked about so far are a symptom, not the cause of why you are having trouble with your tests. The root cause in my opinion is the people. All the software packages like FitNesse and Mocking and theory like test boundaries I have talked about ultimately get used by people. And it is how your people choose to use these things that ultimately get you into a mess. What if your people don’t believe in testing? Have they seen the benefits? Or are they just feeling the pain of cleaning up a messy test suite? If the benefit is all at the management level, why would you expect at the individual contributors to fully embrace testing? Gamification – could back fire if moral is low. Example – 2 teams, manager publishes some statistics that he wants to measure and introduces a little competition against each other. But if moral is low, or the incentives are low, then no one cares. Everyone will want to beat the game. Eg ratio of emergency releases into production to releases is a low percentage. I don’t have a solution for people problems – but are you even aware of it?