cbell@CamilleBellConsulting.com 1
Testing for Agility:
Bringing Testing into Everything
Camille Bell
Agile Coach & Trainer...
cbell@CamilleBellConsulting.com 2
About Me
•  Agile Coach, Trainer and Developer
•  Over 4 decades of IT. All sorts of com...
cbell@CamilleBellConsulting.com 3
• Waiting to test
• Transforming Waterfall into Agility using Feedback
• Agile testing a...
cbell@CamilleBellConsulting.com 4
The Trouble
with
Waiting to Test
cbell@CamilleBellConsulting.com 5
Agility puts more focus on
testing to improve quality
and time to market
Which is great ...
cbell@CamilleBellConsulting.com 6
But many shops bolt on Waterfall
phased testing to Agile development
While still better ...
cbell@CamilleBellConsulting.com 7
Release Testing vs. Continuous Testing
Ref: ”Lean from the Trenches” by Henrick Kniberg
cbell@CamilleBellConsulting.com 8
End Testing vs. Continuous Testing
Ref: ”Lean from the Trenches” by Henrick Kniberg
cbell@CamilleBellConsulting.com 9
• It doesn’t guarantee customer acceptance !
• It’s extremely labor intensive !
• It’s e...
cbell@CamilleBellConsulting.com 10
Agile Development
practices turn Waterfall
completely around
with Feedback
cbell@CamilleBellConsulting.com 11
Design
High Level
Requirements
Requirements
Analysis
Code
Business
Vision
Deploy
Tradit...
cbell@CamilleBellConsulting.com 12
Code
The Waterfall Ideal Writes All the Code
and then Tests it
Ref: “BDD in 5 minutes” ...
cbell@CamilleBellConsulting.com 13
Code
bugs
But that never works creating
the code, test & fix cycle
cbell@CamilleBellConsulting.com 14
Code
Writing the Test First Prevents Coding Bugs
cbell@CamilleBellConsulting.com 15
Code
Test First Feedback Loop
Short cycles make it work
cbell@CamilleBellConsulting.com 16
Code
Test-First, while better isn’t enough
Design
untestable
design
cbell@CamilleBellConsulting.com 17
Code
The Test-Code Cycle can provide feedback to design
Design
cbell@CamilleBellConsulting.com 18
Code
Heavy refactoring of design is part of
Test Driven Development (TDD)
Design
Again ...
cbell@CamilleBellConsulting.com 19
Code
Design
Requirements
Analysis
Now Test, Code & Design all provide
continuing feedba...
cbell@CamilleBellConsulting.com 20
The Test Driven Development Cycle
provides feedback to analysis
Requirements
Analysis
C...
cbell@CamilleBellConsulting.com 21
Code
Adding in Feature Analysis to TDD creates
Behavior Driven Development (BDD)
Design...
cbell@CamilleBellConsulting.com 22
Code
Design
Requirements
Analysis
High Level
Requirements
Not exactly what
the Market
N...
cbell@CamilleBellConsulting.com 23
Code
Design
Requirements
Analysis
The Behavior Driven Development Cycle, when
deployed ...
cbell@CamilleBellConsulting.com 24
Continuous Deployment further
improves feedback
Code
Design
Requirements
Analysis
Deplo...
cbell@CamilleBellConsulting.com 25
But what if our initial vision was flawed?
Code
Design
Requirements
Analysis
Deployed
H...
cbell@CamilleBellConsulting.com 26
Lean Startup’s Build-Measure-Learn feedback loop
tests the ideas behind the business vi...
cbell@CamilleBellConsulting.com 27
Agility Testing
starts with
Testable
Agile Requirements
cbell@CamilleBellConsulting.com 28
Idea
Test Cases
Software
Conversation
Agile Requirements Evolve from Feedback
Ref: Rach...
cbell@CamilleBellConsulting.com 29
Customer Collaborates with Dev Team on
Confirmation of Stories
User Story: Vetting Sigh...
cbell@CamilleBellConsulting.com 30
User Stories Confirmed Through
Automated Tests During Iteration
• XYZ sighting is viewe...
cbell@CamilleBellConsulting.com 31
Incrementally turn
High Level Requirements
(features, epics)
into Tests
cbell@CamilleBellConsulting.com 32
In order to conduct banking
when the bank is closed
As a bank customer
I want to use an...
cbell@CamilleBellConsulting.com 33
Break Down Big Stories into
Smaller User Stories
In order to get money
when the bank is...
cbell@CamilleBellConsulting.com 34
Choose a Story
(to become an automated test)
In order to get money
when the bank is clo...
cbell@CamilleBellConsulting.com 35
Define Some High Level Scenarios
for that Story
Feature: Cash Withdrawal
In order to ge...
cbell@CamilleBellConsulting.com 36
• Successful Withdrawal
Given my account has been
credited with $100
When I withdraw $2...
cbell@CamilleBellConsulting.com 37
• Successful Withdrawal
Given my account has been
credited with $100
When I withdraw $2...
cbell@CamilleBellConsulting.com 38
With Minor Changes it can become an
Automated Test (Cucumber example)
• Successful With...
cbell@CamilleBellConsulting.com 39
Functional & Unit Tests
Behavior Driven Development
&
Test Driven Development
cbell@CamilleBellConsulting.com 40
Behavior Driven Development
(aka Specification by Example)
•  Builds upon and formalize...
cbell@CamilleBellConsulting.com 41
So To Understand BDD,
First Understand TDD
cbell@CamilleBellConsulting.com 42
3 Rules of Test Driven Development
1.  You are not allowed to write any production code...
cbell@CamilleBellConsulting.com 43
The Test Driven Development Cycle
Write a failing test
for new functionality
Write just...
cbell@CamilleBellConsulting.com 44
JUnit 4 Tests for Movie List Story
Starting Test An empty list should have a size of ze...
cbell@CamilleBellConsulting.com 45
cbell@CamilleBellConsulting.com 46
Class Code for Movie List Story
Test An empty list should have a size of zero.
public c...
cbell@CamilleBellConsulting.com 47
cbell@CamilleBellConsulting.com 48
Old Test An empty list should have a size of zero.
New Test Adding a movie to an empty ...
cbell@CamilleBellConsulting.com 49
cbell@CamilleBellConsulting.com 50
Class Code for Movie List Story
public class Movie {
}
public class MovieList {
private...
cbell@CamilleBellConsulting.com 51
cbell@CamilleBellConsulting.com 52
•  Programmers restructure system by improving the design of existing
code without chan...
cbell@CamilleBellConsulting.com 53
Refactored Class Code after 4th Test
import java.util.ArrayList;
public class Movie {
}...
cbell@CamilleBellConsulting.com 54
cbell@CamilleBellConsulting.com 55
BDD & Cucumber
Stories to Tests
cbell@CamilleBellConsulting.com 56
Parts of a Cucumber Feature File
Feature: Cash Withdrawal
As a bank customer
I want to ...
cbell@CamilleBellConsulting.com 57
The Three Parts of
Cucumber
cbell@CamilleBellConsulting.com 58
The Structure of
Cucumber Features
Feature: [ feature title ]
As a user [ role ]
I want...
cbell@CamilleBellConsulting.com 59
Example
Cucumber Feature
cbell@CamilleBellConsulting.com 60
Step Definitions
cbell@CamilleBellConsulting.com 61
Cucumber Command Line
cbell@CamilleBellConsulting.com 62
Plus the Evolving Code Under Test
cbell@CamilleBellConsulting.com 63
The Cucumber
BDD Cycle
cbell@CamilleBellConsulting.com 64
Create Cucumber Feature File
from BDD Test Definition
cbell@CamilleBellConsulting.com 65
Run Feature from the Command Line
cbell@CamilleBellConsulting.com 66
Copy Snippet from
the Command Line Output
cbell@CamilleBellConsulting.com 67
Create Step Definition File
and Pasted Snippet
cbell@CamilleBellConsulting.com 68
Run Cucumber Again
(Notice the Pending
and Skipped Steps)
cbell@CamilleBellConsulting.com 69
Replace the Pending in the First Step
with Test Code You Need to Implement
cbell@CamilleBellConsulting.com 70
Run Cucumber Again
(Notice the Failed
and Skipped Steps)
cbell@CamilleBellConsulting.com 71
Implement Just Enough Code
to Make the Test Step Pass
cbell@CamilleBellConsulting.com 72
Run Cucumber Again
(Notice the Passed,
Pending and Skipped Steps)
cbell@CamilleBellConsulting.com 73
Repeat Cycle Until All Test Steps Pass
Write
Scenario
Create Step
Definitions
from Snip...
cbell@CamilleBellConsulting.com 74
Example
Scenario Outline
cbell@CamilleBellConsulting.com 75
Fit, Selenium examples
cbell@CamilleBellConsulting.com 76
cbell@CamilleBellConsulting.com 77
Browser Testing with Selenium
Example Selenium Testing with
jUnit and Java in Eclipse
S...
cbell@CamilleBellConsulting.com 78
Agility Testing
Ratios
cbell@CamilleBellConsulting.com 79
Dysfunctional Test Ratio
cbell@CamilleBellConsulting.com 80
Healthy Test Ratio
cbell@CamilleBellConsulting.com 81
Healthy Test Ratio
• Very, very, very Slow
• Usability testing
• Exploratory testing
• ...
cbell@CamilleBellConsulting.com 82
Agile Testing
and
Integration & Deployment
cbell@CamilleBellConsulting.com 83
3. Update
Build
Workspace
Continuous Integration Cycle
Developer
Deploy
Server
Version ...
cbell@CamilleBellConsulting.com 84
3. Update
Build
Workspace
Continuous Integration Cycle
with Performance Testing
Develop...
cbell@CamilleBellConsulting.com 85
Only Check-in Working Tested Code
Version Control
Repository
cbell@CamilleBellConsulting.com 86
Continuous Deployment Cycle
Builds on Continuous Integration
Developer
Deploy
Server
Ve...
cbell@CamilleBellConsulting.com 87
Tools for the Deployment Pipeline
Configuration
Management
Continuous
Integration
Testi...
cbell@CamilleBellConsulting.com 88
cbell@CamilleBellConsulting.com 89
Build-Measure-Lean with Lean Startup
Split Tests
Customer Interviews
Customer Developme...
cbell@CamilleBellConsulting.com 90
Examples of Testing of Ideas with Lean Startup
• Split A/B Testing
– controlled experim...
cbell@CamilleBellConsulting.com 91
cbell@CamilleBellConsulting.com 92
Bug Growth on
Non-Agile Projects
T i m e
B
u
g
s
code fix code codecode codecodecode fi...
cbell@CamilleBellConsulting.com 93
TDD and BDD prevents a Great Many Bugs, but …
a few will probably still occur
Write a f...
cbell@CamilleBellConsulting.com 94
Create a RED test BEFORE fixing the bug,
… so you are sure you really fixed it,
and the...
cbell@CamilleBellConsulting.com 95
cbell@CamilleBellConsulting.com 96
• By acting as a servant leader and removing obstacles
• By providing whole team traini...
cbell@CamilleBellConsulting.com 97
•  Agile processes like Scrum, Lean, Kanban and XP:
•  often make short near term predi...
cbell@CamilleBellConsulting.com 98
Running Tested Features Definition
1.  The	
  desired	
  so,ware	
  is	
  broken	
  dow...
cbell@CamilleBellConsulting.com 99
Calculating RTF is Simple
•  No weighting - No partials credit: A Feature is
either all...
cbell@CamilleBellConsulting.com 100
RTF Works on any Project, even Waterfall
RTF on Agile software
development projects
sh...
cbell@CamilleBellConsulting.com 101
Example 1: Buggy New Feature Breaks Old
Monday 6 running tested
features. Tuesday no
n...
cbell@CamilleBellConsulting.com 102
Example 2: New Feature Breaks Everything
Monday 6 running tested
features. Tuesday no
...
cbell@CamilleBellConsulting.com 103
CUMULATIVE BURNUP
StoryPointsorFeaturesComplete(RTF)
Time
Burnup of Story Points or Ru...
cbell@CamilleBellConsulting.com 104
Camille Bell
Agile Coaching & Consulting
Retrospectives
Agile Boot Camps
Agile Trainin...
cbell@CamilleBellConsulting.com 105
cbell@CamilleBellConsulting.com 106
Code Coverage
C0, C1 and C2 Coverage
C0 – Line coverage analysis measures which lines ...
cbell@CamilleBellConsulting.com 107
Code Coverage
C0, C1 and C2 Tools
C0- Every automated code coverage tool measures at l...
cbell@CamilleBellConsulting.com 108
public class CoverageExample {
// This code is logically equivalent to the next exampl...
cbell@CamilleBellConsulting.com 109
Code Coverage
C0 – Line Coverage Limitations
public class CoverageExample
// This code...
cbell@CamilleBellConsulting.com 110
Code Coverage
C1 – Branch Coverage
public class CoverageExample
// This code is identi...
cbell@CamilleBellConsulting.com 111
Code Coverage
C2 – Path Coverage Explosion
false false false
false false true
false tr...
cbell@CamilleBellConsulting.com 112
Code Coverage
C2 – Basis Path Simplification
false false false
true false false
false ...
cbell@CamilleBellConsulting.com 113
Code Coverage
Line, Branch & Basis Path
// JUnit 4 tests with 100% line, 100% branch c...
cbell@CamilleBellConsulting.com 114
Cucumber Code
Coverage with
RCov
(today Simplecov would better tool)
cbell@CamilleBellConsulting.com 115
RCov and Clover
•  RCov, an open source Ruby Gem provides code
coverage measurement fo...
cbell@CamilleBellConsulting.com 116
RCov Cucumber/Ruby Example
cbell@CamilleBellConsulting.com 117
RCov Cucumber/Ruby Example
cbell@CamilleBellConsulting.com 118
RCov Cucumber/Ruby Example
Upcoming SlideShare
Loading in...5
×

Testing for Agility: Bringing Testing into Everything

1,143

Published on

A testing focus can drive agility in every aspect of software development. Topics include: the cost of waiting to test, how waterfall can be transformed into agility with feedback, how testing can drive requirements discovery, testing your vision, testing during development, testing as part of integration and deployment, testing for customer acceptance, agile ratios of different types of tests, agile testing for bug fixes and maintenance, management concerns for agile testing and code coverage.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,143
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
32
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Testing for Agility: Bringing Testing into Everything

  1. 1. cbell@CamilleBellConsulting.com 1 Testing for Agility: Bringing Testing into Everything Camille Bell Agile Coach & Trainer cbell@CamilleBellConsulting.com Twitter @agilecamille TDD User Story: Vetting Sighting In order to show confirmation of a sighting, As a US-CERT analyst, I want to mark a sighting a vetted • XYZ sighting is viewed by Charley US-CERT analyst • Charley marks sighting as vetted • Tracy from Treasury searches for sightings vetted today • Tracy sees XYZ as a confirmed sighting
  2. 2. cbell@CamilleBellConsulting.com 2 About Me •  Agile Coach, Trainer and Developer •  Over 4 decades of IT. All sorts of companies, domains, technologies & IT roles. •  Email: cbell@CamilleBellConsulting.com •  Twitter: @agilecamille •  Slideshare: camille_bell
  3. 3. cbell@CamilleBellConsulting.com 3 • Waiting to test • Transforming Waterfall into Agility using Feedback • Agile testing and requirements • Agile testing and development • Agile testing and integration and deployment • Agile testing and acceptance • Agile testing ratios • Agile testing, bug fixes and maintenance work • Agile testing and management • Code Coverage Agenda
  4. 4. cbell@CamilleBellConsulting.com 4 The Trouble with Waiting to Test
  5. 5. cbell@CamilleBellConsulting.com 5 Agility puts more focus on testing to improve quality and time to market Which is great assuming 21st Century Agile testing and complementary practices
  6. 6. cbell@CamilleBellConsulting.com 6 But many shops bolt on Waterfall phased testing to Agile development While still better than no Agility, it is costly in time and money
  7. 7. cbell@CamilleBellConsulting.com 7 Release Testing vs. Continuous Testing Ref: ”Lean from the Trenches” by Henrick Kniberg
  8. 8. cbell@CamilleBellConsulting.com 8 End Testing vs. Continuous Testing Ref: ”Lean from the Trenches” by Henrick Kniberg
  9. 9. cbell@CamilleBellConsulting.com 9 • It doesn’t guarantee customer acceptance ! • It’s extremely labor intensive ! • It’s error prone ! • It’s often ambiguous ! • It doesn’t stop fixed bugs from reappearing in the code ! • Regression testing is a nightmare ! • It doesn’t prevent “but it works on my machine” thinking ! • It’s sooner, but still after the fact ! (code exists) • It kills your test team ! Some other Waterfall phased testing (there are many more)
  10. 10. cbell@CamilleBellConsulting.com 10 Agile Development practices turn Waterfall completely around with Feedback
  11. 11. cbell@CamilleBellConsulting.com 11 Design High Level Requirements Requirements Analysis Code Business Vision Deploy Traditional Phased Waterfall No feedback – Assumes perfection! Plan
  12. 12. cbell@CamilleBellConsulting.com 12 Code The Waterfall Ideal Writes All the Code and then Tests it Ref: “BDD in 5 minutes” video by Corey Haines "How test-driven development works" blog by J. B. Rainsberger
  13. 13. cbell@CamilleBellConsulting.com 13 Code bugs But that never works creating the code, test & fix cycle
  14. 14. cbell@CamilleBellConsulting.com 14 Code Writing the Test First Prevents Coding Bugs
  15. 15. cbell@CamilleBellConsulting.com 15 Code Test First Feedback Loop Short cycles make it work
  16. 16. cbell@CamilleBellConsulting.com 16 Code Test-First, while better isn’t enough Design untestable design
  17. 17. cbell@CamilleBellConsulting.com 17 Code The Test-Code Cycle can provide feedback to design Design
  18. 18. cbell@CamilleBellConsulting.com 18 Code Heavy refactoring of design is part of Test Driven Development (TDD) Design Again short cycles are key
  19. 19. cbell@CamilleBellConsulting.com 19 Code Design Requirements Analysis Now Test, Code & Design all provide continuing feedback for improvement, but …. Not what the customer really wants
  20. 20. cbell@CamilleBellConsulting.com 20 The Test Driven Development Cycle provides feedback to analysis Requirements Analysis Code Design
  21. 21. cbell@CamilleBellConsulting.com 21 Code Adding in Feature Analysis to TDD creates Behavior Driven Development (BDD) Design Short cycles again Requirements Analysis
  22. 22. cbell@CamilleBellConsulting.com 22 Code Design Requirements Analysis High Level Requirements Not exactly what the Market Needed (even though customer rep thought it was) Now Behavior Driven Development provides continuing feedback for improvement, but ….
  23. 23. cbell@CamilleBellConsulting.com 23 Code Design Requirements Analysis The Behavior Driven Development Cycle, when deployed to the field provides feedback to marketing Deployed High Level Requirements
  24. 24. cbell@CamilleBellConsulting.com 24 Continuous Deployment further improves feedback Code Design Requirements Analysis Deployed High Level Requirements
  25. 25. cbell@CamilleBellConsulting.com 25 But what if our initial vision was flawed? Code Design Requirements Analysis Deployed High Level Requirements Business Vision Product Concept wrong !!! Entire effort fails at great expense !!!
  26. 26. cbell@CamilleBellConsulting.com 26 Lean Startup’s Build-Measure-Learn feedback loop tests the ideas behind the business vision
  27. 27. cbell@CamilleBellConsulting.com 27 Agility Testing starts with Testable Agile Requirements
  28. 28. cbell@CamilleBellConsulting.com 28 Idea Test Cases Software Conversation Agile Requirements Evolve from Feedback Ref: Rachel Davies & Liz Sedley – “Agile Coaching”
  29. 29. cbell@CamilleBellConsulting.com 29 Customer Collaborates with Dev Team on Confirmation of Stories User Story: Vetting Sighting In order to show confirmation of a sighting, As a US-CERT analyst, I want to mark a sighting a vetted • XYZ sighting is viewed by Charley US-CERT analyst • Charley marks sighting as vetted • Tracy from Treasury searches for sightings vetted today • Tracy sees XYZ as a confirmed sighting Ref: Ron Jeffries 3 Cs of User Stories Card Confirmation Collaboration
  30. 30. cbell@CamilleBellConsulting.com 30 User Stories Confirmed Through Automated Tests During Iteration • XYZ sighting is viewed by Charley US-CERT analyst • Charley marks sighting as vetted • Tracy from Treasury searches for sighting vetted today • Tracy sees XYZ as a confirmed sighting User Story: Vetting Sighting In order to show confirmation of a sighting, As a US-CERT analyst, I want to mark a sighting a vetted describe ”Imports TEWI sightings" do context "a simple line item should know its name, quanity and price" do let(:purchase) { LineItem.new("1 book at 12.49") } it "should have a name of 'book'" do purchase.name.should == 'book' end it "should have a quanity of 1" do purchase.quantity.should == 1 end it "should have a price of 12.49" do purchase.price.should == 12.49 end end describe ”CERT analyst finds a recent TEWI sighting" do context "a simple line item should know its name, quanity and price" do let(:purchase) { LineItem.new("1 book at 12.49") } it "should have a name of 'book'" do purchase.name.should == 'book' end it "should have a quanity of 1" do purchase.quantity.should == 1 end it "should have a price of 12.49" do purchase.price.should == 12.49 end end describe ”Vets a sighting" do context "a simple line item should know its name, quanity and price" do let(:purchase) { LineItem.new("1 book at 12.49") } it "should have a name of 'book'" do purchase.name.should == 'book' end it "should have a quanity of 1" do purchase.quantity.should == 1 end it "should have a price of 12.49" do purchase.price.should == 12.49 end end describe ” Trusted partner finds un-vetted sighting" do context "a simple line item should know its name, quanity and price" do let(:purchase) { LineItem.new("1 book at 12.49") } it "should have a name of 'book'" do purchase.name.should == 'book' end it "should have a quanity of 1" do purchase.quantity.should == 1 end it "should have a price of 12.49" do purchase.price.should == 12.49 end end describe "Trusted partner finds vetted sighting" do context ”a vetted sighting should be clearly vetted" do before(:each) do new_sighting.build_from_TEWI() cert_user.new(‘default_cert’) new_sighting.vet(cert_user) new_partner.new(‘default_partner’) end it "should have a tag of ’vetted'" do new_sighting.tag.should_include == ’vetted' end it "should be viewable by partner" do new_sighting.accesable_by(new_partner).should == true end end
  31. 31. cbell@CamilleBellConsulting.com 31 Incrementally turn High Level Requirements (features, epics) into Tests
  32. 32. cbell@CamilleBellConsulting.com 32 In order to conduct banking when the bank is closed As a bank customer I want to use an ATM Why? Who? What? Why does your user want this? Your idea probably needs to be broken down into smaller stories. Who is this user? How is he/she different from other users. What exactly does your user want? Start with an idea for an App (like ATM banking) These ideas are called User Stories.
  33. 33. cbell@CamilleBellConsulting.com 33 Break Down Big Stories into Smaller User Stories In order to get money when the bank is closed As a bank customer I want to withdraw cash at the ATM In order to deposit my checks when the bank is closed As a bank customer I want to deposit checks at the ATM In order to earn interest even when the bank is closed As a bank customer I want to transfer money from checking to saving at the ATM In order to not overdraw my account when the bank is closed As a bank customer I want to transfer money from savings to checking at the ATM
  34. 34. cbell@CamilleBellConsulting.com 34 Choose a Story (to become an automated test) In order to get money when the bank is closed As a bank customer I want to withdraw cash at the ATM Feature: Cash Withdrawal In order to get money when the bank is closed As a bank customer I want to withdraw cash at the ATM
  35. 35. cbell@CamilleBellConsulting.com 35 Define Some High Level Scenarios for that Story Feature: Cash Withdrawal In order to get money when the bank is closed As a bank customer I want to withdraw cash at the ATM • Successful Withdrawal • Withdrawal Failed Because Cash Machine Doesn’t Have Dollar Bills • Withdrawal Failed Due to Insufficient Funds • Withdrawal Failed Because Account Closed
  36. 36. cbell@CamilleBellConsulting.com 36 • Successful Withdrawal Given my account has been credited with $100 When I withdraw $20 Then $20 should be dispensed And the balance of my account should be $80 Choose a Scenario and Define the Steps in Testable Detail
  37. 37. cbell@CamilleBellConsulting.com 37 • Successful Withdrawal Given my account has been credited with $100 When I withdraw $20 Then $20 should be dispensed And the balance of my account should be $80 Use Given, When, Then format. OK to add And and But for readability Test steps must be unambiguous. Quantities are especially testable.
  38. 38. cbell@CamilleBellConsulting.com 38 With Minor Changes it can become an Automated Test (Cucumber example) • Successful Withdrawal Given my account has been credited with $100 When I withdraw $20 Then $20 should be dispensed And the balance of my account should be $80 Scenario: Successful Withdrawal Given my account has been credited with $100 When I withdraw $20 Then $20 should be dispensed And the balance of my account should be $80
  39. 39. cbell@CamilleBellConsulting.com 39 Functional & Unit Tests Behavior Driven Development & Test Driven Development
  40. 40. cbell@CamilleBellConsulting.com 40 Behavior Driven Development (aka Specification by Example) •  Builds upon and formalizes the best practices of Test Driven Development •  Works outside-in •  Starts with a failing customer acceptance test •  Test describes needed behavior from a customer viewpoint •  Tests must be readable by anyone on team
  41. 41. cbell@CamilleBellConsulting.com 41 So To Understand BDD, First Understand TDD
  42. 42. cbell@CamilleBellConsulting.com 42 3 Rules of Test Driven Development 1.  You are not allowed to write any production code unless it is to make a failing unit test pass. 2.  You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures. 3.  You are not allowed to write any more production code than is sufficient to pass the one failing unit test. With tests you are in control; without tests you aren’t! Bob Martin - The Three Rules of TDD
  43. 43. cbell@CamilleBellConsulting.com 43 The Test Driven Development Cycle Write a failing test for new functionality Write just enough code to pass test Refactored code must also pass tests, no new functionality CamillesCareer@gmail.com 43
  44. 44. cbell@CamilleBellConsulting.com 44 JUnit 4 Tests for Movie List Story Starting Test An empty list should have a size of zero. import static org.junit.Assert.*; import org.junit.Test; public class MovieListTest { @Test public void testEmptyListSize() { MovieList emptyMovieList = new MovieList(); assertEquals("Size of movie list should be 0.", 0, emptyMovieList.size()); } } Ref: Java code examples from “Test-Driven Development: A Practical Guide” by David Astels
  45. 45. cbell@CamilleBellConsulting.com 45
  46. 46. cbell@CamilleBellConsulting.com 46 Class Code for Movie List Story Test An empty list should have a size of zero. public class MovieList { public int size() { return 0; } } Couldn't be simpler, but that is the point!
  47. 47. cbell@CamilleBellConsulting.com 47
  48. 48. cbell@CamilleBellConsulting.com 48 Old Test An empty list should have a size of zero. New Test Adding a movie to an empty list should result in a list with the size of one. public class MovieListTest { @Test public void testEmptyListSize() { MovieList emptyMovieList = new MovieList(); assertEquals("Size of movie list should be 0.", 0, emptyMovieList.size()); } @Test public void testSizeAfterAddingOne () { Movie starWars = new Movie(); MovieList oneItemList = new MovieList(); oneItemList.add(starWars); assertEquals("Size of movie list should be 1.", 1, oneItemList.size()); } } JUnit 4 Tests for Movie List Story
  49. 49. cbell@CamilleBellConsulting.com 49
  50. 50. cbell@CamilleBellConsulting.com 50 Class Code for Movie List Story public class Movie { } public class MovieList { private int numberOfMovies = 0; public int size() { return numberOfMovies; } public void add(Movie movieToAdd) { numberOfMovies = 1; } } Baby Steps: Change as little as possible to get to GREEN New Test Adding a movie to an empty list should result in a list with the size of one.
  51. 51. cbell@CamilleBellConsulting.com 51
  52. 52. cbell@CamilleBellConsulting.com 52 •  Programmers restructure system by improving the design of existing code without changing its behavior –  to remove duplication, improve communication, simplify, or add flexibility •  Refactoring is not random change, instead it’s driven by learning from our tests and from identifying and fixing code smells •  DRY (Don’t Repeat Yourself) •  Refactoring can occur just prior or just after writing new code •  Test, code, refactor, re-test –  Kent Beck suggests short cycle (10 minutes) Refactoring When fixing bugs, write a Red test, BEFORE fixing the bug. Test will run Green when bug fixed. When working with legacy code make sure to write automated tests BEFORE refactoring code, If tests didn’t exist.
  53. 53. cbell@CamilleBellConsulting.com 53 Refactored Class Code after 4th Test import java.util.ArrayList; public class Movie { } public class MovieList { private int numberOfMovies = 0; private ArrayList<Movie> movies = new ArrayList<Movie>(); public int size() { return movies.size(); } public void add(Movie movieToAdd) { numberOfMovies++; movies.add(movieToAdd); } public boolean contains(Movie movieToCheckFor) { return movies.contains(movieToCheckFor); } } New Test If we add a movie to a list, we should be able to ask if it’s there and receive a positive response. Delete Delete Change
  54. 54. cbell@CamilleBellConsulting.com 54
  55. 55. cbell@CamilleBellConsulting.com 55 BDD & Cucumber Stories to Tests
  56. 56. cbell@CamilleBellConsulting.com 56 Parts of a Cucumber Feature File Feature: Cash Withdrawal As a bank customer I want to withdraw cash at the ATM So that I can get money when the bank is closed Scenario: Successful Withdrawal Given my account has been credited with $100 When I withdraw $20 Then $20 should be dispensed And the balance of my account should be $80 Scenario Title Scenario Feature Title Feature Steps cash_withdrawal.feature Ref: Cucumber code examples from “The Cucumber Book” by Matt Wynne and Aslak Hellesøy
  57. 57. cbell@CamilleBellConsulting.com 57 The Three Parts of Cucumber
  58. 58. cbell@CamilleBellConsulting.com 58 The Structure of Cucumber Features Feature: [ feature title ] As a user [ role ] I want [ feature ] So that [ business value ] Scenario: [ scenario title ] Given [ a pre-condition ] And [another pre-condition ] When [ event ] Then [ post-condition outcome ] And [ another post-condition outcome ] Plain TextKey Words From User Story Ref: Cucumber code examples from “The Cucumber Book” by Matt Wynne and Aslak Hellesøy
  59. 59. cbell@CamilleBellConsulting.com 59 Example Cucumber Feature
  60. 60. cbell@CamilleBellConsulting.com 60 Step Definitions
  61. 61. cbell@CamilleBellConsulting.com 61 Cucumber Command Line
  62. 62. cbell@CamilleBellConsulting.com 62 Plus the Evolving Code Under Test
  63. 63. cbell@CamilleBellConsulting.com 63 The Cucumber BDD Cycle
  64. 64. cbell@CamilleBellConsulting.com 64 Create Cucumber Feature File from BDD Test Definition
  65. 65. cbell@CamilleBellConsulting.com 65 Run Feature from the Command Line
  66. 66. cbell@CamilleBellConsulting.com 66 Copy Snippet from the Command Line Output
  67. 67. cbell@CamilleBellConsulting.com 67 Create Step Definition File and Pasted Snippet
  68. 68. cbell@CamilleBellConsulting.com 68 Run Cucumber Again (Notice the Pending and Skipped Steps)
  69. 69. cbell@CamilleBellConsulting.com 69 Replace the Pending in the First Step with Test Code You Need to Implement
  70. 70. cbell@CamilleBellConsulting.com 70 Run Cucumber Again (Notice the Failed and Skipped Steps)
  71. 71. cbell@CamilleBellConsulting.com 71 Implement Just Enough Code to Make the Test Step Pass
  72. 72. cbell@CamilleBellConsulting.com 72 Run Cucumber Again (Notice the Passed, Pending and Skipped Steps)
  73. 73. cbell@CamilleBellConsulting.com 73 Repeat Cycle Until All Test Steps Pass Write Scenario Create Step Definitions from Snippet Turn Pending Step into Test Watch it Fail Make it Pass by Evolving Code Pending Steps? YesNoDone with Scenario Start with User Story
  74. 74. cbell@CamilleBellConsulting.com 74 Example Scenario Outline
  75. 75. cbell@CamilleBellConsulting.com 75 Fit, Selenium examples
  76. 76. cbell@CamilleBellConsulting.com 76
  77. 77. cbell@CamilleBellConsulting.com 77 Browser Testing with Selenium Example Selenium Testing with jUnit and Java in Eclipse Selenium IDE (Firefox only) Record/Playback Recorded tests don’t scale, Only run test individually or single test suite – good for first cut tests Selenium Remote Control cross browser tests with Java, Ruby, Python, Perl, PHP or .Net
  78. 78. cbell@CamilleBellConsulting.com 78 Agility Testing Ratios
  79. 79. cbell@CamilleBellConsulting.com 79 Dysfunctional Test Ratio
  80. 80. cbell@CamilleBellConsulting.com 80 Healthy Test Ratio
  81. 81. cbell@CamilleBellConsulting.com 81 Healthy Test Ratio • Very, very, very Slow • Usability testing • Exploratory testing • Labor intensive • Automate exploration • Very slow • Validate UI functionality • Brittle, if using capture playback • Very, very fast • Uses mocks • Validate class and function/method code • Some fast, some slow • Data and other integrity • Validate at higher level than unit tests, but not directly user facing
  82. 82. cbell@CamilleBellConsulting.com 82 Agile Testing and Integration & Deployment
  83. 83. cbell@CamilleBellConsulting.com 83 3. Update Build Workspace Continuous Integration Cycle Developer Deploy Server Version Control Repository 1. Check-in CI Server2. Monitor changes 4. Build 5. Run Tests 6. Send Build & Test Report 7. Deploy App at Release
  84. 84. cbell@CamilleBellConsulting.com 84 3. Update Build Workspace Continuous Integration Cycle with Performance Testing Developer Deploy Server Version Control Repository 1. Check-in CI Server2. Monitor changes 4. Build 5. Run Tests 6. Send Build & Test Report 7b. Deploy App at Release Performance Test Server Frequently 7a. Performance Test App
  85. 85. cbell@CamilleBellConsulting.com 85 Only Check-in Working Tested Code Version Control Repository
  86. 86. cbell@CamilleBellConsulting.com 86 Continuous Deployment Cycle Builds on Continuous Integration Developer Deploy Server Version Control Repository 1. Check-in 2. Monitor changes 3. Update Build Workspace 4. Build 5. Run Tests CI Server 6. Send Build & Test Report 7. Clean Build & Tests Pass Deploy App Immediately
  87. 87. cbell@CamilleBellConsulting.com 87 Tools for the Deployment Pipeline Configuration Management Continuous Integration Testing Strategy Deployment Capistrano
  88. 88. cbell@CamilleBellConsulting.com 88
  89. 89. cbell@CamilleBellConsulting.com 89 Build-Measure-Lean with Lean Startup Split Tests Customer Interviews Customer Development 5 Whys Root Cause Analysis Customer Advisory Board Falsifiable Hypotheses Product Owner Accountability Customer Archetypes Cross-functional Teams Semi-autonomous Teams Smoke Tests Unit Tests Usability Tests Continuous Integration Incremental Deployment Free & Open-Source Components Cloud Computing Cluster Immune System Just-in-time Scalability Refactoring Developer Sandbox Split Tests Clear Product Owner Continuous Deployment Usability Tests Real-time Monitoring Customer Liaison Funnel Analysis Cohort Analysis Net Promoter Score Real-Time Altering Predictive Monitoring Measure Faster Learn Faster Code Faster
  90. 90. cbell@CamilleBellConsulting.com 90 Examples of Testing of Ideas with Lean Startup • Split A/B Testing – controlled experiments – two versions of product released simultaneously – test metric comparison ($ earned, # of signups, etc.) – Netflix huge user of A/B testing • Crowd Testing: Validating the existence of a customer base – bypassing investors with Kickstarter, etc. (e.g. Torment) – creating dummy site to sign up and buy (e.g. Zappos)
  91. 91. cbell@CamilleBellConsulting.com 91
  92. 92. cbell@CamilleBellConsulting.com 92 Bug Growth on Non-Agile Projects T i m e B u g s code fix code codecode codecodecode fixfixfixfix fix release 1 release 2 release 3 release 4 release 5 release 6 release 7
  93. 93. cbell@CamilleBellConsulting.com 93 TDD and BDD prevents a Great Many Bugs, but … a few will probably still occur Write a failing test for new functionality Write just enough code to pass test Refactored code must also pass tests, no new functionality cbell@CamilleBellConsulting.com 93
  94. 94. cbell@CamilleBellConsulting.com 94 Create a RED test BEFORE fixing the bug, … so you are sure you really fixed it, and the bug never sneaks through the tests again. cbell@CamilleBellConsulting.com 94 Write a failing test that exposes the bug Write just enough code to fix the bug Refactor if needed, must also pass tests, no new functionality
  95. 95. cbell@CamilleBellConsulting.com 95
  96. 96. cbell@CamilleBellConsulting.com 96 • By acting as a servant leader and removing obstacles • By providing whole team training in agile processes • By providing technical training in automated test tools to current employees • By requiring automated test experience for new hires • By making testability a must have for languages & tools (many of the best are open source) • By providing the automated test tools themselves • By providing continuous integration and deployment servers • By tracking progress differently Management Needs to Support Agility
  97. 97. cbell@CamilleBellConsulting.com 97 •  Agile processes like Scrum, Lean, Kanban and XP: •  often make short near term predictions •  track actual progress •  determine if the predictions, if made, were correct •  use actual progress metrics to project future outcomes •  adapt and adjust based on real outcomes • Managers must ensure that agile practices: •  help the team improve projections •  never become a stick Agility Organizational Processes Have Built-in Feedback Loops and Validation
  98. 98. cbell@CamilleBellConsulting.com 98 Running Tested Features Definition 1.  The  desired  so,ware  is  broken  down  into  named  features   (requirements,  stories),  which  are  part  of  what  it  means  to   deliver  the  desired  system.     2.  For  each  named  feature,  there  are  one  or  more  automated   acceptance  tests  which,  when  they  work,  will  show  that  the   feature  in  quesAon  is  implemented.   3.  The  RTF  metric  shows,  at  every  moment  in  the  project,  how   many  features  are  passing  all  their  acceptance  tests.     Ron Jeffries, A Metric Leading to Agility
  99. 99. cbell@CamilleBellConsulting.com 99 Calculating RTF is Simple •  No weighting - No partials credit: A Feature is either all running & passing all tests or it isn’t. •  So a feature either counts as 0 or 1. •  Add up the 1s. •  Run all tests for each measurement. •  Track over time.
  100. 100. cbell@CamilleBellConsulting.com 100 RTF Works on any Project, even Waterfall RTF on Agile software development projects should steadily increase over time RTF on Waterfall development only increases at project end
  101. 101. cbell@CamilleBellConsulting.com 101 Example 1: Buggy New Feature Breaks Old Monday 6 running tested features. Tuesday no new features added, but existing features still pass tests. On Wednesday, new feature added which fails test and causes side effect that causes pre- existing feature to also fail tests. On Thursday, new tests added to old feature and new feature fixed. All tests pass.
  102. 102. cbell@CamilleBellConsulting.com 102 Example 2: New Feature Breaks Everything Monday 6 running tested features. Tuesday no new features added, but existing features still pass tests. On Wednesday, new feature added which breaks the build, crashes system, corrupts DB or something similarly catastrophic. On Thursday, new tests added and new feature fixed. All tests pass.
  103. 103. cbell@CamilleBellConsulting.com 103 CUMULATIVE BURNUP StoryPointsorFeaturesComplete(RTF) Time Burnup of Story Points or Running Tested Features metrics can be more informative than burndowns. New Stories when added caused existing stories to fail automated tests Stories fixed, tests added, all tests now pass Stories Removed to meet Release DateStories Added New Baseline New Baseline Baseline ReleaseDate Complex Code needed refactoring Ref: Alistair Cockburn http://alistair.cockburn.us/Earned-value+and+burn+charts Ron Jeffries http://xprogramming.com/articles/jatrtsmetric/
  104. 104. cbell@CamilleBellConsulting.com 104 Camille Bell Agile Coaching & Consulting Retrospectives Agile Boot Camps Agile Training Updated Slides or just to chat about things agile CamillesCareer@gmail.com
  105. 105. cbell@CamilleBellConsulting.com 105
  106. 106. cbell@CamilleBellConsulting.com 106 Code Coverage C0, C1 and C2 Coverage C0 – Line coverage analysis measures which lines of code have been executed. C0 coverage is typically used it to find the areas of your program that have not been sufficiently tested, i.e. those that were not run by any your test cases. C1 – Branch coverage analysis measures which of the different possible branches of conditional statements have been tested. It is easy to have 100% C0 line coverage and only partial C1 branch coverage, because if, then and if, then, else statements may be contained all on one line in many languages. C2 - Path coverage measures which of the different possible execution paths through your code were tested. Paths are a combination of linear code execution with alternate branches. Since each new conditional that is encountered gives rise to new path choices, the permutations of possible unique paths gets huge.
  107. 107. cbell@CamilleBellConsulting.com 107 Code Coverage C0, C1 and C2 Tools C0- Every automated code coverage tool measures at least Line Coverage. Open source tools rcov for Ruby and Cobertura for Java provide C0 coverage. C1 - Many automated code coverage tools (both open source and commercial) measure Branch coverage. For instance open source tool Emma for Java measures both Line and Branch coverage as does Clover, a commercial Java test tool. C2 – Complete path coverage throughout an entire large application is almost unheard of because of its complexity and cost. I know of only two large software applications (both in Ada) that had 100% path coverage. Path coverage through multiple limited sections of code is much more feasible. All path coverage tools also include line and branch coverage. All are commercial and very expensive.
  108. 108. cbell@CamilleBellConsulting.com 108 public class CoverageExample { // This code is logically equivalent to the next example, but because of the // code structure, untested branches are detected by line coverage. public int echo (int x, boolean state1, boolean state2, boolean state3) { if (state1) { x++; } if (state2) { x--; } if (state3) { x = x; } return x; } } // This test provides 100% Line coverage, but it doesn’t provide any branch coverage. public class CoverageTestLineOnly { CoverageExample lineBranchPath; @Before public void runBeforeEveryTest() { lineBranchPath = new CoverageExample(); } @Test public void testReturnInput0FalseFalseFalse() { assertEquals(0, lineBranchPath.echo(0, false, false, false); } } Code Coverage C0 – Line Coverage
  109. 109. cbell@CamilleBellConsulting.com 109 Code Coverage C0 – Line Coverage Limitations public class CoverageExample // This code is logically equivalent to the prior example, but because of the // code structure, untested branches are not detected by line coverage. public int echo (int x, boolean state1, boolean state2, boolean state3) { if (state1) { x++; } if (state2) { x--; } if (state3) { x = x; } return x; } } // This test provides 100% Line coverage, but it doesn’t provide any branch coverage. public class CoverageTestLineOnly { CoverageExample lineBranchPath; @Before public void runBeforeEveryTest() { lineBranchPath = new CoverageExample(); } @Test public void allFalse() { assertEquals(0, lineBranchPath.echo(0, false, false, false); } }
  110. 110. cbell@CamilleBellConsulting.com 110 Code Coverage C1 – Branch Coverage public class CoverageExample // This code is identical to the prior example. public int echo (int x, boolean state1, boolean state2, boolean state3) { if (state1) { x++; } if (state2) { x--; } if (state3) { x = x; } return x; } } // Adding the new test provide 100% branch coverage, but doesn’t detect the bug. public class CoverageTestLineAndBranch { CoverageExample lineBranchPath; @Before public void runBeforeEveryTest() { lineBranchPath = new CoverageExample(); } @Test public void allFalse() { assertEquals(0, lineBranchPath.echo(0, false, false, false); } @Test public void allTrue() { assertEquals(0, lineBranchPath.echo(0, true, true, true); } }
  111. 111. cbell@CamilleBellConsulting.com 111 Code Coverage C2 – Path Coverage Explosion false false false false false true false true false false true true true false false true false true true true false true true true To test all possible paths of N Boolean conditionals is 2 ^ N. So for our example of 3 Booleans that’s 2x2x2 or 8. As shown in the example on the left, it’s not too bad. However, the set of all possible paths grows exponentially, so a combination of 10 Booleans, becomes 2x2x2x2x2x2x2x2x2x2 or 1024 possible paths, an unmanageable number of tests.
  112. 112. cbell@CamilleBellConsulting.com 112 Code Coverage C2 – Basis Path Simplification false false false true false false false true false false false true Basis path testing is a simplification of path testing, that significantly lowers the number of needed tests A complete set of basis path sets are the number of boolean decisions +1. It grows linearly instead of exponentially. that’s 3+1 or 4 instead of 8 for our example and 10+1 or 11 instead of 1024. Path 1: Any path will do for the baseline, so pick all trues or all falses for simplification. We are picking false. This is the first path in our basis set. Path 2: To find the next basis path, we flip the first decision (only) in our baseline. Path 3: Next we flip the second decision (only) in our baseline path, the first baseline decision remains fixed with the false outcome. Path 4: Finally, you flip the third decision in your baseline path. Again, the first baseline decision remains fixed with the false outcome.
  113. 113. cbell@CamilleBellConsulting.com 113 Code Coverage Line, Branch & Basis Path // JUnit 4 tests with 100% line, 100% branch coverage, and 100% basis path. public class CoverageTestPath { CoverageExample lineBranchPath; @Before public void runBeforeEveryTest() { lineBranchPath = new CoverageExample(); } @After public void runBeforeEveryTest() { lineBranchPath = null ; } @Test // First basis path. public void allFalse() { assertEquals ("All false inputs ", 0, lineBranchPath.echo(0, false, false, false); } @Test // Second basis path. public void trueFalseFalse() { assertEquals ("True-False-False inputs ", 0, lineBranchPath.echo(0, true, false, false); } @Test // Third basis path. public void falseTrueFalse() { assertEquals ("False-True-False inputs ", 0, lineBranchPath.echo(0, false, true, false); } @Test // Fourth basis path. public void falseFalseTrue() { assertEquals ("False-False-True inputs ", 0, lineBranchPath.echo, false, false, true); } }
  114. 114. cbell@CamilleBellConsulting.com 114 Cucumber Code Coverage with RCov (today Simplecov would better tool)
  115. 115. cbell@CamilleBellConsulting.com 115 RCov and Clover •  RCov, an open source Ruby Gem provides code coverage measurement for Cucumber •  RCov provides C0 – line code coverage •  Clover, a commercial tool provides code coverage measurement for JUnit •  Clover provides C1 – branch code coverage
  116. 116. cbell@CamilleBellConsulting.com 116 RCov Cucumber/Ruby Example
  117. 117. cbell@CamilleBellConsulting.com 117 RCov Cucumber/Ruby Example
  118. 118. cbell@CamilleBellConsulting.com 118 RCov Cucumber/Ruby Example

×