This is a practitioner’s view of testing and testing practices within an iterative development environment. We will explore the challenges of testing within such an environment and ways to better integrate the QA professional into what is inherently a developer-centric methodology. If quality is paramount, then we ought to move testing to the front of the line and test early and often. Automation lies at the heart of agility and we will look at how test automation techniques and test-first design philosophy might be applied at multiple-levels to drive quality.

1. Effective Testing Practices in an Agile
Environment
Raj Indugula
raj.indugula@lithespeed.com
www.lithespeed.com

2. ON-­‐DECK
AT-­‐BAT
DONE
What
is
the
problem
context?
Can
I
see
it
in
ac8on?
Concluding
thoughts.
Any
ques8ons?
What
are
the
guiding
principles
of
Agile
Tes8ng
?
What
are
the
enabling
prac8ces?

3. Agile Manifesto
Individuals and
interactions
Processes and toolsover
Working software
Comprehensive
documentation
over
Customer collaboration Contract negotiationover
Responding to change Following a planover
That is, while there is value in the items on the right, we value the items on the left
more.
We are uncovering better ways of developing software by doing it and
helping others do it. Through this work we have come to value:
http://www.agilemanifesto.org
3

4. Shared Context drives Quality
Individuals and
interactions
Working
software
Customer
collaboration
Responding to
change
4

5. Quality is a priority
Agile Approach
Maximize value and
quality within specified
project constraints.
5
Quality
5

6. Integrated Teams & Iterative Delivery
Challenges
6
How do we bridge the communication gap between Bus./QA/Dev?
How do we ensure that the evolving software does not regress?

7. ON-­‐DECK
AT-­‐BAT
DONE
What
is
the
problem
context?
Can
I
see
it
in
ac8on?
Concluding
thoughts.
Any
ques8ons?
What
are
the
guiding
principles
of
Agile
Tes8ng
?
What
are
the
enabling
prac8ces?

8. Move Quality Upstream
• Testing not a trailing activity
• QA leads the Sprint by providing guidance
and feedback on the business needs of
the emerging product
8

9. Testing is Continuous, NOT a Phase
http://testobsessed.com/
9

10. Testing is Collaborative
• Quality is everyone’s problem, not just of
the testers
• Testing is the responsibility of the whole
team
10

11. Tests represent Customer Expectations
• Shared understanding of what it means
for a story to be done
11

12. Quick Feedback
• Faster feedback loops increase Agility –
the ability to respond to change
• Test automation provides rapid feedback
on how the software is behaving
12

13. “Leave No Broken Windows”
• Fix bugs as they are found
• The sooner you find a defect, the cheaper
it is to fix
13

14. It isn't Done until it’s…
“Done Done”
Coded Tested Done
14

15. Test-Driven
Defining tests with the requirements guides development
http://testobsessed.com/
15

16. ON-­‐DECK
AT-­‐BAT
DONE
What
is
the
problem
context?
Can
I
see
it
in
ac8on?
Concluding
thoughts.
Any
ques8ons?
What
are
the
guiding
principles
of
Agile
Tes8ng
?
What
are
the
enabling
prac8ces?
ATDD
TDD
Other
prac8ces

17. Agile Engineering Practices
17
Agile Engineering practices allow teams to move fast, be flexible and
deliver high quality software
Bill Wake, http://www.xp123.com

18. What are acceptance tests?
• Tests that demonstrate business intent of
system from end user’s point of view
• Black-box testing
18

19. What is Acceptance Test Driven
Development (ATDD)?
A practice in which the whole team
collaboratively discusses acceptance
criteria, with examples, and then distills
them into a set of concrete acceptance
tests before development begins.
- Elisabeth Hendrickson
19

20. Key Characteristics
1. Elicit details from the business
stakeholder(s) about their expectations
2. Distill acceptance criteria into
automatable tests expressed in a natural
language
3. Wire the tests to SUT with “glue” code (as
part of implementation)
20

21. 21

22. When to Use
• During Requirements Elaboration
“First Cut” once the story is written
Card level acceptance criteria
• During a Three Amigos session (Business, QA, and
Dev)
Expand the story card acceptance criteria
Complete set of test scenarios
22

23. User Story
As a <type of user>, I can
<immediate goal> so that
<business outcome>.
User Role (Who?)
Desired Function
(What?) End Result (Why?)
23
3 C’s – CARD, CONVERSATION, CONFIRMATION

24. Modern Agile Acceptance Model
Conditions of Satisfaction –
Broad Terms
Acceptance Criteria –
Further Refined
Examples –
Actual scenarios or data
Executable Examples –
Ready to automate
24

25. Acceptance – Conditions of Satisfaction
Testing a Registration Function
• What constraints should we impose?
• Business stakeholders and PO agree that
passwords should not be easy to crack.
Please fill in to register.
Email Address
Password
Register
25

26. Acceptance – Acceptance Criteria
Acceptance Criteria, the Details
The PO works with testers and developers from the team,
business stakeholders, users or SMEs to come up with this
definition of “not easy to crack”:
• Must be at least 8 characters and no more than 12
• Must contain only alphanumerics and the period
• Must contain at least one digit
• Must contain at least one alpha character
Please fill in to register.
Email Address
Password ????????????
26

27. Acceptance – Examples Written
• Examples, Making it Real
• Actual examples are the ultimate way to communicate
requirements
Password Expected Expected Message Comment
abc123 Invalid You password must be at least 8 characters long, and no
more than 12 characters long.
abcdefghi Invalid Your password must contain at least one character and
one number.
1aaaaaaaaa Valid Why valid?
ajx972dab Valid
27

28. Acceptance – Testable Examples in Gherkin
• Executable Example, Making it Repeatable
• Examples that can be executed are the final step
Given the “Unregistered User” user has navigated to the “register” page
When entering “newuser” in the “Username” field
And entering “abc123” in the “Password” field
And entering “abc123” in the “Confirm Password” field
And pressing the “Register” button
Then the text “Thank you for Registering” should appear on page
And the URL should end with “use/accountPage”
28

29. Gherkin – Another Example
• Plain English with a specific format
§ Feature
§ Scenario
§ Given When Then And But
• Can be used for both manual testing and automation
29

30. Automated Acceptance Tests
Tool
(Cucumber, SpecFlow, FitNesse)
Specification
expressed in
common language
“Glue” code that
ties specification
to SUT
30

31. Business Analyst
Develop usage
scenarios
Developer
Create and maintain
acceptance test “glue”
code that ties test
specification to
system under test
Tester
Create and execute
tests to simulate
usage scenarios;
Automate regression
testing
• Should be an integral part of every iteration/sprint, not just a trailing
activity
• Vital not only to prove completeness of a feature in repeatable fashion,
but also to prove that software does not regress as it evolves
• Success depends on cooperation & collaboration
Testing Collaboratively
31

32. Benefits of ATDD
• Improved requirements elicitation
• Consensus between BA/QA/Dev on the story helps
prevent bugs & gives clear target for development
• Reuse of Acceptance Scenarios for all phases of
testing
• Creates clear examples that everyone understands;
discovers some problems before any development
32

33. ON-­‐DECK
AT-­‐BAT
DONE
What
is
the
problem
context?
Can
I
see
it
in
ac8on?
Concluding
thoughts.
Any
ques8ons?
What
are
the
guiding
principles
of
Agile
Tes8ng
?
What
are
the
enabling
prac8ces?
ATDD
TDD
Other
prac8ces

34. What are unit tests?
• Developer tests that determine whether
the smallest piece of testable software in
an application is behaving as expected
• White-box testing
34

35. Structure of a Unit Test
@Test!
public void itAddsOnePlayerToTheGame()!
{!
!Game game = new Game();!
!
!Player al = game.AddPlayer("Al");!
!
!Assert.assertEquals(1, game.numPlayers());!
!Assert.assertTrue(game.isPlaying(al));!
!Assert.assertFalse(game.isPlayable());!
}!
Assert
Arrange
Act
35
…all related to
same behavior
…title is a behavior

36. Tests should…
• Not depend on one another
• Not affect one another
• Leave the system under test unaltered
• Be fast executing
• Be executable in any order, number or
combination of one another
36

37. What is Test Driven Development (TDD)?
1. Write a unit test
1. Use unit testing and mocking framework (JUnit, Mockito etc.)
2. May need to write some skeleton code to make it compile
2. Run it and watch it fail (no code yet)
3. Write code until it passes
4. Repeat
37
Write a
new test
Red (Failing
Test)
Write Code
Green (Test
Passes)
The tests help determine
what code you write
(“Incremental Design”)

38.
Write a new
test
Test fails
Integrate Write code
Refactor code,
make sure
tests pass
Test passes
Developer
heartbeat
Red, Green, Refactor
Refactoring is supported by the tests and good unit test coverage
makes refactoring safe to do
38

39. TDD – Benefits
• Better design – less coupling
• Supports change
• Prevents gold-plating
• Prevent bugs
• Discover bugs
39

40. Test Levels
Isolation Tests
• True unit tests, focusing on behaviors
• Many, and fast
Collaboration Tests
• “End-to-end” tests
• Find bugs in conversations between collaborating
modules
40

41. Test Doubles (Mocks)
Used to isolate unit tests form external dependencies
Why use them?
• Ease of setup
• Fast executing
• To work with a subsystem that doesn’t exist
• Simulate various execution paths of external system
41

42. Test Automation Pyramid
42
Unit Tests/
Component Tests
Cucumber,
FitNesse,
SpecFlow
xUnit, Mocks
Selenium
Developer-centric
(Are we building
the code right?)
Business-centric
(Are we building
the right code?)
Adaptation of Mike Cohn's test automation pyramid
Exploratory
Testing
GUI
Tests
Acceptance Tests

43. ATDD/TDD Cycle
43

44. TDD vs. ATDD
Historically ATDD did not come about till well after TDD
ATTD
§ is a specification and testing practice
§ Determines what “Done” means for a story
§ Creates a common understanding and shared language
to discuss features (especially with Gherkin)
§ Cycle times on the order of hours to days
TDD
§ is a coding and design practice
§ Provides a mechanism for exploring the solution space
§ Creates base of tests to support refactoring
§ Cycle times on the order of seconds to minutes
44

45. ON-­‐DECK
AT-­‐BAT
DONE
What
is
the
problem
context?
Can
I
see
it
in
ac8on?
Concluding
thoughts.
Any
ques8ons?
What
are
the
guiding
principles
of
Agile
Tes8ng
?
What
are
the
enabling
prac8ces?
ATDD
TDD
Other
prac8ces

46. Version control Tests with Code
Source Code
Unit/Component Tests
Acceptance Tests
Source Control
Repository
(Git, Subversion…)
46

47. Requirements gathering Application Development
Source Control
Continuous Integration (CI) Server
Continuously commit
changes
and merge changes from
others
Pull changes,
Build, run tests
Deploy to higher
environment
Poll for changes
Development
tasks from requirements
test scenarios from business
requirements for acceptance criteria
Acceptance Test Environment
- Test harness
- Tests (automated + manual)
Execute acceptance
tests
against the deployed
application
QA
Environment
Deployable
artifacts
Build Automation & Continuous Integration
47

48. ON-­‐DECK
AT-­‐BAT
DONE
What
is
the
problem
context?
Can
I
see
it
in
ac8on?
Concluding
thoughts.
Any
ques8ons?
What
are
the
guiding
principles
of
Agile
Tes8ng
?
What
are
the
enabling
prac8ces?

49. Barriers to Adoption
Developer : “I’m a developer, not a tester”
Management: “Why does Dev help with testing,
when we have QA?”
• Necessitates behavioral change
• Quality is not free
• Simple, but not easy
• Requires discipline
• Needs generalizing specialists
49

50. Bridging the Divide
SharedUnderstanding
Collaborative
Testing
Quick
Feedback
50

51. “Before beginning a hunt, it is wise to ask
someone what you are looking for before
you begin looking for it.”
—Winnie the Pooh
51

52. Thanks!
52

53. Resources
53

54. Some Tools (open source)
• Unit testing:
JUnit, Mockito, EasyMock
• Acceptance testing:
FitNesse
Cucumber
• GUI testing:
Selenium, Sahi
• Javascript testing:
Jasmine, Qunit
• Testing the database:
DBUnit, SQLUnit, DBFit
• Performance testing:
JUnitPerf, JMeter
• Usability testing:
Manual unfortunately
54

55. Some Tools (open source)
• Unit testing:
MSTest, NUnit, Moq
• Acceptance testing:
SpecFlow
• GUI testing:
Selenium
• Javascript testing:
Chutzpa
• Testing the database:
Tsqlt, SSDT SQL Test
• Performance testing:
JetBrains, VisualStudio 2013, SQLServerProfiler, SQLLoadTest
• Usability testing:
Manual unfortunately
55

56. Agile Software Development
• Extreme Programming Explained - Kent Beck
• TDD: A Practical Guide – Dave Astels
• Refactoring – Martin Fowler
• Agile Estimating and Planning – Mike Cohn
• Pragmatic Project Automation – Pragmatic Prog.
56

57. Acceptance Testing
• Bridging the Communications Gap – Gojko Adzic
• Agile Testing – Lisa Crispin and Janet Gregory
• The Cucumber Book - Wynne M. and Hellesoy A.
Gherkin
https://github.com/aslakhellesoy/cucumber/wiki/gherkin
Cucumber
http://cukes.info/
https://github.com/aslakhellesoy/cucumber/wiki/
57

58. TDD/Refactoring
• TDD: By Example - Kent Beck
• TDD: A Practical Guide – Dave Astels
§ The Art of Agile Development – James Shore
• Refactoring – Martin Fowler
• Working Effectively with Legacy Code – Mike Feathers
• Refactoring to Patterns – Joshua Kerievsky
• xprogramming.com, refactoring.com, testdriven.com
58