Testing
A brief introduction
Main concepts
“I get paid for code that works, not for
tests, so my philosophy is to test as little
as possible to reach a given level o...
No silver bullets
What is testing about?
Getting feedback,
as much frequent as we can
the sooner, the better
TAXONOMY
Taxonomy
Scope
Unit
Integration
System
Visibillity
Black box
White box
Intention
Acceptance
Functional
Non Functional
…
Te...
Visibillity
Black
box
White
box
System
Integration
Unit
Intention
Acceptance
Functional
Non
Functional
Technique
Dynamic
S...
Scope
Database
PORT1 PORT2
ADAPTER1
Unittest
Database
Dependency injection
•Dependency:
A depends on B when A needs B to do
its job.
•Injection:
Object which uses A tells A who...
Dependency injection
• Separate business logic from creation logic
• Avoid use of new for service objects.
• Value objects...
Dependency injection
public UserService(UserValidator userValidator, UserDao userDao) {
this.userValidator = userValidator...
Dependency injection
public UserService(UserValidator userValidator, UserDao userDao) {
this.userValidator = userValidator...
Test doubles
•Fake
•Stub
•Spy
•Mock
Test doubles (Fake)
public UserDaoFake implements UserDao {
@Override
public User create(User user) {
return new User(...)...
Test doubles (Stub)
UserValidator validatorMock = mock(UserValidator.class);
stub(validatorMock.validate(any(User.class)))...
Test doubles (Spy)
Spies are objects that also record some
information based on how they were called
var validatorSpy = Si...
Test doubles (Spy)
Spies are objects that also record some
information based on how they were called
var validatorSpy = Si...
Test doubles (Mocks)
UserValidator validatorMock = mock(UserValidator.class);
when(validatorMock.validate(any(User.class))...
Integrationtestwhichwanttobeunittests
Database
FIRST(IT)
• Fast
Hundreds or thousands per second
• Isolates
Failure reasons become obvious
• Repeatable
In any order, any...
Integrationtestwhichworkswithexternalsystem
Database
IntegrationtestwhichusestheUI
Database
Systemtest
Database
Who, when and where run the tests?
• Unit
• Owner: developer
• When: after every change
• Where: every computer
• Integrat...
STRATEGIES
Static evaluation
•Informal review
•Formal review (inspection)
• Checklists
• Sucessive abstraction
•Walkthrough
Dynamic evaluation
• White box
• Path Coverage
• Statement Coverage
• Condition Coverage
• Function Coverage
• Black box
•...
White box (*-coverage)
1. Get flow diagram of the SUT
2. Calculate cyclomatic complexity
3. Determine a data set which for...
White box (*-coverage)
1. Get flow diagram of the SUT
2. Calculate cyclomatic complexity
3. Determine a data set which for...
White box (*-coverage)
1. Get flow diagram of the SUT
2. Calculate cyclomatic complexity
3. Determine a data set which for...
Black box (partitioning)
1. Identify equivalence classes
2. Select dataset:
1. Assign a unique value for every class
2. Se...
Black box (partitioning)
Register
Username*
Password (6-10 chars including numbers)
Black box (partitioning)
Register
Username*
Password (6-10 chars including numbers)
Username Password
U1: myNick P1: p4ssw...
Black box (partitioning)
Register
Username*
Password (6-10 chars including numbers)
Username Password
U1: myNick P1: p4ssw...
AUTOMATIC TESTING
4 phases-tests
1.Set Up
2.Exercise
3.Verify
4.TearDown
Testing frameworks families
• X-Unit
• @Before(Class)
• @Test
• @After(Class)
• Rspec
• describe
• beforeEach
• it
• after...
XUnit
@Before
public void setUp() {
this.userValidator = mock(UserValidator.class);
this.userDao = mock(UserDao.class);
th...
Rspec (suite per class)
describe('UserService test suite:', function(){
beforeEach(function(){
// setup the SUT
})
it('whe...
Rspec (suite per setup)
describe('UserService test suite:', function(){
describe("when create a valid user ", function() {...
BDD (specification)
Feature: User registration
Scenario: User tries to register sending valid data so the system will crea...
BDD(implementation)
@given("the user has introduced (w)+ and (w)+ into the registration form")
public void populateForm(St...
TESTABLE DESIGN
Non-Testable design smells
(by Misko Hevery*)
•Constructors does Real Work
•Digging into collaborators
•Brittle Global Sta...
Constructors does Real Work
• new keyword in a constructor or at field declaration
• Static method calls in a constructor ...
Digging into collaborators
• Objects are passed in but never used directly
(only used to get access to other objects)
• La...
Brittle Global State & Singletons
• Adding or using singletons
• Adding or using static fields or static
methods
• Adding ...
Class Does Too Much Work
• Summing up what the class does includes the
word “and”
• Class would be challenging for new tea...
Questions & Stupid questions
• ¿Where I place my tests?
• ¿Who tests the classes which test our classes?
• ¿Could you be a...
¿Where I place my tests?
• Unit tests:
• Test Class per Class
• Test Class per SetUp (useful in Xunit frameworks)
• Import...
¿Where I place my tests?
Java Project (Test Class per Class)
MyProject/
src/
main/
java/
com.groupId.artifactId.MyClass.ja...
¿Where I place my tests?
Android Project
MyProject/
AndroidManifest.xml
res/
... (resources for main application)
src/
......
¿Who tests the classes which
test our classes?
• Exactly, this is why it’s so important our tests follow
KISS
¿Couldyoube able to rewrite the codeonly
reading the tests definitions?
• Tests (specially Black Box tests) should tell us...
IspendmoretimewritingcodetosetupmySUTthan
writingthetest,howdo you solveit?
• Read about Fixtures (Xunit Patterns is a
goo...
Iduplicatetoomuchcodeonobjectscreation,mocks
definitionandassertion…
• Writing a lot of code to initialize value objects?
...
¿What is the minimum coverage
should I expect for my code?
• It depends on the project.
• “… test as little as possible to...
I’ve never write a test ¿where can I
start?
Database
PORT1 PORT2
ADAPTER1
I’ll bet you a
beer , you
called it *Util…
My code is not testable at all,
¿what can I do?
• First of all, go to http://refactoring.com/
• I suggest:
1. Add integrat...
Recommended reading
• Growing Object Oriented Software Guided Tests
• Xunit Patterns
• Continuous delivery
• Hexagonal Arc...
Place your question here!
Introduction to Software Testing
Upcoming SlideShare
Loading in …5
×

Introduction to Software Testing

2,171 views

Published on

Brief introduction to testing talk prepared to creates a starting point at TAPTAP Networks & SONATA Usa Corp. The companies where I work as Q&A Team member.

Published in: Software, Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,171
On SlideShare
0
From Embeds
0
Number of Embeds
1,447
Actions
Shares
0
Downloads
17
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Introduction to Software Testing

  1. 1. Testing A brief introduction
  2. 2. Main concepts
  3. 3. “I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence …” Kent Beck
  4. 4. No silver bullets
  5. 5. What is testing about?
  6. 6. Getting feedback, as much frequent as we can
  7. 7. the sooner, the better
  8. 8. TAXONOMY
  9. 9. Taxonomy Scope Unit Integration System Visibillity Black box White box Intention Acceptance Functional Non Functional … Technique Static Dynamic Execution Automatic Manual
  10. 10. Visibillity Black box White box System Integration Unit Intention Acceptance Functional Non Functional Technique Dynamic Static
  11. 11. Scope Database PORT1 PORT2 ADAPTER1
  12. 12. Unittest Database
  13. 13. Dependency injection •Dependency: A depends on B when A needs B to do its job. •Injection: Object which uses A tells A who is B.
  14. 14. Dependency injection • Separate business logic from creation logic • Avoid use of new for service objects. • Value objects can be created any where. • Service objects in charge to implement business logic. • IOC Container or factories in charge of creation logic.
  15. 15. Dependency injection public UserService(UserValidator userValidator, UserDao userDao) { this.userValidator = userValidator; this.userDao = userDao; } public User createUser(User user) throws ValidationException { this.userValidator.validate(user); user = this.userDao.create(user); return user; } public User createUser(User user) throws ValidationException { UserValidator userValidator = new UserValidator(...); userValidator.validate(user); UserDao userDao = new UserDao(...); user = userDao.create(user); return user; } VS
  16. 16. Dependency injection public UserService(UserValidator userValidator, UserDao userDao) { this.userValidator = userValidator; this.userDao = userDao; } public User createUser(User user) throws ValidationException { this.userValidator.validate(user); user = this.userDao.create(user); return user; } public User createUser(User user) throws ValidationException { UserValidator userValidator = new UserValidator(...); userValidator.validate(user); UserDao userDao = new UserDao(...); user = userDao.create(user); return user; } VS
  17. 17. Test doubles •Fake •Stub •Spy •Mock
  18. 18. Test doubles (Fake) public UserDaoFake implements UserDao { @Override public User create(User user) { return new User(...); } } Fake implementation in order to make test pass.
  19. 19. Test doubles (Stub) UserValidator validatorMock = mock(UserValidator.class); stub(validatorMock.validate(any(User.class))).toThrow(new ValidationException()); var validateCall = Sinon.stub(); validatorStub.withArgs(user) .onFirstCall().returns(validationError); var userValidator = { validate: validatorStub; } OR WITH JS Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what’s programmed in for the test.
  20. 20. Test doubles (Spy) Spies are objects that also record some information based on how they were called var validatorSpy = Sinon.spy(); var userValidator = { validate: validatorSpy; } userValidator.validate(user); sinon.assert.calledOnce(validatorSpy); sinon.assert.calledWith(validatorSpy, user); OR WITH JS UserValidator validatorSpy = spy(new UserValidator()); doThrow(new ValidationException()).when(validatorSpy).validate(); verify(validatorMock).validate(any(User.class))
  21. 21. Test doubles (Spy) Spies are objects that also record some information based on how they were called var validatorSpy = Sinon.spy(); var userValidator = { validate: validatorSpy; } userValidator.validate(user); sinon.assert.calledOnce(validatorSpy); sinon.assert.calledWith(validatorSpy, user); OR WITH JS UserValidator validatorSpy = spy(new UserValidator()); doThrow(new ValidationException()).when(validatorSpy).validate();
  22. 22. Test doubles (Mocks) UserValidator validatorMock = mock(UserValidator.class); when(validatorMock.validate(any(User.class))).thenTrhow(new ValidationException()); verify(validatorMock).validate(any(User.class)) Informal: think in a Stub which is also a Spy. It also responds with default values to non-explicitly declared methods var validatorAPI = {validate: function()}; var validatorMock = Sinon.mock(validatorAPI); validatorMock.expects('validate').once() .withArgs(user).throws(validationError) validatorAPI.validate(user) validatorMock.verify() OR WITH JS
  23. 23. Integrationtestwhichwanttobeunittests Database
  24. 24. FIRST(IT) • Fast Hundreds or thousands per second • Isolates Failure reasons become obvious • Repeatable In any order, any time • Self-validating No manual execution required • Timely Written before code • Immutable* SUT is in the same state after execute the tests • Trusted* When the test fails, the system fail and when the test works, the system works
  25. 25. Integrationtestwhichworkswithexternalsystem Database
  26. 26. IntegrationtestwhichusestheUI Database
  27. 27. Systemtest Database
  28. 28. Who, when and where run the tests? • Unit • Owner: developer • When: after every change • Where: every computer • Integration • Owner: developer || QA team • When: as part or after commit stage • Where: devel and pre-pro environments • System • Owner: QA team • When: as part or after commit stage • Where: devel and pre-pro environments
  29. 29. STRATEGIES
  30. 30. Static evaluation •Informal review •Formal review (inspection) • Checklists • Sucessive abstraction •Walkthrough
  31. 31. Dynamic evaluation • White box • Path Coverage • Statement Coverage • Condition Coverage • Function Coverage • Black box • Equivalence partitioning • Boundary values analysis
  32. 32. White box (*-coverage) 1. Get flow diagram of the SUT 2. Calculate cyclomatic complexity 3. Determine a data set which force going one path or another 4. Exercise the SUT with this dataset. ... errors = [] if (user.name ==null || user.email == null) { errors.push('mandatory fields not found'); } //do the rest of whatever for (var i=0; i < user.friends ; i++) { errors.push(checkFriendShipt(user.friends[i])) } ...
  33. 33. White box (*-coverage) 1. Get flow diagram of the SUT 2. Calculate cyclomatic complexity 3. Determine a data set which force going one path or another 4. Exercise the SUT with this dataset. a b c d …x ... errors = [] if (user.name ==null || user.email == null) { errors.push('mandatory fields not found'); } //do the rest of whatever for (var i=0; i < user.friends ; i++) { errors.push(checkFriendShipt(user.friends[i])) } ...
  34. 34. White box (*-coverage) 1. Get flow diagram of the SUT 2. Calculate cyclomatic complexity 3. Determine a data set which force going one path or another 4. Exercise the SUT with this dataset. edges – nodes + 2 = predicate nodes +1 = number of regions a b c d …x ... errors = [] if (user.name ==null || user.email == null) { errors.push('mandatory fields not found'); } //do the rest of whatever for (var i=0; i < user.friends ; i++) { errors.push(checkFriendShipt(user.friends[i])) } ...
  35. 35. Black box (partitioning) 1. Identify equivalence classes 2. Select dataset: 1. Assign a unique value for every class 2. Select tests cases which cover the most valid classes 3. Select tests cases which cover only one invalid class at the same time
  36. 36. Black box (partitioning) Register Username* Password (6-10 chars including numbers)
  37. 37. Black box (partitioning) Register Username* Password (6-10 chars including numbers) Username Password U1: myNick P1: p4ssw0rd U2: “empty” P2: p4ss P3: l4rg3p4ssw0rd P4: password
  38. 38. Black box (partitioning) Register Username* Password (6-10 chars including numbers) Username Password U1: myNick P1: p4ssw0rd U2: “empty” P2: p4ss P3: l4rg3p4ssw0rd P4: password Test Cases myNick, p4ssw0rd √ myNick, p4ss X myNick, l4rg3p4ssw0rd X myNick, password X “empty”, p4ssw0rd X
  39. 39. AUTOMATIC TESTING
  40. 40. 4 phases-tests 1.Set Up 2.Exercise 3.Verify 4.TearDown
  41. 41. Testing frameworks families • X-Unit • @Before(Class) • @Test • @After(Class) • Rspec • describe • beforeEach • it • afterEach • Specification by example (A.K.A BDD) • Given • When • Then
  42. 42. XUnit @Before public void setUp() { this.userValidator = mock(UserValidator.class); this.userDao = mock(UserDao.class); this.userService = new UserService(userValidator, userDao); } @Test public void createValidUserShouldNotFail() { //Exercise User expectedCreatedUser = new User("irrelevantUser"); when(userValidator.validate(any(User.class))); when(userValidator.validate(any(User.class))).thenReturn(createdUser); User createdUser = userService.create(new User()); //Assertions assertThat(createdUser, equalTo(expectedCreatedUser)); } @Test(expected=ValidationException) public void createInvalidUserShouldFail() { when(userValidator.validate(any(User.class))) .thenReturn(new ValidationException()); userService.create(new User("irrelevantUser")); } @After public void tearDown() { //clean the state here }
  43. 43. Rspec (suite per class) describe('UserService test suite:', function(){ beforeEach(function(){ // setup the SUT }) it('when create a valid user should not fail', function(){ // exercise + assertions }) it('when create an invalid user should fail', function(){ // exercise + assertions }) afterEach(function(){ // clean the state }) }) • UserService test suite: • When create a valid user should not fail √ • When create an invalid user should fail √ The report will look like:
  44. 44. Rspec (suite per setup) describe('UserService test suite:', function(){ describe("when create a valid user ", function() { beforeEach(function(){ // setup and exercise }) it('should return valid user', function(){ // partial assertions }) it('should call validator', function(){ // partial assertions }) it('should call dao', function(){ // partial assertions }) afterEach(function(){ // clean the state }) }) })
  45. 45. BDD (specification) Feature: User registration Scenario: User tries to register sending valid data so the system will create new account Given the user has introduced <username> and <password> into the registration form And has accepted terms and agreements When send the registration from Then the user with <username> should be created Example: | username | password | | myNick | p4ssw0rd | Scenario: User tries to register sending invalid data so the system will reject user Given the user has introduced <username> and <password> into the registration form And has accepted terms and agreements When send the registration from Then the system should notify <error> Example: | username | password | error | | myNick | p4ss | password should have at least 6 characters | | myNick | l4rg3p4ssword | password should have at less than 10 characters | | myNick | password | password should contains at least a number | | | p4ssword | username is mandatory |
  46. 46. BDD(implementation) @given("the user has introduced (w)+ and (w)+ into the registration form") public void populateForm(String username, String password) { ... } @given("has accepted terms and agreements") public void acceptTerms() { ... } @when("send the registration from") public void sendRegistrationForm() { ... } @then("the user with (w)+ should be created") public void verifyUserIsCreated(String username) { ... } @then("the system should notify <error>") public void verifyErrors(String error) { ... }
  47. 47. TESTABLE DESIGN
  48. 48. Non-Testable design smells (by Misko Hevery*) •Constructors does Real Work •Digging into collaborators •Brittle Global State & Singletons •Class Does Too Much Work *See http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf
  49. 49. Constructors does Real Work • new keyword in a constructor or at field declaration • Static method calls in a constructor or at field declaration • Anything more than field assignment in constructors • Object not fully initialized after the constructor finishes (watch out for initialize methods) • Control flow (conditional or looping logic) in a constructor • CL does complex object graph construction inside a constructor rather than using a factory or builder • Adding or using an initialization block
  50. 50. Digging into collaborators • Objects are passed in but never used directly (only used to get access to other objects) • Law of Demeter violation: method call chain walks an object graph with more than one dot (.) • Suspicious names: context, environment, principal, container, or manager
  51. 51. Brittle Global State & Singletons • Adding or using singletons • Adding or using static fields or static methods • Adding or using static initialization blocks • Adding or using registries • Adding or using service locators
  52. 52. Class Does Too Much Work • Summing up what the class does includes the word “and” • Class would be challenging for new team members to read and quickly “get it” • Class has fields that are only used in some methods • Class has static methods that only operate on parameters
  53. 53. Questions & Stupid questions • ¿Where I place my tests? • ¿Who tests the classes which test our classes? • ¿Could you be able to rewrite the code only reading the tests definitions? • I spend more time writing code to setup my SUT than writing the test, how do you solve it? • ¿What is the minimum coverage should I expect for my code? • I’ve never write a test ¿where can I start? • My code is not testable at all, ¿what can I do?
  54. 54. ¿Where I place my tests? • Unit tests: • Test Class per Class • Test Class per SetUp (useful in Xunit frameworks) • Important naming convention (<ClassName>Test, <TestSuite>IntegrationTest, …) • System tests: • Different project
  55. 55. ¿Where I place my tests? Java Project (Test Class per Class) MyProject/ src/ main/ java/ com.groupId.artifactId.MyClass.java resources/ test/ java/ com.groupId.artifactId.MyClassTest.java com.groupId.artifactId.it.suite.MyTestCaseIntegrationTest.java resources/ NodeJs Project MyProject/ lib/ myClass.js main.js test/ ut/ /suite it/ lib/ myClassTest.js Java Project (Class per SetUp) MyProject/ src/ main/ … test/ java/ com.groupId.artifactId.myclass.<SetUp1>Test.java com.groupId.artifactId.myclass.<SetUp2>Test.java …
  56. 56. ¿Where I place my tests? Android Project MyProject/ AndroidManifest.xml res/ ... (resources for main application) src/ ... (source code for main application) ... tests/ AndroidManifest.xml res/ ... (resources for tests) src/ ... (source code for tests) IOS Project MyIOSProject/ MyIOSProject/ ... app code ... MyIOSProjectTests/ ... test code ...
  57. 57. ¿Who tests the classes which test our classes? • Exactly, this is why it’s so important our tests follow KISS
  58. 58. ¿Couldyoube able to rewrite the codeonly reading the tests definitions? • Tests (specially Black Box tests) should tell us an story. • Use descriptive name methods for unit tests: • User well defined, and complete scenarios for system tests: • Use business vocabulary for acceptance tests: public void testValidaterUser1 { ... } VS public void validateUserWithNoPasswordShouldThrowsError { ... } com.mycompany.artifactId.it.TestSteps ... VS com.mycompany.artifactId.it.usermanagement.UserCreationSteps ...
  59. 59. IspendmoretimewritingcodetosetupmySUTthan writingthetest,howdo you solveit? • Read about Fixtures (Xunit Patterns is a good reference) • Fresh fixtures • Shared fixtures • Persistent fixtures
  60. 60. Iduplicatetoomuchcodeonobjectscreation,mocks definitionandassertion… • Writing a lot of code to initialize value objects? • Create DataBuilders • Writing a lot of code to initialize mock/stub objects? • Create MockBuilders • Writing a lot of asserts (more purist says only one assertion)? • Create CustomAsserts User user = userDataBuilder.createValidUser(); VS User user = new User("irrelevantUsername", "v4l1dP4ss", irrelevant@myCompany.com", ...); assertNotNull(user.getUsername()); assertNotNull(user.getPassword()); assertNotNull(user.getEmail()); ... VS assertContainsAllMandatoryData(user);
  61. 61. ¿What is the minimum coverage should I expect for my code? • It depends on the project. • “… test as little as possible to reach a given level of confidence …” • Do not get obsess over test coverage, it’s a metric, not a goal.
  62. 62. I’ve never write a test ¿where can I start? Database PORT1 PORT2 ADAPTER1 I’ll bet you a beer , you called it *Util…
  63. 63. My code is not testable at all, ¿what can I do? • First of all, go to http://refactoring.com/ • I suggest: 1. Add integration regression test. 2. Remove new from methods and ad it to constructors (this will prepare your class for dependency injection). 3. Creates a constructor which receive every dependency your class will need. 4. Remove static classes and methods (adding the new non-static as a class dependency). 5. Add as much tests as you want to ;) Important!!! Do it step by step
  64. 64. Recommended reading • Growing Object Oriented Software Guided Tests • Xunit Patterns • Continuous delivery • Hexagonal Architecture • Inversion of Control Container and the Dependency Injection pattern • Mocks aren’t Stubs • Misko Hevery blog • http://refactoring.com/ • …
  65. 65. Place your question here!

×