SOFTWARE TESTING
Get the bugs out!
1
Why Test Software?
2
 Software testing enables making objective assessments
regarding the degree of conformance of the system to stated
requirements and specifications. Testing verifies that the
system meets the different requirements including, functional,
performance, reliability, security, usability and so on.
Objective Assessment: So What?
3
 Consider the role of product owner (individual or company)
 What is their responsibility regarding software?
 When to release it?
 What level or risks are tolerable?
 To answer these questions what do they need to know?
 How many bugs exist?
 What’s the likelihood of a bug happening to users
 What could that bug do to users?
 Cost of removing bugs vs cost of lost revenue vs brand
damage costs
More than finding bugs
4
 It is important for software testing to verify and validate that
the product meets the stated requirements / specifications.

What is the value of good software?
5
 Good will with customers  more revenue
 Word of mouth  sell more
 Trust  buy more
 Reduces support costs
 Allows development team to work on adding more value (new
features and feature improvements
 Greater employee morale
Types of Software Development Tests
7
 Unit testing
 Integration testing
 Functional testing
Unit testing
8
 Unit tests are used to test individual code components and
ensure that code works the way it was intended to.
 Unit tests are written and executed by developers.
 Most of the time a testing framework like JUnit or TestNG is
used.
 Test cases are typically written at a method level and executed
via automation.
Integration Testing
9
 Integration tests check if the system as a whole works.
 Integration testing is also done by developers, but rather than
testing individual components, it aims to
test across components.
 A system consists of many separate components like code,
database, web servers, etc. (more than JUST CODE)
 Integration tests are able to spot issues like wiring of
components, network access, database issues, etc.
Functional Testing
10
 Functional tests check that each feature is implemented
correctly by comparing the results for a given input against the
specification.
 Typically, this is not done at a developer level.
 Functional tests are executed by a separate testing team.
 Test cases are written based on the specification and the actual
results are compared with the expected results.
 Several tools are available for automated functional testing
like Selenium and QTP.
UNIT TESTING
11
Today's Goal
 Functions/methods are the basic building block of programs.
 Today, you'll learn to test functions, and how to design testable
functions.
12
What is a unit test?
13
 A unit test is a piece of code that invokes a unit of work and
checks one specific end result of that unit of work. If the
assumptions on the end result turn out to be wrong, the unit
test has failed. A unit test’s scope can span as little as a
method or as much as multiple classes.
Testing Functions
 Amy is writing a program. She adds functionality by defining a
new function, and adding a function call to the new function in
the main program somewhere. Now she needs to test the new
code.
 Two basic approaches to testing the new code:
1. Run the entire program
 The function is tested when it is invoked during the program
execution
2. Test the function by itself, somehow
Which do you think will be more efficient?
14
Manual Unit Test
 A unit test is a program that tests
a function to determine if it works
properly
 A manual unit test
 Gets test input from the user
 Invokes the function on the test
input
 Displays the result of the function
 How would you use a unit test to
determine that roundIt() works
correctly?
 How does the test - debug cycle
go?
def roundIt(num: float) -> int:
return int(num + .5)
# ---- manual unit test ----
x = float(input('Enter a number:'))
result = roundIt(x)
print('Result: ', result)
15
Automated Unit Test
 An automated unit test
 Invokes the function on
predetermined test input
 Checks that the function
produced the expected result
 Displays a pass / fail
message
 Advantages?
 Disadvantages?
def roundIt(num: float) -> int:
return int(num + .5)
# ---- automated unit test ----
result = roundIt(9.7)
if result == 10:
print("Test 1: PASS")
else:
print("Test 1: FAIL")
result = roundIt(8.2)
if result == 8:
print("Test 2: PASS")
else:
print("Test 2: FAIL")
16
Automated Unit Test with assert
 The assert statement
 Performs a comparison
 If the comparison is True,
does nothing
 If the comparison is False,
displays an error and stops
the program
 assert statements help us
write concise, automated
unit tests
 Demo: Run the test
def roundIt(num: float) -> int:
return int(num + .5)
# ---- automated unit test ----
result = roundIt(9.7)
assert result == 10
result = roundIt(8.2)
assert result == 8
print("All tests passed!")
17
A Shorter Test
 This is Nice.
 Two issues:
 To test the function, we have
to copy it between the "real
program" and this unit test
 Assertion failure messages
could be more helpful
To solve these, we'll use pytest,
a unit test framework
def roundIt(num: float) -> int:
return int(num + .5)
# ---- automated unit test ----
assert roundIt(9.7) == 10
assert roundIt(8.2) == 8
print("All tests passed!")
18
A pytest Unit Test
 To create a pytest Unit Test:
 Define a unit test function named
test_something
 Place one or more assertions
inside
 These tests can be located in the
same file as the "real program," as
long as you put the real program
inside a special if statement, as
shown
 To run a pytest Unit Test from
command prompt:
 pytest program.py
 Note: pytest must first be
installed...
def roundIt(num: float) -> int:
return int(num + .5)
def test_roundIt():
assert roundIt(9.7) == 10
assert roundIt(8.2) == 8
if __name__ == "__main__":
# --- "real program" here ---
19
What's with this if statement?
if __name__ == "__main__":
# --- "real program" here ---
 This if condition above is true when the
program is executed using the python
command:
 python myprog.py
 The if condition is false when the program is
executed using pytest
 pytest myprog.py
We don't want pytest to execute the main
program...
20
pytest Unit Tests
 A pytest Unit Test can have
more than one test method
 It's good practice to have a
separate test method for
each type of test
 That way, when a test fails,
you can debug the issue
more easily
def roundIt(num: float) -> int:
return int(num + .5)
# ---- automated unit test ----
def test_roundIt_rounds_up():
assert roundIt(9.7) == 10
def test_roundIt_rounds_down():
assert roundIt(8.2) == 8
21
Designing Testable Functions
 Some functions can't be
tested with an automated
unit test.
 A testable function
 Gets input from parameters
 Returns a result
 Does no input / output
# This function is not testable
def roundIt(num: float) -> int:
print(int(num + .5))
22
Python unit test tutorial
35

Upstate CSCI 540 Unit testing

  • 1.
  • 2.
    Why Test Software? 2 Software testing enables making objective assessments regarding the degree of conformance of the system to stated requirements and specifications. Testing verifies that the system meets the different requirements including, functional, performance, reliability, security, usability and so on.
  • 3.
    Objective Assessment: SoWhat? 3  Consider the role of product owner (individual or company)  What is their responsibility regarding software?  When to release it?  What level or risks are tolerable?  To answer these questions what do they need to know?  How many bugs exist?  What’s the likelihood of a bug happening to users  What could that bug do to users?  Cost of removing bugs vs cost of lost revenue vs brand damage costs
  • 4.
    More than findingbugs 4  It is important for software testing to verify and validate that the product meets the stated requirements / specifications. 
  • 5.
    What is thevalue of good software? 5  Good will with customers  more revenue  Word of mouth  sell more  Trust  buy more  Reduces support costs  Allows development team to work on adding more value (new features and feature improvements  Greater employee morale
  • 6.
    Types of SoftwareDevelopment Tests 7  Unit testing  Integration testing  Functional testing
  • 7.
    Unit testing 8  Unittests are used to test individual code components and ensure that code works the way it was intended to.  Unit tests are written and executed by developers.  Most of the time a testing framework like JUnit or TestNG is used.  Test cases are typically written at a method level and executed via automation.
  • 8.
    Integration Testing 9  Integrationtests check if the system as a whole works.  Integration testing is also done by developers, but rather than testing individual components, it aims to test across components.  A system consists of many separate components like code, database, web servers, etc. (more than JUST CODE)  Integration tests are able to spot issues like wiring of components, network access, database issues, etc.
  • 9.
    Functional Testing 10  Functionaltests check that each feature is implemented correctly by comparing the results for a given input against the specification.  Typically, this is not done at a developer level.  Functional tests are executed by a separate testing team.  Test cases are written based on the specification and the actual results are compared with the expected results.  Several tools are available for automated functional testing like Selenium and QTP.
  • 10.
  • 11.
    Today's Goal  Functions/methodsare the basic building block of programs.  Today, you'll learn to test functions, and how to design testable functions. 12
  • 12.
    What is aunit test? 13  A unit test is a piece of code that invokes a unit of work and checks one specific end result of that unit of work. If the assumptions on the end result turn out to be wrong, the unit test has failed. A unit test’s scope can span as little as a method or as much as multiple classes.
  • 13.
    Testing Functions  Amyis writing a program. She adds functionality by defining a new function, and adding a function call to the new function in the main program somewhere. Now she needs to test the new code.  Two basic approaches to testing the new code: 1. Run the entire program  The function is tested when it is invoked during the program execution 2. Test the function by itself, somehow Which do you think will be more efficient? 14
  • 14.
    Manual Unit Test A unit test is a program that tests a function to determine if it works properly  A manual unit test  Gets test input from the user  Invokes the function on the test input  Displays the result of the function  How would you use a unit test to determine that roundIt() works correctly?  How does the test - debug cycle go? def roundIt(num: float) -> int: return int(num + .5) # ---- manual unit test ---- x = float(input('Enter a number:')) result = roundIt(x) print('Result: ', result) 15
  • 15.
    Automated Unit Test An automated unit test  Invokes the function on predetermined test input  Checks that the function produced the expected result  Displays a pass / fail message  Advantages?  Disadvantages? def roundIt(num: float) -> int: return int(num + .5) # ---- automated unit test ---- result = roundIt(9.7) if result == 10: print("Test 1: PASS") else: print("Test 1: FAIL") result = roundIt(8.2) if result == 8: print("Test 2: PASS") else: print("Test 2: FAIL") 16
  • 16.
    Automated Unit Testwith assert  The assert statement  Performs a comparison  If the comparison is True, does nothing  If the comparison is False, displays an error and stops the program  assert statements help us write concise, automated unit tests  Demo: Run the test def roundIt(num: float) -> int: return int(num + .5) # ---- automated unit test ---- result = roundIt(9.7) assert result == 10 result = roundIt(8.2) assert result == 8 print("All tests passed!") 17
  • 17.
    A Shorter Test This is Nice.  Two issues:  To test the function, we have to copy it between the "real program" and this unit test  Assertion failure messages could be more helpful To solve these, we'll use pytest, a unit test framework def roundIt(num: float) -> int: return int(num + .5) # ---- automated unit test ---- assert roundIt(9.7) == 10 assert roundIt(8.2) == 8 print("All tests passed!") 18
  • 18.
    A pytest UnitTest  To create a pytest Unit Test:  Define a unit test function named test_something  Place one or more assertions inside  These tests can be located in the same file as the "real program," as long as you put the real program inside a special if statement, as shown  To run a pytest Unit Test from command prompt:  pytest program.py  Note: pytest must first be installed... def roundIt(num: float) -> int: return int(num + .5) def test_roundIt(): assert roundIt(9.7) == 10 assert roundIt(8.2) == 8 if __name__ == "__main__": # --- "real program" here --- 19
  • 19.
    What's with thisif statement? if __name__ == "__main__": # --- "real program" here ---  This if condition above is true when the program is executed using the python command:  python myprog.py  The if condition is false when the program is executed using pytest  pytest myprog.py We don't want pytest to execute the main program... 20
  • 20.
    pytest Unit Tests A pytest Unit Test can have more than one test method  It's good practice to have a separate test method for each type of test  That way, when a test fails, you can debug the issue more easily def roundIt(num: float) -> int: return int(num + .5) # ---- automated unit test ---- def test_roundIt_rounds_up(): assert roundIt(9.7) == 10 def test_roundIt_rounds_down(): assert roundIt(8.2) == 8 21
  • 21.
    Designing Testable Functions Some functions can't be tested with an automated unit test.  A testable function  Gets input from parameters  Returns a result  Does no input / output # This function is not testable def roundIt(num: float) -> int: print(int(num + .5)) 22
  • 22.
    Python unit testtutorial 35