Testing in the lifecycle

TESTING IN THE LIFECYCLE
Hoang V. Nguyen
startnewday85@gmail.com
startnewday85.blogspot.com
Dept. of Computer Science – FIT - HUA

Tester Training Course

User Acceptance
requirements Testing

System
System Testing
requirements

Integration
Global Design
Testing

Detailed Design Unit Testing

implementation

Tester Training Course 2011

 Also know as component, module, program testing
 Each “part” is tested in isolation
 lowest level and most thorough look at detail
 Goals:
• to ensure that the code written for the unit meets its specification,
prior to its integration with other units
• both functional and non-functional characteristics

 Test bases:
• component requirements
• detailed design
• the code


 Who do it?
• usually done by programmer

 Reports
• Defects are typically fixed as soon as they are found, with out
formally recording the incidents found

 Tools:
• Programmer often writes pieces of code for unit test
• Mock frameworks
• Dependency Injection and IoC Containers
• Unit test frameworks


 Programmers:
• have to create pieces of code are separately testable

 Test Driven Development
• An approach to software development using Unit Test
• Test –first approach
• is an iterative approach to unit test
• is good in practice
A A Driver
• Stubs and Drivers

B Stubs B


 more than two components/systems
 Goals:
• discover defects in interface and interaction between
components/systems
• what the set can perform that is not possible individually

 Test bases:
• software and system design
• software architecture
• workflows and use-cases


 There may be more than one level of integration
testing
• component integration testing
• system integration testing

 Who do it?
• Developer
• Designer
• Independent tester


 Integration strategy
• big-bang
• incremental


 In theory
• if we have already tested components why not just combine them
all at once? Wouldn’t this save time?
• based on false assumption of no faults

 In practice: difficult to isolate any errors found
• takes longer to locate and fix faults
• re-testing after fixes more extensive
==> don’t save time


 In theory
• start with a tested component/system
• increment: add a tested component/system and test it
• baselines

 Advantages
• easier fault location and fix
• easier recovery from disaster/problems

 Some types
• top-down
• bottom-up
• functional

 Baselines
• baseline 0: a tested component a
a
• baseline 1: a+b b c
• baseline 2: a+b+c
• baseline 3: a+b+c+d d e f g
• …

 Need to call to lower level h i j k l m
components not yet integrated
n o Stubs
 Stubs: simulate missing
components


 Advantages:
• critical control structure tested first and most often
• can demonstrate system early (show working
menus)
 Disadvantages:
• needs stubs
• detail left until last
• may be difficult to "see" detailed output (but
should have been tested in component test)
• may look more finished than it is


 Baselines Drivers
• baseline 0: component n
a
• baseline 1: n+i b c
• baseline 2: n+i+o
• baseline 3: n+i+o+d d e f g
• …

 Needs driver to call the h i j k l m
base line configuration
 Also needs stubs for some n o
baselines
 Drivers: test harness/scaffolding
Stubs

 Advantages:
• lowest levels tested first and most thoroughly (but should
have been tested in unit testing)
• good for testing interfaces to external environment
(hardware, network)
• visibility of detail

 Disadvantages
• no working system until last baseline
• needs both drivers and stubs
• major control problems found last


 Baselines
• baseline 0: component a
a
• baseline 1: a+b b c
• baseline 2: a+b+d
• baseline 3: a+b+d+h d e f g
• …

 Need Stubs h i j k l m
 Shouldn’t need drivers(if top
down) n o


 Advantages:
• control level tested first and most often
• visibility of detail
• real working partial system earliest

 Disadvantages
• needs stubs


Thread Integration
 order of processing some event
a
determines integration order
 interrupt, user transaction b c
 minimum capability in time
d e f g
 advantages:
• critical processing first h i j k l m
• early warning of
performance problems
n o
 disadvantages:
• may need complex drivers and stubs

Integration Guidelines
 minimise support software needed

 integrate each component only once

 each baseline should produce an easily
verifiable result

 integrate small numbers of components at once
• one at a time for critical or fault-prone components
• combine simple related components


 integration should be planned in the
architectural design phase

 the integration order then determines the
build order
• components completed in time for their
baseline
• component development and integration
testing can be done in parallel - saves time


Question
Which of the following statements about
component testing is not true?

Component testing should be performed by
developer

Component testing is alse know as isolation or
module testing

Component testing should have completion criteria
planned

Component testing does not involve regression
testing


Question
Which of the following combinations correctly describes a
valid aproach to component testing
i. Functional testing of the component in isolation
ii. Structure-based testing of the code with out recording
incidents
iii. Automated tests that rare run until the component
passes
iv. Functional testing of the interfaces between modules

i and ii

i, ii and iii

iii

ii and iv


Question
Testing activity which is performed to expose
defects in the interfaces and in the interaction
between integrated components is:

System Testing

Component Testing

Integration Testing

Unit Testing


Question
A common test technique during
component test is

Statement and branch testing

Usability testing

Security testing

Performance testing


Question
The process starting with the terminal modules
is called

Top-down integration

Module integration

Bottom-up integration

None of the above


Question
Why is incremental integration preferred
over big bang integration?

Because incremental integration has better early
defects screening and isolation ability

Because big bang integration is suitable only for real
time applications

Incremental integration is preferred over big bang
integration only for bottom-up development model

Because incremental integration can compensate
for weak and inadequate component testing


 Last integration step
 end-to-end perspective
 Goals:
• discover failure in behaviors of whole system in environment

 Test bases
• System and software requirement specifications
• Use cases
• system, user and operation manuals

 Who do it
• often and should be independent test team


• Usability
• configuration / installation
• Security
• reliability / qualities
• Documentation
• back-up / recovery
• Storage
• performance, load, stress
• volume


Performance Tests
 Timing Tests
• response and service times
• database back-up times
 Capacity & Volume Tests
• maximum amount or processing rate
• number of records on the system
• graceful degradation
 Endurance Tests (24-hr operation?)
• robustness of the system
• memory allocation


Multi-User Tests
 Concurrency Tests
• small numbers, large benefits
• detect record locking problems
 Load Tests
• the measurement of system behaviour under
realistic multi-user load
 Stress Tests
• go beyond limits for the system - know what will
happen
• particular relevance for e-commerce


Usability Tests
 messages tailored and meaningful to (real)
users?
 coherent and consistent interface?
 sufficient redundancy of critical information?
 within the "human envelope"? (7±2 choices)
 feedback (wait messages)?
 clear mappings (how to escape)?
Who should design / perform these tests?


Security Tests
 passwords
 encryption
 hardware permission devices
 levels of access to information
 authorisation
 covert channels
 physical security


Configuration and Installation
 Configuration Tests
• different hardware or software environment
• configuration of the system itself
• upgrade paths - may conflict
 Installation Tests
• distribution (CD, network, etc.) and timings
• physical aspects: electromagnetic fields, heat,
humidity, motion, chemicals, power supplies
• uninstall (removing installation)


Reliability / Qualities
 Reliability
• "system will be reliable" - how to test this?
• "2 failures per year over ten years"
• Mean Time Between Failures (MTBF)
• reliability growth models
 Other Qualities
• maintainability, portability, adaptability, etc.


Back-up and Recovery
 Back-ups
• computer functions
• manual procedures (where are tapes stored)
 Recovery
• real test of back-up
• manual procedures unfamiliar
• should be regularly rehearsed
• documentation should be detailed, clear and
thorough


Documentation Testing
 Documentation review
• check for accuracy against other documents
• gain consensus about content
• documentation exists, in right format
 Documentation tests
• is it usable? does it work?
• user manual
• maintenance documentation


 Final stage of validation
 Goals
• To provide the end users with confidence
that the system will function according to
their expectations
 Test bases
• Requirement s specification
 Who do it
• Customers/users
• Also can include testers


 Done by users
 To check that the system meets
their business needs
 Can include
• Factory acceptance testing
• Site acceptance testing


 Also called operational readiness testing
 To check that the processes and procedures
are in place to allow the system to be used
and maintained
 Include checking:
• Back-up facilities
• Procedures for disaster recovery
• Training for end users
• Maintenance procedures
• Data load and migration tasks
• Security procedures


 The criteria for accepting a system can
documented in a contact
 To check that these criteria have been met
• Not what the user wish they had asked for!
• This system, not wish system


Check that governmental, legal or safety standard
have been met


 Similarities:
• When software is stable
• To collect comments
 fault found
 how the product meets their expectations
 improvement suggestions

 Differences :
• Alpha testing is done on developer site
• Beta testing is done on customer site


Question
System Integration testing should be
done after

Integration testing

System testing

Unit testing

Component integration testing


Question
When is beta testing?

Testing performed by potential customers at the
developers location.

Testing performed by portential customer at their
own locations.

Testing performed by product developers at the
customer ’s location

Testing performed by product developers at their
own locations.


Question
Which testing is used to verify that the
system can perform properly when internal
program or system limitations have been
exceeed

Stress testing

Load testing

Performance testing

Volume testing


 Additional features being required
 The system being migrated to a new
operating platform
 The system being retired
 Planned upgrade to COTS-based system
 New faults being found requiring fixing
(these can be ‘hot fixes’)


Maintenance testing is important!


 To ensure that software system still
works correctly (both new and old
characteristics)
 We have to:
• Breath test to establish overall confidence
• Depth test to investigate changes and critical areas

 Predominantly regression testing


What to test in maintenance testing
 Test any new or changed code
 Impact analysis
• what could this change have an impact on?
• how important is a fault in the impacted area?
• test what has been affected, but how much?
most important affected areas?
areas most likely to be affected?
whole system?
 The answer: “It depends”


Question
Which of the following is correct?

Impact Analysis assesses the effect on the system of
a defect found in regression testing

Impact Analysis assesses the effect of a new person
joining the regression test team

Impact Analysis assesses whether or not a defect
found in regression testing has been fixed correctly

Impact Analysis assesses the effect of a change to
the system to determine how much regression to do


Question
Testing where in we subject the target of the
test, to varying workloads to measure and
evaluate the performance behaviors and
ability of the target and of the test to
continue to function properly under these
different workloads.

Unit testing

Load testing

Usability testing

Intergration testing


Question
Methodologies adopted while
performing maintenance testing:

Breadth Test and Depth Test

Re-Testing

Confirmation Testing

Sanity testing


Question
What is the normal order of activities in
which software testing is orgranized?

Unit, integration, system, validation

System, integration, unit, validation

Unit, integration, validation, system

None of the above


Question
Beta testing is performed at developing
organization’s site where as Alpha testing is
performed by people at their own locations.

True

False


Question
Beta testing is

Performed by customers at their own site

Performed by customers at their software
developer’s site

Performed by an independent test team

Useful for test bespoke software


Question
Impact Analysis helps to decide

Exit Criteria

How much regression testing should be done

How many more test cases need to written

Different tools to perform regression testing


Testing in the lifecycle

Recommended

Recommended

More Related Content

What's hot

What's hot (17)

Viewers also liked

Viewers also liked (7)

Similar to Testing in the lifecycle

Similar to Testing in the lifecycle (20)

More from Hoang Nguyen

More from Hoang Nguyen (20)

Recently uploaded

Recently uploaded (20)

Testing in the lifecycle