SlideShare a Scribd company logo
ISTQB Advanced Level 2012 –
Test Manager
Objectives
 Build on ISTQB CTFL in the area of test manager
 Prepare for the ISTQB CTAL – Test Manager exam
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 2
Prerequisites
 ISTQB CTFL or equivalent
 Practical experience in SW testing
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 3
Notes
 Ask any time.
 Turn your cell silent.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 4
References
 ISTQB CTAL – TM syllabus version 2012
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 5
Outline
 Testing Process
 Test Management
 Reviews
 Defect Management
 Improving the Test Process
 Test Tools and Automation
 People Skills – Team Composition
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 6
Outline
 Testing Process
 Test Management
 Reviews
 Defect Management
 Improving the Test Process
 Test Tools and Automation
 People Skills – Team Composition
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 7
Testing Process
 Introduction
 Test Planning, Monitoring and Control
 Test Analysis
 Test Design
 Test Implementation
 Test Execution
 Evaluating Exit Criteria and Reporting
 Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 8
Testing Process
 Introduction
 Test Planning, Monitoring and Control
 Test Analysis
 Test Design
 Test Implementation
 Test Execution
 Evaluating Exit Criteria and Reporting
 Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 9
Fundamental Test Process in ISTQB
CTFL Syllabus
 Can be executed:
 Sequentially
 In parallel (exploratory)
 Or overlapping
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 10
P
A
&
D
I
&
E
E
&
R
C
C
Refinement of Fundamental Test
Process in ISTQB CTAL Syllabi
 Activities are considered separately in order:
 To provide additional refinement and optimization of the processes
 To better fit the software development lifecycle
 To facilitate effective test monitoring and control
 Tailoring these main activities within the context of the system and the
project is usually required.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 11
Refinement of Fundamental Test
Process in ISTQB CTAL Syllabi cont’d
 It is important to understand the other steps in the test process.
 Majority of the TM’s work usually is done during the planning, monitoring,
and control of the testing project.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 12
P
A D I E E & R
C
M & C
Testing Process
 Introduction
 Test Planning, Monitoring and Control
 Test Analysis
 Test Design
 Test Implementation
 Test Execution
 Evaluating Exit Criteria and Reporting
 Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 13
Learning Objectives
 TM-1.2.1 (K4) Analyze the test needs for a system in order to plan test
activities and work products that will achieve the test objectives
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 14
Test Planning
 Identification of:
 Activities and resources to meet mission and
objectives defined in test strategy
 Methods for gathering and tracking metrics that:
 Guide the project
 Determine adherence to the plan
 Assess achievement of objectives
 Tools, training and documentation guidelines based on
me metrics defined in the planning
 For each level, starts @ test process initiation and
continues till closure activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 15
Test Strategy Affects Test Planning
 Strategy(ies) determine tasks that should occur during the planning.
 In risk-based testing strategy , risk analysis guides planning regarding
mitigating activities of identified product risks and help contingency
planning.
 Test effort allocation
 Testing levels needed
 Test activities prioritization
 Test techniques selection
 Need for static testing
 In a reactive strategy, planning for the creation of test charters and tools
for dynamic testing techniques such as exploratory testing may be
warranted.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 16
Examples: Risk-Analysis Guiding Test
Planning
 If a number of likely and serious potential security defects identified, a
significant amount of effort should be spent developing and executing
security tests.
 If serious defects are usually found in the design specification, the test
planning process could result in additional static testing (reviews) of the
design specification.
 If performance is a high risk, performance testing may be conducted as
soon as integrated code is available.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 17
Example: DO-178 B Needed Coverage
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 18
 Used in an airborne environment
Failure Level Description Testing Technique
A: Catastrophic
Failure may cause lack of critical function
needed to safely fly or land the plane
MC/DC
B: Hazardous
Failure may have a large –ve impact on safety
or performance
Decision Testing
and MC/DC
optional
C: Major
Failure is significant, but less serious than A or
B
Statement testing
@ minimum
D: Minor
Failure is noticeable, but with less impact than
C
E: No effect Failure has no impact on safety
Test Management PoV
Test Basis, Test Conditions, and Test
Cases Complex Relationship
 Need to be understood for effective
planning, monitoring and control
 Affects tool decisions
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 19
Test Development PoV
Test Object
Test Items
Test Basis
Test Conditions
Test Cases
Test Procedures
Test
Analysis
Test
Design
Test
Implem
entation
Relation with the Development Team
Impacts Test Planning
 Development work products affect traceability and traceability matrix
requirements.
 Development work products must be approved before specific testing
activities can start.
 Development life cycle model determines which and how information
should be exchanged between development and testing teams.
 Test plan lists SW features both in and outside testing scope w/
appropriate project formality and documentation.
 TM defines initial test environment specification
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 20
Test Planning Considers Dependencies
 External dependencies and associated SLA’s must be identified and initial
contact should be made.
 Example dependencies:
 Outside groups
 Other projects
 External vendors or development partners
 Deployment team
 Database administrators
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 21
Example: Test Planning in Sequential
Models
 Has many test planning issues
that must be considered:
1. Quality and delivery trade-off
due to schedule compression @
the end
2. Unstable or untestable SW
delivered to test team
3. Failure to include all activities
shown in the model
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 22
Example: Test Planning in Iterative
Models
 Has many test planning issues
that must be considered:
1. Regression testing
2. Planning for bugs
3. Lack of rigorous testing
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 23
Example: Test Planning in Agile
Models (Iterative Models ++)
Challenges
 Volume and speed o change
 Keeping w/ the pace
 Increased regression risk
 Inadequate unit testing
 Poor test oracles and test basis
 Scrum or scrums
 Over expectation
Opportunities
 Automated unit testing
 Static code analysis
 Code coverage
 Continuous integration
 Automated functional tests
 Informal reviews and power o 3
 Control of technical debt
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 24
Example: Test Planning in Spiral
Models
 Has many test planning issues
that must be considered:
1. Testing must be very flexible
2. Testing objective varies along
the development lifecycle (early
testing is for knowing the
unknown not for finding
defects).
3. Unpredictable schedule
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 25
Test Monitoring
 Providing feedback and visibility about all test activities
 Efficient test control requires defining a schedule and monitoring
framework.
 This framework should include detailed measures and targets needed to
relate test work products and activities to plan and strategic objectives.
 Easy for small or less complex projects
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 26
Defining Monitoring Objectives and
Targets
 Test monitoring framework must be defined a manner that is
understandable and relevant to the project and business stakeholders
(residual risks vs. defects and tests cases for example).
 Test conditions are a key player as they link test basis to test work
products.
 Properly configured traceability wit properly defined test conditions make
the complex relationships more transparent and comprehensible.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 27
Involvement of Stakeholders in Defining
Monitoring Frameworks
 Sometimes, stakeholders require targets that are not related to system
functionality or specifications (common if no or lacking formal
documentation).
 Involvement stakeholders @ early stage help define measures and targets
that provide better control and influence testing activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 28
Example: Quality Objectives as Entry
Criteria
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 29
Test Control
 Ongoing comparing of actual progress against plan and implementing
corrective actions when needed
 Guides testing to fulfill mission, strategies, and objectives
 Appropriate reactions to the control data depend (decision taking, test re-
prioritizing, test rescheduling or changing entry criteria) on detailed
planning information (traceability between test basis, test conditions and
test work products).
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 30
Exercise: Test Planning, Monitoring
and Control
 Assume, you are a TM who is planning test for a project that will follow
iterative SDLC.
 Your requirements are prioritized into 5 levels (VH, H, M, L, VL).
 Follow the ISTQB fundamental test process and outline the test activities
across each iteration.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 31
Testing Process
 Introduction
 Test Planning, Monitoring and Control
 Test Analysis
 Test Design
 Test Implementation
 Test Execution
 Evaluating Exit Criteria and Reporting
 Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 32
Learning Objectives
 TM-1.3.1 (K3) Use traceability to check completeness and consistency of
defined test conditions with respect to the test objectives, test strategy,
and test plan
 TM-1.3.2 (K2) Explain the factors that might affect the level of detail at
which test conditions may be specified and the advantages and
disadvantages for specifying test conditions at a detailed level
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 33
Test Analysis
 Test analysis = defining “what” to test (test conditions)
 Test design = defining “how” to test (test cases)
 Test analysis and design can be implemented as parallel, integrated, or
iterative activities.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 34
Test Conditions
 Identified by analysis of the test basis, test objectives, and product risks
 Viewed as detailed measures and targets for success (exit criteria)
 Test conditions coverage, test conditions achievement % ...
 Should bi-directionally traceable
 To test basis, test objectives, risks or any other project or stakeholder criteria
for success
 To test design and resulting test work products
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 35
When and How Test Analysis?
 As soon as test basis is established for a test level (test basis and test
conditions will differ according to test level)
 Formal test techniques and analytical techniques can identify test
conditions
 Test conditions may vary in level o detail.
 Specifying test conditions in a detailed fashion will tend to result in a
larger number of test conditions.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 36
How much Detail in Test Conditions?
 Answer depends on some factors:
 Testing level
 Detail and quality of test basis
 System/software complexity
 Risks
 The relationship between the test basis, what is to be tested and how it is to
be tested
 SDLC in use
 Test management tool being utilized
 Level at which test design and other test work products are to be specified
and documented
 Skills and knowledge of the test analysts
 The level of maturity of the test process and the organization itself
 Availability of other project stakeholders for consultation
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 37
Exercise: Test Analysis
 Perform a complete test analysis for the given screenshot.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 38
Detailed Test Conditions Advantages
 Facilitates more flexibility in relating other test work products to test basis
and test objectives yielding better monitor and control eventually
 Contributes to defects prevention
 Relates testing work products to stakeholders in terms that they can
understand
 Helps influence and direct testing and development activities
 More efficient coverage of detailed measures and targets
 Basis for clearer horizontal traceability within a test level
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 39
Detailed Test Conditions Usage
 Effective in the following situations:
 Lightweight test design documentation methods are being used to
accommodate SDLC, cost and/or time constraints or other factors
 Little or no formal requirements or other development work products are
available as the test basis
 The project is large-scale, complex or high risk
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 40
Disadvantages of Detailed Test
Conditions
 Potentially time-consuming
 Maintainability can become difficult in a changing environment.
 Level of formality needs to be defined and implemented across the team.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 41
General Test Conditions Usage
 Effective in the following situations:
 CT
 Less complex projects where simple hierarchical relationships exist between
what is to be tested and how it is to be tested
 AT where use cases can be utilized to help define tests
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 42
Testing Process
 Introduction
 Test Planning, Monitoring and Control
 Test Analysis
 Test Design
 Test Implementation
 Test Execution
 Evaluating Exit Criteria and Reporting
 Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 43
Learning Objectives
 TM-1.4.1 (K3) Use traceability to check completeness and consistency of
designed test cases with respect to the defined test conditions
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 44
Test Implementation
 Fulfillment of the test design
 Checking against explicit and implicit entry criteria
 Creating automated tests
 Organizing tests (both manual and automated) into execution order
 Finalizing test data and test environment
 Forming a test execution schedule, including resource allocation
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 45
Test Design
 Test design = defining “how” to test (test cases) by stepwise elaboration of
the identified test conditions or test basis using test techniques defined
during test planning
 Test design includes test data design and test environment design as well.
 After the TA’s do their work, TM must check the design exit criteria before
moving to the next step.
 Test design exit criteria include review and approval from relevant
stakeholders, coverage, cost, time ...
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 46
Test Design Traceability
 Test cases can be related directly to test basis or indirectly through test
conditions (preferably detailed ones).
 Test basis items could be individual requirements (atomic?!), risk item,
strategic objectives, test objectives or project or stakeholder success
criterion.
 Traceability resolution loss should be avoided.
 Must be bi-directional to allow TM to relate test results in terms of project
aspects
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 47
Inputs and Outputs for Test Design
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 48
Test Plan
Test
Conditions
Test
strategy
Quality
Risks
Test
Oracles
Concrete
or
Logical?
Test Cases w/
Traceability
Test Data
Design
Test
Environment
Design
TA’s
Concrete or Logical Test Cases?
Concrete Test Cases
 Specific information
 Preconditions, inputs, outputs, post
conditions, and procedures
 Useful when:
 Requirements are well-defined.
 Testers are inexperienced
 External verification like audits is
needed.
 Benefit is excellent reproducibility.
 Maintenance is a significant issue and
limits testers creativity.
Logical Test Cases
 Guidelines for what should be tested
 TAs can vary data and/or procedure
 Useful when:
 Requirements are not well-defined.
 Testers are experienced in testing and
the product.
 Formality is not needed.
 Benefits are better coverage due to
variation and lower costs of creation
and maintenance.
 Reproducibility is a significant issue.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 49
Sequential or Iterative Test
Development Process?
 Decision is influenced by the test level and test strategy.
 In ST, SIT and AT, it is more sequential as test basis are available much
earlier than the test execution start, while in CIT and CT, it is more
iterative.
 Analytical strategies are sequential while reactive tend are iterative by
nature.
 Even in sequential development, some overlap may be beneficial
especially test data generation after test design and before test
implementation can reveal problems in the test design.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 50
Example: IEC 61508 Test Implications
 Risk-analysis is required.
 Risk-based testing is a mandatory risk mitigation technique.
 All risk-relevant test work products must be formally documented.
 SIL mandates selection of test techniques, required structural coverage
and test levels to be used.
 Reviews are mandatory even for test work products.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 51
Exercise: Test Design
 Derive minimal number of test cases from previously defined test
conditions.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 52
Testing Process
 Introduction
 Test Planning, Monitoring and Control
 Test Analysis
 Test Design
 Test Implementation
 Test Execution
 Evaluating Exit Criteria and Reporting
 Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 53
Learning Objectives
 TM-1.5.1 (K3) Use risks, prioritization, test environment and data
dependencies, and constraints to develop a test execution schedule which
is complete and consistent with respect to the test objectives, test
strategy, and test plan
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 54
Test Implementation
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 55
Test Execution Schedule
 Tests must be scheduled and sequenced during test implementation
taking into considerations:
 Order of automated and manual test execution
 Prioritization assigned to tests from requirements or risks
 Execution constraints like tester or environment availability or test
environment reconfiguration
 TM should carefully review schedule to avoid any test execution blockage.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 56
Ready for Test Execution?
 Trust, but verify the test environment and test data.
 If the test environment to evolve during execution, double check all changes
that will occur in accordance with the test execution schedule.
 Contingencies should be defined if these anticipated changes did not occur for
a reason or another.
 Tests must be reviewed and verified before running.
 Double check execution schedule to make sure every thing will be in place.
 Make sure SW is ready for testing.
 Smoke tests are effective in checking the test execution readiness.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 57
Example: Test Execution Entry Criteria
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 58
Detail and Complexity
 Same argument considered for test conditions apply for test cases, test
data description and test environment description.
 Using testware as a way of system documentation, testware reuse or
having testers w/ different skill set mandates documenting more details.
 Regulatory rules may also mandates documentation details.
 Project complexity can also determine level of details needed in
implementation.
 TM must defined implementation level of details to guide testers during
implementation.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 59
Early Test Implementation Pros
 Concrete tests provide worked examples of how SW should behave, if
written in accordance with test basis.
 Business domain experts verify concrete tests easier than abstract
business rules, and thereby identify further weaknesses in SW
specifications.
 Verified tests may provide illuminating illustrations of required behavior
for SW designers and developers.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 60
Early Test Implementation Cons
 Changing test basis may lead to change in the testware already developed.
 In Agile SDLC, embracing changes is the norm!.
 In sequential SDLC, poorly managed projects can lead to unhappy
surprises.
 Before embarking on an extensive test implementation, it is wise to
understand SDLC and the predictability of SW features that will be
available for testing.
 In conclusion, late implementation may be favored.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 61
Testing Process
 Introduction
 Test Planning, Monitoring and Control
 Test Analysis
 Test Design
 Test Implementation
 Test Execution
 Evaluating Exit Criteria and Reporting
 Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 62
Learning Objectives
 TM-1.6.1 (K3) Use traceability to monitor test progress for completeness
and consistency with the test objectives, test strategy, and test plan
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 63
Test Execution
 Begins once test object is delivered and entry criteria to test execution are
satisfied (or waived carefully)
 Criteria examples are:
 Tests should be designed
 Tools should be in place
 Test results tracking should be working and tracked data is understandable by
all team members.
 Standards for test logging and defect reporting should be available and
published.
 Bottom line, do whatever it takes to make execution proceeds efficiently.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 64
Blending Reactive Test Strategies
 10-20% of testing should be reactive (even more in case formal testing is
automated).
 Blending offsets weaknesses in test strategies as they complement each other.
 Reactive testing blend is done by giving permissions to explore other options
during formal execution.
 Time boxing can help.
 Experienced testers should do experience- and defect-based testing more.
 Time boxing and chartered testing can help.
 Reproducibility for unscripted test should be considered.
 Capture-play back tools can help.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 65
TM Role in Test Execution
 Being there all day during test execution
when needed unless he is in meetings
 Daily team debriefing of progress with the
entire team
 Take proper control actions if needed
 Checks and ensures proper bi-directional
traceability carried out by the team
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 66
Pass 2 Defects Status Count
Verified 47
To be verified 1
Needs approval 5
Deferred 21
Failed verification 3
Not a defect 8
New 13
Total 103
Exercise: Metrics
 Describe one or more metric that you can use to report residual risks
during test execution.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 67
Testing Process
 Introduction
 Test Planning, Monitoring and Control
 Test Analysis
 Test Design
 Test Implementation
 Test Execution
 Evaluating Exit Criteria and Reporting
 Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 68
Learning Objectives
 TM-1.7.1 (K2) Explain the importance of accurate and timely information
collection during the test process to support accurate reporting and
evaluation against exit criteria
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 69
Testing as a Source of Information
 Testing produces lots of information (relevant or not).
 TM must ensure that right information is captured and delivered to
appropriate stakeholders.
 TM should be able to relate the test results to the exit criteria.
 Test management tools must be properly selected and in place to support
accurately and timely gather information so as to facilitate effective
evaluation and reporting.
 TM should never use gathered info for personal appraisals.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 70
Proper Information Collection
 Requires clear definition of what to be collected, by whom and when
 Should be documented during test planning
 Frequent check on gathered information is a must (never assume they are
OK).
 Team should be trained and aware of how to gather/report information w/
or w/o tools.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 71
Reporting Frequency and Detail
 There is no one size fits all reporting approach.
 Close monitoring of exit criteria does not necessitate detailed reporting.
 Close monitoring = proper control
 Reporting = providing supporting information
 Details and type of reporting should differ according to recipient
stakeholders.
 They need different information to do their work.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 72
Testing Process
 Introduction
 Test Planning, Monitoring and Control
 Test Analysis
 Test Design
 Test Implementation
 Test Execution
 Evaluating Exit Criteria and Reporting
 Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 73
Learning Objectives
 TM-1.8.1 (K2) Summarize the four groups of test closure activities
 TM-1.8.2 (K3) Implement a project retrospective to evaluate processes
and discover areas to improve
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 74
Test Closure Activities
 Once execution is complete, key outputs should be captured and either
passed to the relevant person or archived.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 75
Test Closure
Activities
Test
Completion
Check
Test Artifacts
Handover
Lessons
Learned
Archiving
Test Closure Activities cont’d
 These tasks are important, often missed, and should be explicitly included
as part of the test plan.
 It is common for one or more of these tasks to be omitted due to
premature reassignment or dismissal of project team members, resource
or schedule pressures on subsequent projects, or team burnout.
 On projects carried out under contract, such as custom development, the
contract should specify the tasks required.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 76
Example: Test Closure Checklists
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 77
Test Completion
• All planned tests
run or
deliberately
skipped?
• All known
defects fixed,
deferred or
accepted?
Test Artifacts
Handover
• Known defects
communicated
to operation
and support
team?
• Test and
environment to
maintenance
team?
• Regression tests
documented?
Lessons Learned
• Were relevant
stakeholders
involved in risk
analysis?
• Were estimates
accurate?
• RCA done and
actions defined?
• Process
improvement?
• Any
unanticipated
deviation?
Archiving
• All test work
product
archived in CM?
• If the project to
be reopened, do
we have every
thing we need?
Project Retrospective
 A meeting held at the end of each iteration/increment
 To discuss:
 What went well?
 What can be improved?
 How to improve?
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 78
Exercise: Retrospective
 Assume that 25% of defects opened on a certain feature were rejected as
not actual problems.
 If rejection rate is more than 5%, is unacceptable.
 List 5 actions, that could investigate the root cause of the high defect
rejection rate.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 79
Summary
 Introduction
 Test Planning, Monitoring and Control
 Test Analysis
 Test Design
 Test Implementation
 Test Execution
 Evaluating Exit Criteria and Reporting
 Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 80
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 81

More Related Content

What's hot

Chapter 1 - Fundamentals of Testing
Chapter 1 - Fundamentals of TestingChapter 1 - Fundamentals of Testing
Chapter 1 - Fundamentals of TestingNeeraj Kumar Singh
 
Chapter 4 - Test Design Techniques
Chapter 4 - Test Design TechniquesChapter 4 - Test Design Techniques
Chapter 4 - Test Design TechniquesNeeraj Kumar Singh
 
Chapter 2 - Testing Throughout the Development LifeCycle
Chapter 2 - Testing Throughout the Development LifeCycleChapter 2 - Testing Throughout the Development LifeCycle
Chapter 2 - Testing Throughout the Development LifeCycleNeeraj Kumar Singh
 
Software Testing Life Cycle (STLC) | Software Testing Tutorial | Edureka
Software Testing Life Cycle (STLC) | Software Testing Tutorial | EdurekaSoftware Testing Life Cycle (STLC) | Software Testing Tutorial | Edureka
Software Testing Life Cycle (STLC) | Software Testing Tutorial | EdurekaEdureka!
 
Chapter 6 - Test Tools and Automation
Chapter 6 - Test Tools and AutomationChapter 6 - Test Tools and Automation
Chapter 6 - Test Tools and AutomationNeeraj Kumar Singh
 
Software test management overview for managers
Software test management overview for managersSoftware test management overview for managers
Software test management overview for managersTJamesLeDoux
 
Software testing life cycle
Software testing life cycleSoftware testing life cycle
Software testing life cycleNikhil Sharma
 
Test Management introduction
Test Management introductionTest Management introduction
Test Management introductionOana Feidi
 
Chapter 4 - Quality Characteristics for Technical Testing
Chapter 4 - Quality Characteristics for Technical TestingChapter 4 - Quality Characteristics for Technical Testing
Chapter 4 - Quality Characteristics for Technical TestingNeeraj Kumar Singh
 
ISTQB - Foundation level testing topics
ISTQB - Foundation level testing topicsISTQB - Foundation level testing topics
ISTQB - Foundation level testing topicsShan Kings
 
ISTQB Test level, Test type
ISTQB Test level, Test typeISTQB Test level, Test type
ISTQB Test level, Test typeHoangThiHien1
 

What's hot (20)

Chapter 3 - Reviews
Chapter 3 - ReviewsChapter 3 - Reviews
Chapter 3 - Reviews
 
Chapter 1 - Fundamentals of Testing
Chapter 1 - Fundamentals of TestingChapter 1 - Fundamentals of Testing
Chapter 1 - Fundamentals of Testing
 
Chapter 4 - Test Design Techniques
Chapter 4 - Test Design TechniquesChapter 4 - Test Design Techniques
Chapter 4 - Test Design Techniques
 
Chapter 2 - Testing Throughout the Development LifeCycle
Chapter 2 - Testing Throughout the Development LifeCycleChapter 2 - Testing Throughout the Development LifeCycle
Chapter 2 - Testing Throughout the Development LifeCycle
 
Chapter 3 - Test Automation
Chapter 3 - Test AutomationChapter 3 - Test Automation
Chapter 3 - Test Automation
 
Chapter 4 - Defect Management
Chapter 4 - Defect ManagementChapter 4 - Defect Management
Chapter 4 - Defect Management
 
Chapter 2 - Test Management
Chapter 2 - Test ManagementChapter 2 - Test Management
Chapter 2 - Test Management
 
Software Testing Life Cycle (STLC) | Software Testing Tutorial | Edureka
Software Testing Life Cycle (STLC) | Software Testing Tutorial | EdurekaSoftware Testing Life Cycle (STLC) | Software Testing Tutorial | Edureka
Software Testing Life Cycle (STLC) | Software Testing Tutorial | Edureka
 
Chapter 6 - Test Tools and Automation
Chapter 6 - Test Tools and AutomationChapter 6 - Test Tools and Automation
Chapter 6 - Test Tools and Automation
 
Istqb chapter 5
Istqb chapter 5Istqb chapter 5
Istqb chapter 5
 
Chapter 1 - Testing Process
Chapter 1 - Testing ProcessChapter 1 - Testing Process
Chapter 1 - Testing Process
 
Software test management overview for managers
Software test management overview for managersSoftware test management overview for managers
Software test management overview for managers
 
Software testing life cycle
Software testing life cycleSoftware testing life cycle
Software testing life cycle
 
Test Management introduction
Test Management introductionTest Management introduction
Test Management introduction
 
Chapter 4 - Quality Characteristics for Technical Testing
Chapter 4 - Quality Characteristics for Technical TestingChapter 4 - Quality Characteristics for Technical Testing
Chapter 4 - Quality Characteristics for Technical Testing
 
ISTQB Test Process
ISTQB Test ProcessISTQB Test Process
ISTQB Test Process
 
Chapter 5 - Reviews
Chapter 5 - ReviewsChapter 5 - Reviews
Chapter 5 - Reviews
 
ISTQB - Foundation level testing topics
ISTQB - Foundation level testing topicsISTQB - Foundation level testing topics
ISTQB - Foundation level testing topics
 
Chapter 2 - Testing in Agile
Chapter 2 - Testing in AgileChapter 2 - Testing in Agile
Chapter 2 - Testing in Agile
 
ISTQB Test level, Test type
ISTQB Test level, Test typeISTQB Test level, Test type
ISTQB Test level, Test type
 

Viewers also liked

Free-ebook-rex-black advanced-software-testing
Free-ebook-rex-black advanced-software-testingFree-ebook-rex-black advanced-software-testing
Free-ebook-rex-black advanced-software-testingQualister
 
FiSTB - agile testing
FiSTB - agile testingFiSTB - agile testing
FiSTB - agile testingKari Kakkonen
 
Der Test Manager ist tot - lang lebe der Test Master
Der Test Manager ist tot - lang lebe der Test MasterDer Test Manager ist tot - lang lebe der Test Master
Der Test Manager ist tot - lang lebe der Test MasterSwissQ Consulting AG
 
ISTQB - Advanced Test Manager Certificate
ISTQB - Advanced Test Manager CertificateISTQB - Advanced Test Manager Certificate
ISTQB - Advanced Test Manager CertificateKranthi Ram Nagelli
 
ISTQB Agile Extension
ISTQB Agile ExtensionISTQB Agile Extension
ISTQB Agile ExtensionDavis Thomas
 
Decision Testing and Decision Coverage. ISTQB Whitebox techniques with TestCo...
Decision Testing and Decision Coverage. ISTQB Whitebox techniques with TestCo...Decision Testing and Decision Coverage. ISTQB Whitebox techniques with TestCo...
Decision Testing and Decision Coverage. ISTQB Whitebox techniques with TestCo...Radoslaw Smilgin
 
Statement Testing and Statement Coverage. ISTQB whitebox techniques with Test...
Statement Testing and Statement Coverage. ISTQB whitebox techniques with Test...Statement Testing and Statement Coverage. ISTQB whitebox techniques with Test...
Statement Testing and Statement Coverage. ISTQB whitebox techniques with Test...Radoslaw Smilgin
 

Viewers also liked (20)

Istqb ctal tm
Istqb ctal tmIstqb ctal tm
Istqb ctal tm
 
Free-ebook-rex-black advanced-software-testing
Free-ebook-rex-black advanced-software-testingFree-ebook-rex-black advanced-software-testing
Free-ebook-rex-black advanced-software-testing
 
ISTQB REX BLACK book
ISTQB REX BLACK bookISTQB REX BLACK book
ISTQB REX BLACK book
 
ISTQB Foundation Agile Tester 2014 Training, Agile SW Development
ISTQB Foundation Agile Tester 2014 Training, Agile SW DevelopmentISTQB Foundation Agile Tester 2014 Training, Agile SW Development
ISTQB Foundation Agile Tester 2014 Training, Agile SW Development
 
Introduction to Embedded Systems a Practical Approach
Introduction to Embedded Systems a Practical ApproachIntroduction to Embedded Systems a Practical Approach
Introduction to Embedded Systems a Practical Approach
 
ISTQB Technical Test Analyst 2012 Training - Structure-Based Testing
ISTQB Technical Test Analyst 2012 Training - Structure-Based TestingISTQB Technical Test Analyst 2012 Training - Structure-Based Testing
ISTQB Technical Test Analyst 2012 Training - Structure-Based Testing
 
Introduction to Software Test Automation
Introduction to Software Test AutomationIntroduction to Software Test Automation
Introduction to Software Test Automation
 
ISTQB Technical Test Analyst 2012 Training - The Technical Test Analyst's Tas...
ISTQB Technical Test Analyst 2012 Training - The Technical Test Analyst's Tas...ISTQB Technical Test Analyst 2012 Training - The Technical Test Analyst's Tas...
ISTQB Technical Test Analyst 2012 Training - The Technical Test Analyst's Tas...
 
Android Booting Scenarios
Android Booting ScenariosAndroid Booting Scenarios
Android Booting Scenarios
 
ISTQB-Test Manager
ISTQB-Test ManagerISTQB-Test Manager
ISTQB-Test Manager
 
FiSTB - agile testing
FiSTB - agile testingFiSTB - agile testing
FiSTB - agile testing
 
Der Test Manager ist tot - lang lebe der Test Master
Der Test Manager ist tot - lang lebe der Test MasterDer Test Manager ist tot - lang lebe der Test Master
Der Test Manager ist tot - lang lebe der Test Master
 
ISTQB - Advanced Test Manager Certificate
ISTQB - Advanced Test Manager CertificateISTQB - Advanced Test Manager Certificate
ISTQB - Advanced Test Manager Certificate
 
ISTQB Agile Extension
ISTQB Agile ExtensionISTQB Agile Extension
ISTQB Agile Extension
 
Decision Testing and Decision Coverage. ISTQB Whitebox techniques with TestCo...
Decision Testing and Decision Coverage. ISTQB Whitebox techniques with TestCo...Decision Testing and Decision Coverage. ISTQB Whitebox techniques with TestCo...
Decision Testing and Decision Coverage. ISTQB Whitebox techniques with TestCo...
 
Statement Testing and Statement Coverage. ISTQB whitebox techniques with Test...
Statement Testing and Statement Coverage. ISTQB whitebox techniques with Test...Statement Testing and Statement Coverage. ISTQB whitebox techniques with Test...
Statement Testing and Statement Coverage. ISTQB whitebox techniques with Test...
 
FreeRTOS Course - Queue Management
FreeRTOS Course - Queue ManagementFreeRTOS Course - Queue Management
FreeRTOS Course - Queue Management
 
Simulation Using Isim
Simulation Using Isim Simulation Using Isim
Simulation Using Isim
 
Introduction to stm32-part1
Introduction to stm32-part1Introduction to stm32-part1
Introduction to stm32-part1
 
Fpga programming
Fpga programmingFpga programming
Fpga programming
 

Similar to ISTQB Advanced Test Manager Training 2012 - Testing Process

Testing Metrics: Project, Product, Process
Testing Metrics: Project, Product, ProcessTesting Metrics: Project, Product, Process
Testing Metrics: Project, Product, ProcessTechWell
 
Quality contamination in agile scrum teams and the remedies - The QA Strategy
Quality contamination in agile scrum teams and the remedies - The QA StrategyQuality contamination in agile scrum teams and the remedies - The QA Strategy
Quality contamination in agile scrum teams and the remedies - The QA StrategySumeet Gupta, CSP, SAFe Agilist (SA)
 
Stefanini.trinh
Stefanini.trinhStefanini.trinh
Stefanini.trinhNASAPMC
 
Episode 24 : Project Quality Management
Episode 24 :  Project Quality ManagementEpisode 24 :  Project Quality Management
Episode 24 : Project Quality ManagementSAJJAD KHUDHUR ABBAS
 
2015 - RT & AWAY Intro Pack
2015 - RT & AWAY Intro Pack2015 - RT & AWAY Intro Pack
2015 - RT & AWAY Intro PackMatt Rendoth
 
Free CAPM Exam Sample Question of the Week
Free CAPM Exam Sample Question of the WeekFree CAPM Exam Sample Question of the Week
Free CAPM Exam Sample Question of the WeekOSP International LLC
 
Process certification
Process certificationProcess certification
Process certificationJean Denis
 
Planning And Monitoring The Process
Planning And Monitoring The ProcessPlanning And Monitoring The Process
Planning And Monitoring The Processahmad bassiouny
 
2 a. project auditing
2 a. project auditing2 a. project auditing
2 a. project auditingDr.R. SELVAM
 
Project Methodology - On The Fly
Project Methodology - On The FlyProject Methodology - On The Fly
Project Methodology - On The Flyibenari
 
Smsvn pmp training_slides_chapter_8
Smsvn pmp training_slides_chapter_8Smsvn pmp training_slides_chapter_8
Smsvn pmp training_slides_chapter_8Nam Trung
 
9.process improvement chapter 9
9.process improvement chapter 99.process improvement chapter 9
9.process improvement chapter 9Warui Maina
 
Project-Planning
Project-PlanningProject-Planning
Project-PlanningRon Drew
 

Similar to ISTQB Advanced Test Manager Training 2012 - Testing Process (20)

Testing Metrics: Project, Product, Process
Testing Metrics: Project, Product, ProcessTesting Metrics: Project, Product, Process
Testing Metrics: Project, Product, Process
 
Project audit
Project auditProject audit
Project audit
 
Quality contamination in agile scrum teams and the remedies - The QA Strategy
Quality contamination in agile scrum teams and the remedies - The QA StrategyQuality contamination in agile scrum teams and the remedies - The QA Strategy
Quality contamination in agile scrum teams and the remedies - The QA Strategy
 
Stefanini.trinh
Stefanini.trinhStefanini.trinh
Stefanini.trinh
 
csc 510 Project
csc 510 Projectcsc 510 Project
csc 510 Project
 
Project managment 7
Project managment  7Project managment  7
Project managment 7
 
Episode 24 : Project Quality Management
Episode 24 :  Project Quality ManagementEpisode 24 :  Project Quality Management
Episode 24 : Project Quality Management
 
Analytical Risk-based and Specification-based Testing - Bui Duy Tam
Analytical Risk-based and Specification-based Testing - Bui Duy TamAnalytical Risk-based and Specification-based Testing - Bui Duy Tam
Analytical Risk-based and Specification-based Testing - Bui Duy Tam
 
2015 - RT & AWAY Intro Pack
2015 - RT & AWAY Intro Pack2015 - RT & AWAY Intro Pack
2015 - RT & AWAY Intro Pack
 
Free CAPM Exam Sample Question
Free CAPM Exam Sample QuestionFree CAPM Exam Sample Question
Free CAPM Exam Sample Question
 
CAPM Exam Sample Q and A
CAPM Exam Sample Q and ACAPM Exam Sample Q and A
CAPM Exam Sample Q and A
 
Free CAPM Exam Sample Question of the Week
Free CAPM Exam Sample Question of the WeekFree CAPM Exam Sample Question of the Week
Free CAPM Exam Sample Question of the Week
 
Process certification
Process certificationProcess certification
Process certification
 
Planning And Monitoring The Process
Planning And Monitoring The ProcessPlanning And Monitoring The Process
Planning And Monitoring The Process
 
2 a. project auditing
2 a. project auditing2 a. project auditing
2 a. project auditing
 
Project Methodology - On The Fly
Project Methodology - On The FlyProject Methodology - On The Fly
Project Methodology - On The Fly
 
Session 12 4th edition PMP
Session  12 4th edition PMPSession  12 4th edition PMP
Session 12 4th edition PMP
 
Smsvn pmp training_slides_chapter_8
Smsvn pmp training_slides_chapter_8Smsvn pmp training_slides_chapter_8
Smsvn pmp training_slides_chapter_8
 
9.process improvement chapter 9
9.process improvement chapter 99.process improvement chapter 9
9.process improvement chapter 9
 
Project-Planning
Project-PlanningProject-Planning
Project-Planning
 

More from Amr Ali (ISTQB CTAL Full, CSM, ITIL Foundation) (12)

Introduction to state machines in Embedded Software Design
Introduction to state machines in Embedded Software DesignIntroduction to state machines in Embedded Software Design
Introduction to state machines in Embedded Software Design
 
Embedded SW Testing
Embedded SW TestingEmbedded SW Testing
Embedded SW Testing
 
Cracking the interview
Cracking the interviewCracking the interview
Cracking the interview
 
Embedded linux network device driver development
Embedded linux network device driver developmentEmbedded linux network device driver development
Embedded linux network device driver development
 
Embedded summer camps 2017
Embedded summer camps 2017Embedded summer camps 2017
Embedded summer camps 2017
 
Introduction to C Programming
Introduction to C ProgrammingIntroduction to C Programming
Introduction to C Programming
 
Introduction to embedded systems
Introduction to embedded systemsIntroduction to embedded systems
Introduction to embedded systems
 
Introduction to stm32-part2
Introduction to stm32-part2Introduction to stm32-part2
Introduction to stm32-part2
 
Synthesis Using ISE
Synthesis Using ISESynthesis Using ISE
Synthesis Using ISE
 
Simulation using model sim
Simulation using model simSimulation using model sim
Simulation using model sim
 
FreeRTOS Course - Semaphore/Mutex Management
FreeRTOS Course - Semaphore/Mutex ManagementFreeRTOS Course - Semaphore/Mutex Management
FreeRTOS Course - Semaphore/Mutex Management
 
Free FreeRTOS Course-Task Management
Free FreeRTOS Course-Task ManagementFree FreeRTOS Course-Task Management
Free FreeRTOS Course-Task Management
 

Recently uploaded

Benefits of Employee Monitoring Software
Benefits of  Employee Monitoring SoftwareBenefits of  Employee Monitoring Software
Benefits of Employee Monitoring SoftwareMera Monitor
 
How to install and activate eGrabber JobGrabber
How to install and activate eGrabber JobGrabberHow to install and activate eGrabber JobGrabber
How to install and activate eGrabber JobGrabbereGrabber
 
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1KnowledgeSeed
 
Agnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in KrakówAgnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in Krakówbim.edu.pl
 
Breaking the Code : A Guide to WhatsApp Business API.pdf
Breaking the Code : A Guide to WhatsApp Business API.pdfBreaking the Code : A Guide to WhatsApp Business API.pdf
Breaking the Code : A Guide to WhatsApp Business API.pdfMeon Technology
 
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?XfilesPro
 
CompTIA Security+ (Study Notes) for cs.pdf
CompTIA Security+ (Study Notes) for cs.pdfCompTIA Security+ (Study Notes) for cs.pdf
CompTIA Security+ (Study Notes) for cs.pdfFurqanuddin10
 
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
KLARNA -  Language Models and Knowledge Graphs: A Systems ApproachKLARNA -  Language Models and Knowledge Graphs: A Systems Approach
KLARNA - Language Models and Knowledge Graphs: A Systems ApproachNeo4j
 
OpenChain @ LF Japan Executive Briefing - May 2024
OpenChain @ LF Japan Executive Briefing - May 2024OpenChain @ LF Japan Executive Briefing - May 2024
OpenChain @ LF Japan Executive Briefing - May 2024Shane Coughlan
 
JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)Max Lee
 
10 Essential Software Testing Tools You Need to Know About.pdf
10 Essential Software Testing Tools You Need to Know About.pdf10 Essential Software Testing Tools You Need to Know About.pdf
10 Essential Software Testing Tools You Need to Know About.pdfkalichargn70th171
 
AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAlluxio, Inc.
 
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...Abortion Clinic
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandIES VE
 
Designing for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web ServicesDesigning for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web ServicesKrzysztofKkol1
 
Workforce Efficiency with Employee Time Tracking Software.pdf
Workforce Efficiency with Employee Time Tracking Software.pdfWorkforce Efficiency with Employee Time Tracking Software.pdf
Workforce Efficiency with Employee Time Tracking Software.pdfDeskTrack
 
GraphAware - Transforming policing with graph-based intelligence analysis
GraphAware - Transforming policing with graph-based intelligence analysisGraphAware - Transforming policing with graph-based intelligence analysis
GraphAware - Transforming policing with graph-based intelligence analysisNeo4j
 
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...Alluxio, Inc.
 
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product UpdatesGraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product UpdatesNeo4j
 

Recently uploaded (20)

5 Reasons Driving Warehouse Management Systems Demand
5 Reasons Driving Warehouse Management Systems Demand5 Reasons Driving Warehouse Management Systems Demand
5 Reasons Driving Warehouse Management Systems Demand
 
Benefits of Employee Monitoring Software
Benefits of  Employee Monitoring SoftwareBenefits of  Employee Monitoring Software
Benefits of Employee Monitoring Software
 
How to install and activate eGrabber JobGrabber
How to install and activate eGrabber JobGrabberHow to install and activate eGrabber JobGrabber
How to install and activate eGrabber JobGrabber
 
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
 
Agnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in KrakówAgnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in Kraków
 
Breaking the Code : A Guide to WhatsApp Business API.pdf
Breaking the Code : A Guide to WhatsApp Business API.pdfBreaking the Code : A Guide to WhatsApp Business API.pdf
Breaking the Code : A Guide to WhatsApp Business API.pdf
 
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
 
CompTIA Security+ (Study Notes) for cs.pdf
CompTIA Security+ (Study Notes) for cs.pdfCompTIA Security+ (Study Notes) for cs.pdf
CompTIA Security+ (Study Notes) for cs.pdf
 
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
KLARNA -  Language Models and Knowledge Graphs: A Systems ApproachKLARNA -  Language Models and Knowledge Graphs: A Systems Approach
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
 
OpenChain @ LF Japan Executive Briefing - May 2024
OpenChain @ LF Japan Executive Briefing - May 2024OpenChain @ LF Japan Executive Briefing - May 2024
OpenChain @ LF Japan Executive Briefing - May 2024
 
JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)
 
10 Essential Software Testing Tools You Need to Know About.pdf
10 Essential Software Testing Tools You Need to Know About.pdf10 Essential Software Testing Tools You Need to Know About.pdf
10 Essential Software Testing Tools You Need to Know About.pdf
 
AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning Framework
 
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New Zealand
 
Designing for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web ServicesDesigning for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web Services
 
Workforce Efficiency with Employee Time Tracking Software.pdf
Workforce Efficiency with Employee Time Tracking Software.pdfWorkforce Efficiency with Employee Time Tracking Software.pdf
Workforce Efficiency with Employee Time Tracking Software.pdf
 
GraphAware - Transforming policing with graph-based intelligence analysis
GraphAware - Transforming policing with graph-based intelligence analysisGraphAware - Transforming policing with graph-based intelligence analysis
GraphAware - Transforming policing with graph-based intelligence analysis
 
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
 
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product UpdatesGraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
 

ISTQB Advanced Test Manager Training 2012 - Testing Process

  • 1. ISTQB Advanced Level 2012 – Test Manager
  • 2. Objectives  Build on ISTQB CTFL in the area of test manager  Prepare for the ISTQB CTAL – Test Manager exam 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 2
  • 3. Prerequisites  ISTQB CTFL or equivalent  Practical experience in SW testing 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 3
  • 4. Notes  Ask any time.  Turn your cell silent. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 4
  • 5. References  ISTQB CTAL – TM syllabus version 2012 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 5
  • 6. Outline  Testing Process  Test Management  Reviews  Defect Management  Improving the Test Process  Test Tools and Automation  People Skills – Team Composition 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 6
  • 7. Outline  Testing Process  Test Management  Reviews  Defect Management  Improving the Test Process  Test Tools and Automation  People Skills – Team Composition 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 7
  • 8. Testing Process  Introduction  Test Planning, Monitoring and Control  Test Analysis  Test Design  Test Implementation  Test Execution  Evaluating Exit Criteria and Reporting  Test Closure Activities 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 8
  • 9. Testing Process  Introduction  Test Planning, Monitoring and Control  Test Analysis  Test Design  Test Implementation  Test Execution  Evaluating Exit Criteria and Reporting  Test Closure Activities 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 9
  • 10. Fundamental Test Process in ISTQB CTFL Syllabus  Can be executed:  Sequentially  In parallel (exploratory)  Or overlapping 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 10 P A & D I & E E & R C C
  • 11. Refinement of Fundamental Test Process in ISTQB CTAL Syllabi  Activities are considered separately in order:  To provide additional refinement and optimization of the processes  To better fit the software development lifecycle  To facilitate effective test monitoring and control  Tailoring these main activities within the context of the system and the project is usually required. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 11
  • 12. Refinement of Fundamental Test Process in ISTQB CTAL Syllabi cont’d  It is important to understand the other steps in the test process.  Majority of the TM’s work usually is done during the planning, monitoring, and control of the testing project. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 12 P A D I E E & R C M & C
  • 13. Testing Process  Introduction  Test Planning, Monitoring and Control  Test Analysis  Test Design  Test Implementation  Test Execution  Evaluating Exit Criteria and Reporting  Test Closure Activities 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 13
  • 14. Learning Objectives  TM-1.2.1 (K4) Analyze the test needs for a system in order to plan test activities and work products that will achieve the test objectives 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 14
  • 15. Test Planning  Identification of:  Activities and resources to meet mission and objectives defined in test strategy  Methods for gathering and tracking metrics that:  Guide the project  Determine adherence to the plan  Assess achievement of objectives  Tools, training and documentation guidelines based on me metrics defined in the planning  For each level, starts @ test process initiation and continues till closure activities 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 15
  • 16. Test Strategy Affects Test Planning  Strategy(ies) determine tasks that should occur during the planning.  In risk-based testing strategy , risk analysis guides planning regarding mitigating activities of identified product risks and help contingency planning.  Test effort allocation  Testing levels needed  Test activities prioritization  Test techniques selection  Need for static testing  In a reactive strategy, planning for the creation of test charters and tools for dynamic testing techniques such as exploratory testing may be warranted. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 16
  • 17. Examples: Risk-Analysis Guiding Test Planning  If a number of likely and serious potential security defects identified, a significant amount of effort should be spent developing and executing security tests.  If serious defects are usually found in the design specification, the test planning process could result in additional static testing (reviews) of the design specification.  If performance is a high risk, performance testing may be conducted as soon as integrated code is available. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 17
  • 18. Example: DO-178 B Needed Coverage 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 18  Used in an airborne environment Failure Level Description Testing Technique A: Catastrophic Failure may cause lack of critical function needed to safely fly or land the plane MC/DC B: Hazardous Failure may have a large –ve impact on safety or performance Decision Testing and MC/DC optional C: Major Failure is significant, but less serious than A or B Statement testing @ minimum D: Minor Failure is noticeable, but with less impact than C E: No effect Failure has no impact on safety
  • 19. Test Management PoV Test Basis, Test Conditions, and Test Cases Complex Relationship  Need to be understood for effective planning, monitoring and control  Affects tool decisions 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 19 Test Development PoV Test Object Test Items Test Basis Test Conditions Test Cases Test Procedures Test Analysis Test Design Test Implem entation
  • 20. Relation with the Development Team Impacts Test Planning  Development work products affect traceability and traceability matrix requirements.  Development work products must be approved before specific testing activities can start.  Development life cycle model determines which and how information should be exchanged between development and testing teams.  Test plan lists SW features both in and outside testing scope w/ appropriate project formality and documentation.  TM defines initial test environment specification 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 20
  • 21. Test Planning Considers Dependencies  External dependencies and associated SLA’s must be identified and initial contact should be made.  Example dependencies:  Outside groups  Other projects  External vendors or development partners  Deployment team  Database administrators 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 21
  • 22. Example: Test Planning in Sequential Models  Has many test planning issues that must be considered: 1. Quality and delivery trade-off due to schedule compression @ the end 2. Unstable or untestable SW delivered to test team 3. Failure to include all activities shown in the model 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 22
  • 23. Example: Test Planning in Iterative Models  Has many test planning issues that must be considered: 1. Regression testing 2. Planning for bugs 3. Lack of rigorous testing 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 23
  • 24. Example: Test Planning in Agile Models (Iterative Models ++) Challenges  Volume and speed o change  Keeping w/ the pace  Increased regression risk  Inadequate unit testing  Poor test oracles and test basis  Scrum or scrums  Over expectation Opportunities  Automated unit testing  Static code analysis  Code coverage  Continuous integration  Automated functional tests  Informal reviews and power o 3  Control of technical debt 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 24
  • 25. Example: Test Planning in Spiral Models  Has many test planning issues that must be considered: 1. Testing must be very flexible 2. Testing objective varies along the development lifecycle (early testing is for knowing the unknown not for finding defects). 3. Unpredictable schedule 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 25
  • 26. Test Monitoring  Providing feedback and visibility about all test activities  Efficient test control requires defining a schedule and monitoring framework.  This framework should include detailed measures and targets needed to relate test work products and activities to plan and strategic objectives.  Easy for small or less complex projects 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 26
  • 27. Defining Monitoring Objectives and Targets  Test monitoring framework must be defined a manner that is understandable and relevant to the project and business stakeholders (residual risks vs. defects and tests cases for example).  Test conditions are a key player as they link test basis to test work products.  Properly configured traceability wit properly defined test conditions make the complex relationships more transparent and comprehensible. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 27
  • 28. Involvement of Stakeholders in Defining Monitoring Frameworks  Sometimes, stakeholders require targets that are not related to system functionality or specifications (common if no or lacking formal documentation).  Involvement stakeholders @ early stage help define measures and targets that provide better control and influence testing activities 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 28
  • 29. Example: Quality Objectives as Entry Criteria 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 29
  • 30. Test Control  Ongoing comparing of actual progress against plan and implementing corrective actions when needed  Guides testing to fulfill mission, strategies, and objectives  Appropriate reactions to the control data depend (decision taking, test re- prioritizing, test rescheduling or changing entry criteria) on detailed planning information (traceability between test basis, test conditions and test work products). 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 30
  • 31. Exercise: Test Planning, Monitoring and Control  Assume, you are a TM who is planning test for a project that will follow iterative SDLC.  Your requirements are prioritized into 5 levels (VH, H, M, L, VL).  Follow the ISTQB fundamental test process and outline the test activities across each iteration. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 31
  • 32. Testing Process  Introduction  Test Planning, Monitoring and Control  Test Analysis  Test Design  Test Implementation  Test Execution  Evaluating Exit Criteria and Reporting  Test Closure Activities 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 32
  • 33. Learning Objectives  TM-1.3.1 (K3) Use traceability to check completeness and consistency of defined test conditions with respect to the test objectives, test strategy, and test plan  TM-1.3.2 (K2) Explain the factors that might affect the level of detail at which test conditions may be specified and the advantages and disadvantages for specifying test conditions at a detailed level 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 33
  • 34. Test Analysis  Test analysis = defining “what” to test (test conditions)  Test design = defining “how” to test (test cases)  Test analysis and design can be implemented as parallel, integrated, or iterative activities. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 34
  • 35. Test Conditions  Identified by analysis of the test basis, test objectives, and product risks  Viewed as detailed measures and targets for success (exit criteria)  Test conditions coverage, test conditions achievement % ...  Should bi-directionally traceable  To test basis, test objectives, risks or any other project or stakeholder criteria for success  To test design and resulting test work products 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 35
  • 36. When and How Test Analysis?  As soon as test basis is established for a test level (test basis and test conditions will differ according to test level)  Formal test techniques and analytical techniques can identify test conditions  Test conditions may vary in level o detail.  Specifying test conditions in a detailed fashion will tend to result in a larger number of test conditions. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 36
  • 37. How much Detail in Test Conditions?  Answer depends on some factors:  Testing level  Detail and quality of test basis  System/software complexity  Risks  The relationship between the test basis, what is to be tested and how it is to be tested  SDLC in use  Test management tool being utilized  Level at which test design and other test work products are to be specified and documented  Skills and knowledge of the test analysts  The level of maturity of the test process and the organization itself  Availability of other project stakeholders for consultation 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 37
  • 38. Exercise: Test Analysis  Perform a complete test analysis for the given screenshot. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 38
  • 39. Detailed Test Conditions Advantages  Facilitates more flexibility in relating other test work products to test basis and test objectives yielding better monitor and control eventually  Contributes to defects prevention  Relates testing work products to stakeholders in terms that they can understand  Helps influence and direct testing and development activities  More efficient coverage of detailed measures and targets  Basis for clearer horizontal traceability within a test level 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 39
  • 40. Detailed Test Conditions Usage  Effective in the following situations:  Lightweight test design documentation methods are being used to accommodate SDLC, cost and/or time constraints or other factors  Little or no formal requirements or other development work products are available as the test basis  The project is large-scale, complex or high risk 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 40
  • 41. Disadvantages of Detailed Test Conditions  Potentially time-consuming  Maintainability can become difficult in a changing environment.  Level of formality needs to be defined and implemented across the team. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 41
  • 42. General Test Conditions Usage  Effective in the following situations:  CT  Less complex projects where simple hierarchical relationships exist between what is to be tested and how it is to be tested  AT where use cases can be utilized to help define tests 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 42
  • 43. Testing Process  Introduction  Test Planning, Monitoring and Control  Test Analysis  Test Design  Test Implementation  Test Execution  Evaluating Exit Criteria and Reporting  Test Closure Activities 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 43
  • 44. Learning Objectives  TM-1.4.1 (K3) Use traceability to check completeness and consistency of designed test cases with respect to the defined test conditions 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 44
  • 45. Test Implementation  Fulfillment of the test design  Checking against explicit and implicit entry criteria  Creating automated tests  Organizing tests (both manual and automated) into execution order  Finalizing test data and test environment  Forming a test execution schedule, including resource allocation 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 45
  • 46. Test Design  Test design = defining “how” to test (test cases) by stepwise elaboration of the identified test conditions or test basis using test techniques defined during test planning  Test design includes test data design and test environment design as well.  After the TA’s do their work, TM must check the design exit criteria before moving to the next step.  Test design exit criteria include review and approval from relevant stakeholders, coverage, cost, time ... 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 46
  • 47. Test Design Traceability  Test cases can be related directly to test basis or indirectly through test conditions (preferably detailed ones).  Test basis items could be individual requirements (atomic?!), risk item, strategic objectives, test objectives or project or stakeholder success criterion.  Traceability resolution loss should be avoided.  Must be bi-directional to allow TM to relate test results in terms of project aspects 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 47
  • 48. Inputs and Outputs for Test Design 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 48 Test Plan Test Conditions Test strategy Quality Risks Test Oracles Concrete or Logical? Test Cases w/ Traceability Test Data Design Test Environment Design TA’s
  • 49. Concrete or Logical Test Cases? Concrete Test Cases  Specific information  Preconditions, inputs, outputs, post conditions, and procedures  Useful when:  Requirements are well-defined.  Testers are inexperienced  External verification like audits is needed.  Benefit is excellent reproducibility.  Maintenance is a significant issue and limits testers creativity. Logical Test Cases  Guidelines for what should be tested  TAs can vary data and/or procedure  Useful when:  Requirements are not well-defined.  Testers are experienced in testing and the product.  Formality is not needed.  Benefits are better coverage due to variation and lower costs of creation and maintenance.  Reproducibility is a significant issue. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 49
  • 50. Sequential or Iterative Test Development Process?  Decision is influenced by the test level and test strategy.  In ST, SIT and AT, it is more sequential as test basis are available much earlier than the test execution start, while in CIT and CT, it is more iterative.  Analytical strategies are sequential while reactive tend are iterative by nature.  Even in sequential development, some overlap may be beneficial especially test data generation after test design and before test implementation can reveal problems in the test design. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 50
  • 51. Example: IEC 61508 Test Implications  Risk-analysis is required.  Risk-based testing is a mandatory risk mitigation technique.  All risk-relevant test work products must be formally documented.  SIL mandates selection of test techniques, required structural coverage and test levels to be used.  Reviews are mandatory even for test work products. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 51
  • 52. Exercise: Test Design  Derive minimal number of test cases from previously defined test conditions. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 52
  • 53. Testing Process  Introduction  Test Planning, Monitoring and Control  Test Analysis  Test Design  Test Implementation  Test Execution  Evaluating Exit Criteria and Reporting  Test Closure Activities 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 53
  • 54. Learning Objectives  TM-1.5.1 (K3) Use risks, prioritization, test environment and data dependencies, and constraints to develop a test execution schedule which is complete and consistent with respect to the test objectives, test strategy, and test plan 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 54
  • 55. Test Implementation 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 55
  • 56. Test Execution Schedule  Tests must be scheduled and sequenced during test implementation taking into considerations:  Order of automated and manual test execution  Prioritization assigned to tests from requirements or risks  Execution constraints like tester or environment availability or test environment reconfiguration  TM should carefully review schedule to avoid any test execution blockage. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 56
  • 57. Ready for Test Execution?  Trust, but verify the test environment and test data.  If the test environment to evolve during execution, double check all changes that will occur in accordance with the test execution schedule.  Contingencies should be defined if these anticipated changes did not occur for a reason or another.  Tests must be reviewed and verified before running.  Double check execution schedule to make sure every thing will be in place.  Make sure SW is ready for testing.  Smoke tests are effective in checking the test execution readiness. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 57
  • 58. Example: Test Execution Entry Criteria 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 58
  • 59. Detail and Complexity  Same argument considered for test conditions apply for test cases, test data description and test environment description.  Using testware as a way of system documentation, testware reuse or having testers w/ different skill set mandates documenting more details.  Regulatory rules may also mandates documentation details.  Project complexity can also determine level of details needed in implementation.  TM must defined implementation level of details to guide testers during implementation. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 59
  • 60. Early Test Implementation Pros  Concrete tests provide worked examples of how SW should behave, if written in accordance with test basis.  Business domain experts verify concrete tests easier than abstract business rules, and thereby identify further weaknesses in SW specifications.  Verified tests may provide illuminating illustrations of required behavior for SW designers and developers. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 60
  • 61. Early Test Implementation Cons  Changing test basis may lead to change in the testware already developed.  In Agile SDLC, embracing changes is the norm!.  In sequential SDLC, poorly managed projects can lead to unhappy surprises.  Before embarking on an extensive test implementation, it is wise to understand SDLC and the predictability of SW features that will be available for testing.  In conclusion, late implementation may be favored. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 61
  • 62. Testing Process  Introduction  Test Planning, Monitoring and Control  Test Analysis  Test Design  Test Implementation  Test Execution  Evaluating Exit Criteria and Reporting  Test Closure Activities 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 62
  • 63. Learning Objectives  TM-1.6.1 (K3) Use traceability to monitor test progress for completeness and consistency with the test objectives, test strategy, and test plan 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 63
  • 64. Test Execution  Begins once test object is delivered and entry criteria to test execution are satisfied (or waived carefully)  Criteria examples are:  Tests should be designed  Tools should be in place  Test results tracking should be working and tracked data is understandable by all team members.  Standards for test logging and defect reporting should be available and published.  Bottom line, do whatever it takes to make execution proceeds efficiently. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 64
  • 65. Blending Reactive Test Strategies  10-20% of testing should be reactive (even more in case formal testing is automated).  Blending offsets weaknesses in test strategies as they complement each other.  Reactive testing blend is done by giving permissions to explore other options during formal execution.  Time boxing can help.  Experienced testers should do experience- and defect-based testing more.  Time boxing and chartered testing can help.  Reproducibility for unscripted test should be considered.  Capture-play back tools can help. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 65
  • 66. TM Role in Test Execution  Being there all day during test execution when needed unless he is in meetings  Daily team debriefing of progress with the entire team  Take proper control actions if needed  Checks and ensures proper bi-directional traceability carried out by the team 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 66 Pass 2 Defects Status Count Verified 47 To be verified 1 Needs approval 5 Deferred 21 Failed verification 3 Not a defect 8 New 13 Total 103
  • 67. Exercise: Metrics  Describe one or more metric that you can use to report residual risks during test execution. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 67
  • 68. Testing Process  Introduction  Test Planning, Monitoring and Control  Test Analysis  Test Design  Test Implementation  Test Execution  Evaluating Exit Criteria and Reporting  Test Closure Activities 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 68
  • 69. Learning Objectives  TM-1.7.1 (K2) Explain the importance of accurate and timely information collection during the test process to support accurate reporting and evaluation against exit criteria 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 69
  • 70. Testing as a Source of Information  Testing produces lots of information (relevant or not).  TM must ensure that right information is captured and delivered to appropriate stakeholders.  TM should be able to relate the test results to the exit criteria.  Test management tools must be properly selected and in place to support accurately and timely gather information so as to facilitate effective evaluation and reporting.  TM should never use gathered info for personal appraisals. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 70
  • 71. Proper Information Collection  Requires clear definition of what to be collected, by whom and when  Should be documented during test planning  Frequent check on gathered information is a must (never assume they are OK).  Team should be trained and aware of how to gather/report information w/ or w/o tools. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 71
  • 72. Reporting Frequency and Detail  There is no one size fits all reporting approach.  Close monitoring of exit criteria does not necessitate detailed reporting.  Close monitoring = proper control  Reporting = providing supporting information  Details and type of reporting should differ according to recipient stakeholders.  They need different information to do their work. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 72
  • 73. Testing Process  Introduction  Test Planning, Monitoring and Control  Test Analysis  Test Design  Test Implementation  Test Execution  Evaluating Exit Criteria and Reporting  Test Closure Activities 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 73
  • 74. Learning Objectives  TM-1.8.1 (K2) Summarize the four groups of test closure activities  TM-1.8.2 (K3) Implement a project retrospective to evaluate processes and discover areas to improve 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 74
  • 75. Test Closure Activities  Once execution is complete, key outputs should be captured and either passed to the relevant person or archived. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 75 Test Closure Activities Test Completion Check Test Artifacts Handover Lessons Learned Archiving
  • 76. Test Closure Activities cont’d  These tasks are important, often missed, and should be explicitly included as part of the test plan.  It is common for one or more of these tasks to be omitted due to premature reassignment or dismissal of project team members, resource or schedule pressures on subsequent projects, or team burnout.  On projects carried out under contract, such as custom development, the contract should specify the tasks required. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 76
  • 77. Example: Test Closure Checklists 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 77 Test Completion • All planned tests run or deliberately skipped? • All known defects fixed, deferred or accepted? Test Artifacts Handover • Known defects communicated to operation and support team? • Test and environment to maintenance team? • Regression tests documented? Lessons Learned • Were relevant stakeholders involved in risk analysis? • Were estimates accurate? • RCA done and actions defined? • Process improvement? • Any unanticipated deviation? Archiving • All test work product archived in CM? • If the project to be reopened, do we have every thing we need?
  • 78. Project Retrospective  A meeting held at the end of each iteration/increment  To discuss:  What went well?  What can be improved?  How to improve? 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 78
  • 79. Exercise: Retrospective  Assume that 25% of defects opened on a certain feature were rejected as not actual problems.  If rejection rate is more than 5%, is unacceptable.  List 5 actions, that could investigate the root cause of the high defect rejection rate. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 79
  • 80. Summary  Introduction  Test Planning, Monitoring and Control  Test Analysis  Test Design  Test Implementation  Test Execution  Evaluating Exit Criteria and Reporting  Test Closure Activities 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 80
  • 81. 24-Sep-15ISTQB Advanced Level 2012 - Test Manager 81

Editor's Notes

  1. Test object is the component or system to be tested. Test item is the individual element to be tested. There usually is one test object and many test items. Test basis is all documents from which the requirements of a component or system can be inferred. The documentation on which the test cases are based. If a document can be amended only by way of formal amendment procedure, then the test basis is called a frozen test basis. Test condition is an item or event that could be verified by one or more test cases (function, transaction, quality C/C or structural element). Test case is a set of input values, execution preconditions, expected results and execution post conditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement. Test procedure is a sequence of actions for the execution of a test.
  2. Tracking detailed design specification from system designers, business requirements from business analysts, and the test work products from testing team. Detailed design documents to be approved before detailed test case creation can start. In Agile lifecycle, informal transfer-of-information sessions convey information between teams prior to the start of testing. Each feature that is within scope may need a corresponding test design specification. W/ architects to verify resources availability, to ensure people configuring environment are committed, and to understand cost/delivery timescales and the work required to deliver the test environment
  3. Normally puts test manager under pressure to sign-off releases then followed by blame if the release is proven to be buggy in the field. Leads to schedule slack, retroactive unit testing done by test teams instead of development or unstable deliveries at the end. A test management weakness .
  4. A business stakeholder may be more interested in establishing coverage against an operational business cycle even though the specification is defined in terms of system functionality.
  5. Reference: https://jazz.net/help-dev/clm/index.jsp?re=1&topic=/com.ibm.rational.test.qm.doc/topics/t_quality_obj_scenario.html&scope=null
  6. We will have project initiation, 5 development iterations and 1 post mortem. Project initiation will have P, MnC, and TA. 5 iterations will have complete process activities. Post mortem will have MnC as retrospective and closure activities.
  7. In formally-documented contexts, test implementation is the activity in which test designs are implemented as concrete test cases, test procedures, and test data. Some organizations following the IEEE 829 [IEEE829] standard define inputs and their associated expected results in test case specifications and test steps in test procedure specifications. More commonly, each test’s inputs, expected results, and test steps are documented together. Test implementation also includes the creation of stored test data (e.g., in flat files or database tables).
  8. Traceability resolution loss occurs when multiple requirements trace to a single test condition which in turn traces to multiple test cases. If a single test fails, it is unclear what is the impacted requirement.
  9. A table showing risk item, planned testes, passed tests, failed tests, defects reported, defects closed.
  10. Check requirements of that feature for any requirements problems. Review test cases and test conditions. Review implemented automated test scripts. Rerun sample test cases that resulted in rejected defects. Review rejected defects. Train testers on the feature.