Your SlideShare is downloading. ×

Istqb ctal tm

3,069
views

Published on

Published in: Business, Technology, Education

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,069
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
117
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. ISTQB CTAL Test Manager (TM) A Subject Overview of the ISTQB CTAL TM Certification Petro Porchuk Lohika 2010 Copyright Petro Porchuk 2010. Lohika
  • 2. My Goal Copyright Petro Porchuk 2010. Lohika  Present topics due to the Syllabus CTAL  90% of the info have the ISTQB colour  Give some references  10% - Express my own Vision http://istqb.org/download/attachments/2326555/CTAL_Syllabus_V_2007.pdf
  • 3. My Main Source Copyright Petro Porchuk 2010. Lohika Software Testing Practice: Test Management: A Study Guide for the Certified Tester Exam Istqb Advanced Level Andreas Spillner (Author) http://www.amazon.com/Software-Testing-Practice-Management-Certified/dp/193395213X/ref=sr_1_6?ie=UTF8&s=books&qid=1275570124&sr=8-6
  • 4. The ISTQB Framework, May 2010 http://istqb.org/display/ISTQB/Certification?atl_token=CrArtYjmmm Copyright Petro Porchuk 2010. Lohika
  • 5. CTAL TM – Part 1 Part 1 1. Introduction 2. Test Process and Test Tools 3. Testing in the software Life Cycle 4. Test Policy and Test Handbook 5. The Test Plan 6. Test Control 7. Assessing and Improving the Development and Test Processes Part 2 8. Deviation Management 9. Risk Management and Risk-Oriented Testing 10. Staff Qualification and Skills 11. Test Metrics 12. Selecting and Implementing Test Tools 13. Standards Copyright Petro Porchuk 2010. Lohika
  • 6. (1) The Introduction Copyright Petro Porchuk 2010. Lohika TL should be:  Technically Skilled  Communication and Language Skilled  Good Analyst  Pro-Active  Brave Organizer http://www.lestnizza.com.ua/upload/image/it_range.jpg
  • 7. Copyright Petro Porchuk 2010. Lohika (2)(2) Test process and Test ToolsTest process and Test Tools  The Fundamental Test Process  Different Types of Test Tools  Tools for Management and Test Control  Tools for Test Data and Test Script Specific.  Tools for Static Testing  Tools for Dynamic Testing
  • 8. 2.1. The Fundamental Test Process Copyright Petro Porchuk 2010. Lohika
  • 9. 2.2. The Test Tools  CASE Tools (Computer Aided Software Engineering)  CAST Tools (Computer Aided Software Testing)  Tools for Management and Test Control (Planning, Defect Management, Requirement Management, Test Spec. Management)  Tools for Test Data and Test Script Specific (Test Data Generators, Test Generators)  Tools for Static Testing (Static analizers, model analize spesifications, reviews tools)  Tools for Dynamic Testing (Debugger, drivers, stubs, test robots, L/P test tools, monitors) Copyright Petro Porchuk 2010. Lohika
  • 10. ”A Fool with a Tool is still a Fool!” Copyright Petro Porchuk 2010. Lohika http://www.sungsblog.com/labels/world.php
  • 11. 2.3. Summary Copyright Petro Porchuk 2010. Lohika ✔ Testing must be divided into individual process steps ✔ For each phase of the Test Process Tools are available ✔ The use of the Test Tools is only of advantage if a Test Process is controlled and defined process ✔ ”A Fool with a Tool is still a Fool”
  • 12. We All Do Work Process OrientedWe All Do Work Process Oriented Copyright Petro Porchuk 2010. Lohika
  • 13. Copyright Petro Porchuk 2010. Lohika (3)(3) Testing in the Software Life CycleTesting in the Software Life Cycle  Test and Development Process  The General V-Model  The W-Model  Rational Unified Process (RUP)  V-Model XT  Extreme Programming  Rapid Application Development (RAD)  Dynamic System Development Method (DSDM)
  • 14. 3.1. Test and Development Process Copyright Petro Porchuk 2010. Lohika QA Process should have strong interaction with the Development Process  Reaction to the Requirement Change  Configuration Management  Project Management  The General Test Process should be adjusted to the specific activities applied Development Processes
  • 15. 3.2. The General V-Model http://www.testingexcellence.com/v-model/ Copyright Petro Porchuk 2010. Lohika
  • 16. 3.3. The V-Model for QA Copyright Petro Porchuk 2010. Lohika Pros:  Emphasizes Test Levels  Verification  Validation Cons:  Preparatory Test Activities are missing
  • 17. 3.4. The W-Model http://www.testingthefuture.net/2009/09/the-w-model/ Copyright Petro Porchuk 2010. Lohika
  • 18. 3.5. The W-Model for QA Copyright Petro Porchuk 2010. Lohika Pros:  Preparatory Test Activities are here  Concurrency of Test and Dev. Processes  PREviews  Debugging Cons:  Needs several Test Managers for large projects  Refers only to a System Development, not Process Oriented
  • 19. 3.6. The Rational Unified Process http://www.wittmannclan.de/ptr/cs/slcycles.html Copyright Petro Porchuk 2010. Lohika
  • 20. 3.7. QA in RUP Copyright Petro Porchuk 2010. Lohika Pros:  Test Activities begin early and accompany other processes Cons:  Which Test Activities to be eprformed when is not made clear
  • 21. 3.8. V-Model XT (eXtreme Tailoring) https://sosa.ucsd.edu/teaching/cse294/fall2004/pres/vmxt.htm Copyright Petro Porchuk 2010. Lohika  The current version of the V-Model is the V-Model 97. Since 1997, no changes or improvements were made. In 2002, a project was initiated to redesign and improve the existing process model. The resulting V-Model XT should reflect new standards and technologies; it also should expose significantly enhanced quality properties such as usability, adaptability, changeability and scalability.  Available since 2005.
  • 22. 3.9. QA in V-Model XT Copyright Petro Porchuk 2010. Lohika Pros:  Tailoring  Particular attention to Customer-Supplier communication  Quality Management Cons:  No Test Manager role, but Quality Manager
  • 23. 3.10. The Extreme Programming Copyright Petro Porchuk 2010. Lohika http://www.extremeprogramming.org/map/project.html
  • 24. 3.11. QA in XP Copyright Petro Porchuk 2010. Lohika Pros:  Testing is at the center of the development effort (Test First)  Advanced Unit Testing Cons:  Missing systematic Test Specification creation  QA misses Aceptance Testing, it's been put on a customer
  • 25. 3.12. Rapid Application Development Copyright Petro Porchuk 2010. Lohika http://www.antrix.com/services/service_list/11_validation_computer_systems.htm
  • 26. 3.13. QA in RAD Copyright Petro Porchuk 2010. Lohika Pros:  Extensive use of Tools  Invloving Customer to evaluate Cons:  Roles of Testing or Test Management are not defined, there is ”SWAT Team” instead. ”SWAT” = Skilled Workers and Advanced Tools
  • 27. 3.14. Dynamic System Develpment Method Copyright Petro Porchuk 2010. Lohika  DSDM = SCRUM  SCRUM = RAD + (precise process steps and roles)  Focus on Teamwork  Close Cooperation with a Customer  Iterative System Development  Time Boxing  ”MoSCoW” Principle
  • 28. 3.15. QA in SCRUM Copyright Petro Porchuk 2010. Lohika Pros:  Team driven evolutionary process Cons:  Test Manager role is not here.  Tester is responsible for component tests mostly.
  • 29. (3) Summary Copyright Petro Porchuk 2010. Lohika ✔TM should adopt Test Process to the chosen Development Model ✔TM should make sure the docs are available to have enough time for the test preparation ✔TM should make sure Test Activities start early in the Process ✔TM should make sure the needed Test Levels are planned and executed
  • 30. (4)(4) Test Policy and Test HandbookTest Policy and Test Handbook Copyright Petro Porchuk 2010. Lohika  Quality Policy and Test Policy  Bring the Test Policy to Life  Test Handbook
  • 31. 4.1. Quality Policy and Test Policy4.1. Quality Policy and Test Policy Copyright Petro Porchuk 2010. Lohika  Quality Policy Expresses the particular importance that an organization attaches to ”quality” and the demands it makes on the quality of it's products, services and processes.  Test Policy A high level document describing the principles, approach and major objectives of the organization regarding testing.  Test Handbook A concretization of the Test Policy http://istqb.org/download/attachments/2326555/ISTQB+Glossary+of+Testing+Terms+2+1.pdf
  • 32. 4.3. Bring the Test Policy to Life4.3. Bring the Test Policy to Life Copyright Petro Porchuk 2010. Lohika  Relevance to Business Objective  Be Realistic  Adequate Maturity  Measurability  Liveliness
  • 33. (4)(4) SummarySummary Copyright Petro Porchuk 2010. Lohika
  • 34. A QuestionA Question Copyright Petro Porchuk 2010. Lohika What is the most Common Question for the QA interview ? http://profiles.friendster.com/98111238
  • 35. A QuestionA Question Copyright Petro Porchuk 2010. Lohika What is the most Common Question for the QA interview ? What is the Test Plan and what does it consist of? http://profiles.friendster.com/98111238
  • 36. (5)(5) The Test PlanThe Test Plan Copyright Petro Porchuk 2010. Lohika  General Test Plan Structure  Defining a Test Strategy  Test Effort Estimation  Flat Models  Detailed Models baced on Test Activities  Models baced on Functional Volume  Organization of Test Teams and Test Levels
  • 37. 5.1. The General Test Plan Structure5.1. The General Test Plan Structure Copyright Petro Porchuk 2010. Lohika Chapter According to IEEE 829 1. Test plan identifier 2. Introduction 3. Test items 4. Features to be tested 5. Features not to be tested 6. Approach 7. Item pass/fail criteria (test exit criteria) 8. Suspension criteria and resumption requirements 9. Test deliverables 10. Testing tasks 11. Environmental needs 12. Responsibilities 13. Staffing and training needs 14. Schedule 15. Risk and contingencies 16. Approvals
  • 38. 5.2. Defining the Test Strategy5.2. Defining the Test Strategy Copyright Petro Porchuk 2010. Lohika
  • 39. 5.3. Test Effort Estimation – Flat Models5.3. Test Effort Estimation – Flat Models Copyright Petro Porchuk 2010. Lohika Adding a percentage flat rate to the development effort QA Effort = 0.5*(DEV Effort)  Assuming development estimation is accurate  Take into an account experience from similar projects  It makes sence to use several sources  But: Cannot be used for big projects
  • 40. 5.4. Test Effort Estimation – Detailed Models5.4. Test Effort Estimation – Detailed Models Copyright Petro Porchuk 2010. Lohika Concatenation of individual activities and different estimation methods must be used for each activity. Consider:  Manpower  Time  HW and SW for the Test Environment  Testware (test automation tools, etc)  Different influencing factors  Keep estimated tasks small  Involve the team in estimation  Use experience
  • 41. 5.5.1. Test Effort Estimation – Model Based5.5.1. Test Effort Estimation – Model Based Copyright Petro Porchuk 2010. Lohika Estimation of the Scope or Functionality of the test objects and establishing a mathematical model  FPA (Function Point Analysis)  Count the number of functions to be developed  Add the non-functional values (14*(1..5)+0.65 * func)  Transform to man*hours The Test Effort is a result of estimates from function points.  But: Used only for simply numbers (of test cases)
  • 42. 5.5.2. Test Effort Estimation – Model Based5.5.2. Test Effort Estimation – Model Based Copyright Petro Porchuk 2010. Lohika Test Point Analysis Uses Quality Atributes (ISO 9126)  Dynamic Test Points (FP; Quality Attributes; Function priority, compexity, influence etc) Are multiplied with each other per function and summed up. This reflects the dynamic test efforts.  Static Test Points (flexibility, testability, tracebility) The total number of FP is multiplied by a factor and added up. Static test efforts  Dynamic and Static test points are summed up and subsequently weighted with environment factors (docs, testware, tools, organization culture etc)  The result – is the number of test hours required
  • 43. (5) Summary Copyright Petro Porchuk 2010. Lohika ✔ The Test Plan is the concrete application of the Test Handbook, mapping the strategic requirement of the Test Policy into the Project ✔ Essential Test Plan components are: Test Strategy Test Effort Estimation Test Organisation ✔ For Test Effort Estimation may be applied: Intuition, experience, detailed itemization, formulas ✔ The WORSE estimation is NO ESTIMATION
  • 44. (6)(6) Test ControlTest Control Copyright Petro Porchuk 2010. Lohika ✔Initializing the Test Tasks ✔Monitoring the Test Process ✔Reacting to Test Results ✔Reacting to Changed Circumstances ✔Evaluating the Test Completion ✔Test Report
  • 45. Standish Group Chaos Report 2009Standish Group Chaos Report 2009 Copyright Petro Porchuk 2010. Lohika Successful Projects 32% Late, Overbudget, Miss features 44% Failed Projects 24% http://www.standishgroup.com/newsroom/chaos_2009.php "These numbers represent a downtick in the success rates from the previous study, as well as a significant increase in the number of failures", says Jim Crear, Standish Group CIO, "They are low point in the last five study periods. This year's results represent the highest failure rate in over a decade"
  • 46. How to Fix This?How to Fix This? Copyright Petro Porchuk 2010. Lohika ?
  • 47. How to Fix This?How to Fix This? Copyright Petro Porchuk 2010. Lohika ?  Total Quality Management (TQM)  Test and Development Processes Improvement
  • 48. (7)(7) Assessing and Improving theAssessing and Improving the Development and Test ProcessesDevelopment and Test Processes Copyright Petro Porchuk 2010. Lohika  General Techniques and Approaches  Total Quality Management  Kaizen  Six Sigma  Improving the Software Development Processes  Capability Maturity Model Integration (CMMI)  SPICE  Evaluating the Test Processes  Testing Maturity Model (TMM)  Test Process Improvement (TPI)
  • 49. 7.1 Total Quality Management (TQM)7.1 Total Quality Management (TQM) Copyright Petro Porchuk 2010. Lohika QUALITY is in the center. No instructions – it's an attitude. http://www.edrawsoft.com/TQM-Diagrams.php
  • 50. 7.2 Kaizen7.2 Kaizen (Japanese for(Japanese for Kai "Kai "improvement" orimprovement" or ZenZen "change for the better")"change for the better") Copyright Petro Porchuk 2010. Lohika  Intensive use of the employee suggestion system  Appreciation and regard for employees' striving for improvement  Established small group discussion circles addressing defects and improvement suggestions  ”Just in time” production to eliminate wasted effort  5 Ss process for the improvement of workspaces:  Seiri – tidiness; old and useless items have no place at the workspace  Seiton – orderliness; ”everything” has its place for quick retrieval and storage  Seiso – cleanliness; keep the workspace clean and tidy at all times  Seiketsu – standartization of all practices and processes  Shitsuke – discipline; all activities are performed in a discipline way  ”Total Productive Maintenance” for maintenance and servicing of all means of production  BUT: difficult to apply in european culture such strong Jananese rules
  • 51. 7.3 Six Sigma7.3 Six Sigma (by Motorola, USA in 1981)(by Motorola, USA in 1981) Copyright Petro Porchuk 2010. Lohika  Six Sigma seeks to improve the quality of process outputs by identifying and removing the causes of defects (errors) and minimizing variability in manufacturing and business processes.  The term six sigma originated from terminology associated with manufacturing, specifically terms associated with statistical modelling of manufacturing processes.  The maturity of a manufacturing process can be described by a sigma rating indicating its yield, or the percentage of defect-free products it creates.  A six-sigma process is one in which 99.99966% of the products manufactured are free of defects (), compared to a one-sigma process in which only 31% are free of defects.  BUT: Requires a lot of statistics from the Test Manager http://en.wikipedia.org/wiki/Six_Sigma
  • 52. 7.4 CMMI7.4 CMMI (Software Engineering Institute (SEI, 2002))(Software Engineering Institute (SEI, 2002)) Copyright Petro Porchuk 2010. Lohika http://en.wikipedia.org/wiki/Capability_Maturity_Model_Integration
  • 53. 7.5 SPICE7.5 SPICE ("Software Process Improvement and Capability Evaluation")("Software Process Improvement and Capability Evaluation") (ISO/IEC 15504 standard)(ISO/IEC 15504 standard) Copyright Petro Porchuk 2010. Lohika http://www.kuglermaagusa.com/tools_spice15504_com.html 0 ”Incomplete process” | General Failure to perform the bace practices in the Process
  • 54. 7.6 Testing Maturity Model (TMM)7.6 Testing Maturity Model (TMM) (1996 Illinois Institute of Technology, Chicago)(1996 Illinois Institute of Technology, Chicago) Copyright Petro Porchuk 2010. Lohika Based on CMM: Levels Process Areas 1 – Initial No process identified 2 – Phase Definition Test Policy and Goals Test Planning Test Techniques and Methods Test Environment 3 - Integration Test Organization Test Training Program Test Life Cycle and Integration Control and Monitor 4 – Management and Measurement Peer Reviews Test Measurement Software Quality Evaluation 5 – Optimization Defect Prevention Quality Control Test Process Optimization
  • 55. 7.7 Test Process Improvement (TPI)7.7 Test Process Improvement (TPI) (Sogeti, Netherlands. http://www.sogeti.com)(Sogeti, Netherlands. http://www.sogeti.com) Copyright Petro Porchuk 2010. Lohika Key / Scale 0 1 2 3 4 5 6 7 8 9 10 11 12 13 1 Test strategy A B C D 2 Life-cycle model A B 3 Moment of involvement A B C D 4 Estimating and planning A B 5 Test Specification techniques A B 6 Static test techniques A B 7 Metrics A B C D 8 Test tools A B C 9 Test environment A B C 10 Office environment A 11 Commitment and motivation A B C 12 Test functions and training A B C 13 Scope of methodology A B C 14 Communication A B C 15 Reporting A B C D 16 Defect management A B C 17 Testware management A B C D 18 Test process management A B C 19 Evaluation A B 20 Low-level testing A B C
  • 56. 7.8 TPI Assessment7.8 TPI Assessment Copyright Petro Porchuk 2010. Lohika Key / Scale 0 1 2 3 4 5 6 7 8 9 10 11 12 13 1 Test strategy A B C D 2 Life-cycle model A B 3 Moment of involvement A B C D 4 Estimating and planning A B 5 Test Specification techniques A B 6 Static test techniques A B ... Key / Scale 0 1 2 3 4 5 6 7 8 9 10 11 12 13 1 Test strategy A B C D 2 Life-cycle model A B 3 Moment of involvement A B C D 4 Estimating and planning A B 5 Test Specification techniques A B 6 Static test techniques A B ... Current.. Required..
  • 57. (7) Summary Copyright Petro Porchuk 2010. Lohika ✔ It is up to the individual company to decide how well a particular model is suited toi it's own specific circumstances.
  • 58. CTAL TM – Part 2 Part 1 1. Introduction 2. Test Process and Test Tools 3. Testing in the software Life Cycle 4. Test Policy and Test Handbook 5. The Test Plan 6. Test Control 7. Assessing and Improving the Development and Test Processes Part 2 8. Deviation Management 9. Risk Management and Risk-Oriented Testing 10. Staff Qualification and Skills 11. Test Metrics 12. Selecting and Implementing Test Tools 13. Standards Copyright Petro Porchuk 2010. Lohika
  • 59. Deviation TerminologyDeviation Terminology Copyright Petro Porchuk 2010. Lohika Error (Human Action) Bug, Fault (Wrong Code) Failure (Wrong Behavior) http://estonovaamisa.files.wordpress.com/2009/02/human-error1.jpg
  • 60. (8)(8) Deviation ManagementDeviation Management Copyright Petro Porchuk 2010. Lohika ✔Terminology ✔ Error ✔ Fault ✔ Failure ✔Documenting Incidents ✔Incident Handling (Bug Tracking Systems Workflow) ✔IEEE 1044/1044.1 Standard (Chapter 13)
  • 61. What is Risk?What is Risk? Copyright Petro Porchuk 2010. Lohika ?
  • 62. What is Risk?What is Risk? Copyright Petro Porchuk 2010. Lohika ? IT Project = Risk = Probability * Damage
  • 63. (9)(9) Risk Management and Risk-Oriented TestingRisk Management and Risk-Oriented Testing Copyright Petro Porchuk 2010. Lohika  Risk Management  Identification of the risk context  Risk identification  Risk analysis and risk evaluation  Risk control  Risk verification and monitoring  Risk-Oriented Test Plan Creation and Test Prioritization  Failure Modes and Effect Analysis (FMEA)
  • 64. 9.1.1 Identification of the Risk Context9.1.1 Identification of the Risk Context Copyright Petro Porchuk 2010. Lohika
  • 65. 9.1.2 Risk Identification. Risk Categories9.1.2 Risk Identification. Risk Categories Copyright Petro Porchuk 2010. Lohika Risk Category Influence Prediction Example External risk Cannot be influenced Difficult to predict •Lightning Damage •Out of Power Strategic risk Difficult to influence Easy to predict •Changing a Corporate standard etc Project risk Easy to influence Easy to predict •Sales Problems •Technical Risks •Servers unavailability •Defect Correction Risks Product risk Can be influenced Easy to predict •Customer suffering from product failures •Poor Quality affects reputation
  • 66. 9.1.3 Risk Identification Techniques9.1.3 Risk Identification Techniques Copyright Petro Porchuk 2010. Lohika  Expert Interviews and Questionnaires  Independent Estimations  Risk Workshops  Risk Brainstorming (Worst Case Scenarios)  Use of risk templates and checklists  Experiences from completed projects
  • 67. 9.1.4 Risk Evaluation9.1.4 Risk Evaluation Copyright Petro Porchuk 2010. Lohika  F – the frequency of execution of use (a function)  C – criticality. Represents the worst possible effects of a function not working well (at all)  Rp – The Project risks  Rt – Technical Product risks  Rb – The Commercial Product risks Risk Factor: RF= RtRbRp 3 F∗RbC∗Rt
  • 68. 9.2 Risk Oriented Test Plan Creation9.2 Risk Oriented Test Plan Creation Copyright Petro Porchuk 2010. Lohika  Target-oriented test plan creation: applying appropriately different test techniques and test intensities to cover system functions with different risks  Prioritized testing: giving areas with higher risks higher priority and testing them earlier  Residual risk awareness: identifying the residual risks that remain in the delivered software and are due to reductions in test or nonperformance of planned tests
  • 69. 9.3.1 Risk Based Testing.9.3.1 Risk Based Testing. Failure Modes and Effect Analysis (FMEA)Failure Modes and Effect Analysis (FMEA) Copyright Petro Porchuk 2010. Lohika  FMEA is a Method used to identify potential error types and their effect on a system. FMEA Procedure: 1. Generate a list of all the risk factors. 2. Specify the type and probability of falures for each factor. 3. Investigate the effect of failures on other factors, applying, e.g., simulations. 4. Investigate the effect on Project Planning. 5. Identify possibilities for detecting the failure. 6. Identify possibilities for compensating the failure. 7. Identify possibilities for preventing the failure. 8. Define measures to avoid defects. 9. Evaluate the effect of the proposed measures. 10. Document the results.
  • 70. 9.3.2 Risk Priority Number.9.3.2 Risk Priority Number. Failure Modes and Effect Analysis (FMEA)Failure Modes and Effect Analysis (FMEA) Copyright Petro Porchuk 2010. Lohika  FMEA defined the Risk Priority Number (RPN). O – Probability of Occurrence E – Expected Damage D – Probability of Detection RPN =O∗E∗D
  • 71. (9)(9) SummarySummary Copyright Petro Porchuk 2010. Lohika ✔ Risk = Rrobability * Damage ✔ Testing is a preventive measure, reducing risks ✔ Based on the most important risks, suitable test techniques are to be selected, and tests are to prioritized to allow tests covering high risks to be executed early and with the appropriate effort ✔ Prioritization and risk estimation criteria, defined in the Test Plan, can be applied to test objects and test cases
  • 72. Skilled people as the key success factorSkilled people as the key success factor Copyright Petro Porchuk 2010. Lohika TOYAKO, JAPAN - JULY 08: (L-R) German Chancellor Angela Merkel, U.S. President George W. Bush, Japanese Prime Minister Yasuo Fukuda, French President Nicolas Sarkozy and Russian President Dmitry Medvedev plant the memorial tree after a working session at the Windsor Hotel Toya on July 8, 2008 in Toyako, Hokkaido, Japan. http://www.life.com/image/81853290
  • 73. (10)(10) Staff Qualification and SkillsStaff Qualification and Skills Copyright Petro Porchuk 2010. Lohika  Individual Skills  Functional Team Roles  Social Team Roles  The Communication Factor  The Motivation Factor
  • 74. 10.1 Individual Skills10.1 Individual Skills Copyright Petro Porchuk 2010. Lohika  The ”perfect” test team member has exellent technical knowledge and high social competency.  There is NO PERFECT Individual from the scratch. Need to develop.  IT Expert as QA:  Great technical competency  DANGER: In love with technology → loses focus  Application Expert as QA:  Great domain expert. Knows non-func. requirements  DANGER: Blinkered atitude → doesn't seek for improvements  Social Competence  The ability to familiarize quickly with complex domains  The ability to detect defects. (”professional ressimism”)  The ability to cope with criticism adequately. Diplomacy  Courage to leave some gaps  Discipline, exactitude, patience, tolerance  The ability to work in a team and to communicate
  • 75. 10.2 Functional Team Roles10.2 Functional Team Roles Copyright Petro Porchuk 2010. Lohika  Test Manager  Leads the Test Team  Test Planning and Control, Test Reporting  Human Resource Management and Test Process Improvement  Test Designer (QA Analyst)  Creation and Maintenance of Test Specification. Prioritizing  Defining and Apply Test Design Techniques  Test Automator  The Test Team's Programmer  Developing Automation Tests, Test frameworks and Tools  Test Administrator  Installation, Administration and Maintenance of the Test Environment  Installing and set up of the System Software (Operating Systems, Databases, Application Servers)  Tester  Execution the tests and documenting the test results  Accuracy and Creativity  Expert  Supporting L/P testing, database or network testing etc
  • 76. 10.3 Social Team Roles10.3 Social Team Roles Copyright Petro Porchuk 2010. Lohika Type Descriptors Strengths Weakness Monitor Evaluator Strategic Power of judgment Lacks drive and the ability to inspire others Shaper Dynamic, open-minded, result oriented Lot's of drive, fight inefficiency, experts pressure Prone to irritate and to hurt others Planter Individualistic Creative. Lot's of intellectual power Ignores practical details and instructions Completer Straight, anxiuos Ability to complete things, perfectionism Inclined to worry undully, reluctant to delegate Team Worker Cooperative, mild, sensitive Ability to cope with different situation and people, promotes team spirit Can be easily influenced Implementer Conservative, dutiful Hard-working, disciplined Inflexible. Rejects unproved ideas Co-ordinator Self-confident, mature Recognizes individual talents. Good at clarifying goals Has limited creativity Resource Investigator Extrovert, enthusiastic, communicative Likes to develop contacts, picks up new ideas, reacts to challenges Too optimistic, tend to lose interest after initial enthusiasm Specialist Single-minded, self-starting, dedicated Provides knowledge and skills in rare supply Contributes only a narrow front, overlooks the ”big picture”
  • 77. 10.4 The Communication Factor10.4 The Communication Factor Copyright Petro Porchuk 2010. Lohika  Communicate problems accurately  Test Team External Communication  Tester → Developer  Developer → Tester  Test Manager → Projects Manager  Project Manager → Test Manager  Tester → User  Test Team Internal Communication  Regular Team Meetings  Knowledge Transition
  • 78. 10.5.1 The Motivation Factor10.5.1 The Motivation Factor Copyright Petro Porchuk 2010. Lohika http://media.photobucket.com/image/funny workers/geet_kunal/happy-best-wishes-monday/rg272945.jpg
  • 79. 10.5.2 The Motivation Checklist10.5.2 The Motivation Checklist Copyright Petro Porchuk 2010. Lohika  The schedule is a motivating factor  The resources provided are a motivating factor  The test process is a motivating factor  The task is a motivating factor  The team will feel valued, recognised and motivated http://www.imbus.de/fileadmin/imbus_repository/Downloads/imbus_Allgemein/Dokumente/Checkliste-Motivation_E.zip
  • 80. (10)(10) SummarySummary Copyright Petro Porchuk 2010. Lohika ✔ A member of the test team should be IT professional, have knowledge from the application domain and have social competence ✔ The different tasks in the test process are reflected in the different roles within the test team ✔ Becides professional role, each team member should play a social role in the team ✔ Detected problems should be communicated to the external parties with relevan communication style ✔ Test Manager should organize regular internal meetings ✔ The Test Manager should provide a healthy motivation environment and professional working condition for the team. Part of this task is to adequately represent the role and contribution of the test team within the organization
  • 81. Why do the People change job?Why do the People change job? Copyright Petro Porchuk 2010. Lohika Stupid Manager Far from Home Low Salary Not Allowed to use Facebook
  • 82. (11)(11) Test MetricsTest Metrics Copyright Petro Porchuk 2010. Lohika  Why do we need Metrics?  Presenting Measurements  Test Metrics Types  Residual Defect Estimations and Reliability
  • 83. 11.1 Why do we need Metrics?11.1 Why do we need Metrics? Copyright Petro Porchuk 2010. Lohika  The Test Metrics is the Tool for Control Tom DeMarco on Measurement: „You can not control what you can not measure. Measurement is the prerequisite to management control.“ from Tom DeMarco American Consultant, 1982
  • 84. 11.2 Presenting Measurements11.2 Presenting Measurements Copyright Petro Porchuk 2010. Lohika  Diagrams help you to visualize measurement values and their progression
  • 85. 11.3.1 Test Metrics Types11.3.1 Test Metrics Types Copyright Petro Porchuk 2010. Lohika  Test Case Based Metrics  Number of Planned, Specified Test Cases  Number of Created Test Procedures  Number of Changed, Deleted Test Cases  Number of Run, Passed, Failed, Blocked TC  Test Object Based Metrics  Number of tested functions / Total number of functions  Number of executed TC / Number od specified TC epr function  Percentage of all requirements covered by Test Cases  Number of platforms covered by test  Percentage of interfaces tested  Number of lines of codes or executable instructions (kilo lines of code – KLOC)  Number of the covered paths in the control flows graph  Percentage of branches in the control flow graph that were covered
  • 86. 11.3.2 Test Metrics Types11.3.2 Test Metrics Types Copyright Petro Porchuk 2010. Lohika  Defect - Baced Metrics  Bug Detection Percentage  Number of defects relative to criticality (defect severity)  Number of defects per status (defect correction progress)  Defect report status  Cost and Effort-Based Metrics  Number of person-days for test planning or specification  Number of person-days for the creation of test procedures  Evaluating Test Effectiveness  Degree of Defect Detection  Confidence Level  Test Effectiveness DDD= numberof weighted defects during test total numberof defects found CL=1− numberof weighted defects number of executed test cases ∗degree of test coverage TE=DDD∗CL
  • 87. 11.4 Residual Defect Estimations and Reliability11.4 Residual Defect Estimations and Reliability Copyright Petro Porchuk 2010. Lohika  Three options to evaluate Test Process based on defects found  Estimate number of defects inherent in the system prior to test and estimate the number of defects to be found  Monitor the Defect Detection Percentage  Inject bugs into the program code and monitor the proportion to be found during testing  Reliability Growth Model Target numberof Defects= residual numberof defects effectiveness percentage
  • 88. (11)(11) SummarySummary Copyright Petro Porchuk 2010. Lohika ✔ For each Test Level suitable Test Metric should be applied ✔ Ensure compliance, if necessary through audits
  • 89. (12)(12) StandardsStandards Copyright Petro Porchuk 2010. Lohika  General Understanding of Standards  Standards related to Software Testing  Standards in the phases of the Test Process
  • 90. 12.1 General Understanding12.1 General Understanding Copyright Petro Porchuk 2010. Lohika  Standards are developed and maintained by national and international organizations  ISO – International Organization od Standardization  ANSI – American National Standard Institute  BSI – British Standard Institution  IEEE – Institute of Electrical and Electronics Engineers  Standard Types  Corporate Standards  Best Practices and Technical specifications  Domain Specific  Generally Applicable Standards
  • 91. 12.2 Main Standards related to Software Testing12.2 Main Standards related to Software Testing Copyright Petro Porchuk 2010. Lohika Standard Description BS 7925-1 British Standard: Software Testing, Part 1: Vocabulary, 1998 BS 7925-2 British Standard: Software Testing, Part 1: Software Component Testing, 1998 ISO 9000 A family of standards for quality management systems IEEE 610.12 1990 IEEE Standard Glossary of Software Engineering Terminology ISO 90003 Software Engineering Quality Management Standard IEEE 730 IEEE Standard for Software Quality Assurance Plans, 2002 IEEE 1008 ANSI/IEEE 1008-1987 IEEE Standard for Software Unit Testing IEEE 1012 IEEE Std 1012-1998 IEEE Standard for Software Verification and Validation ISO 9126-x IEEE Std 1044-1993 IEEE Standard Classification for Software Anomalies ISO 9241 ISO 9241-11:1998. Ergonomic requirements for office work with visual display terminals IEEE 1044 IEEE Std 1044-1993 IEEE Standard Classification for Software Anomalies IEEE 829 IEEE 829-1998, Standard for Software Test Documentation IEEE 1059 IEEE Std 1059-1993 IEEE Guide for Software Verification and Validation Plans
  • 92. 12.3 Standards in the phases of the Test Process12.3 Standards in the phases of the Test Process Copyright Petro Porchuk 2010. Lohika Type of Standard Test Phase Terminology and contracts Processes Products Documentation Methods and techniques Test Planning Test Control BS 7925-1 ISO 2382 ISO 9000 ISO 12207-0 IEEE 610.12 ISO 12207-0 ISO 9000-3 IEEE 730 IEEE 1008 IEEE 1012 IEEE 1061 ISO 9126-x ISO 12119 ISO 12207-1 ISO 15026 IEEE 982.1 IEEE 1228 IEEE 730 IEEE 829 IEEE 1012 IEEE 1228 ISO 12207-2 ISO 16085 IEEE 730.1 IEEE 982.2 IEEE 1059 Test analysis Test design ISO 14598-4 ISO 15408 IEEE 1062 IEEE 1228 IEEE 1008 IEEE 1012 ISO 15939 ISO 9241 ISO 12119 IEEE 1044 IEEE 982.2 IEEE 829 IEEE 1063 BS 7925-2 IEEE 1028 ISO 60812 EN 61508-3 EN 50128 IEC 61025 ISO 5806 Test realization Test execution IEEE 1008 ISO 9241 IEEE 828 IEEE 1209 IEEE 1348 IEEE 829 IEEE 1209 IEEE 1348 IEEE 1028 ETSI 201-873 IEEE 1042 IEEE 1209 IEEE 1348 Test evaluation Reporting IEEE 1008 IEEE 982.2 IEEE 1044 IEEE 829 IEEE 1044.1 ISO 12119
  • 93. (13)(13) SummarySummary Copyright Petro Porchuk 2010. Lohika ✔ Test Manager shouldn't reinvent the wheel, use standards for Testing ✔ Ensure compliance, if necessary through audits