HS
Uploaded byhulk smash
PDF, PPTX31 views

PA2557_SQM_Lecture7 - Defect Prevention.pdf

This document discusses software quality management and defect prevention. It defines what a defect is as a problem that causes software to stop running or produce incorrect results. It discusses how errors by humans can lead to defects in software. Several defect prevention techniques are covered, including elements of agile processes like short iterations and continuous testing, as well as uncertainty management techniques like risk identification and analysis. Root cause analysis is presented as a way to determine the underlying causes of problems or defects.

Embed presentation

Download as PDF, PPTX
Software Quality Management PA2557 Defect prevention Conny Johansson
DEFECT?!
What is a Defect? • A problem which, if not removed, will cause software to stop running or produce incorrect results • Software defects correlates with quality characteristics such as customer satisfaction, correctness, reliability, etc.
Error – Defect – Failure A person makes an error.. …that can cause a failure in operation …that creates a defect in the software…
What is a Defect? – cont. • Error (mistake): a human action producing an incorrect result • Defect (bug, fault): a flaw in the system • Failure: a deviation of the system from its expected delivery
What Is a Defect? – cont. • Distinction: a defect is a state and a fault is an event. • If a calculator program doesn't check the divisor for 0 before dividing by it, that's a defect in the program. • Each time we run the program and it crashes because it divides by zero, that's a fault in the calculations. • If there is no way to recover and get the calculations right anyway, it's also a failure in the system.
What Is a Defect? – cont. Defect Fault Failure Coding Execution Result
Defects and Quality • Two key aspects of quality – Defect density of software applications • Measures the number of errors or bugs found in artifacts divided by the size of the component or system • Requirements, design, source-code, user manuals, bad fixes or defects resulting from repairs – Defect removal efficiency of the operations • Ratio between bugs found during development and during the defined period of measurement, in operation
• Different definitions for software quality – Low level of defects – High reliability – User satisfaction – Quick answers – Rapid repairs for defects – Compliance to requirements Factors Affecting Software Quality • Different origins – Users – Stakeholders – Human factor – Hardware limitations – Requirements – Design – Source code – User manual – Bad fixes or repairs – Flawed test cases
Factors Affecting Software Quality • Root causes – Inadequate training – Inadequate cost estimation to fix defects – Excessive schedule pressure – Insufficient defect removal methods • Defect elimination strategies – Effective defect prevention – High defect removal efficiency – Accurate defect prediction – Accurate defect tracking Ø See for example Standish Group example presented at previous lectures
Defects & Software Process Improvement
DEFECT DETECTION?!
Defect Detection Methods • Static test techniques (No execution of the system) – Reviews/Inspections/Walkthroughs (static techniques) • Requirements • Design • Code – Pair work/programming – Audits • Dynamic test techniques (Execute the system) – Unit – Integration – Function – Feature – System
DEFECT PREVENTION?!
Defect prevention • It is easy to understand, but a challenge to implement • Software systems complexity increases • Software systems are used more intensively • Software is designed by humans (still), and humans do mistakes • If we do not act, the number of defects will increase • Defect prevention is more of a mindset
Why defect prevention? • Did you remember? – 57% of contracts experienced cost overruns – Cost overruns totaled £9.0 billion – The average percentage cost overrun is 30.5% – 33% of contracts suffered major delays – 30% of contracts were terminated
Why defect prevention? • Fixing errors is a waste of time? • Some reports indicate that up to 50% of the development bill is spent on bug-fixing • Do you ever believe that the customers wants to pay for bug-fixing? • Defects can be life-threatening (space programs, pacemakers, nuclear plants, etc.) • Back in time, there are examples of a misplaced ; in a Fortran program, causing severe failures in space programs • In Ariane 5, 1996, a 64 bit floating point number relating to the horizontal velocity of the rocket with respect to the platform was converted to a 16 bit signed integer. The rocket cost was $7 billion and its cargo was worth $500 million.
More famous software errors • NASA’s Mars orbit (1998) – failed to convert from English units to meter, cost was estimated to $125 million • Heathrow terminal 5 opening 2008 – during the first 10 days 140000 bags failed to travel with their owners and 500 flights were cancelled. Cause: Did not consider when bags were removed manually. • Denver airport luggage problem 1995 – 16 months late, $560 million over budget. Cause: Scope creep. Corrective action: A new manual system was built instead.
Defect prevention – principles (Humphrey) • Programmers must evaluate their own errors • Feedback of performance is essential • There is no single cure-all that will solve all problems • Defect prevention includes process improvement • Defect prevention takes time, i.e. costs money
Steps to Defect Prevention (Humphrey) 1. Defect reporting 2. Root cause analysis 3. Action plan development 4. Action implementation 5. Performance tracking 6. Starting over.. • So.. We can use PDCA and/or GQM+Strategies for steps to Defect prevention
ISO9001:2015, clause 10.1 10. Improvement 10.1 General The organization shall determine and select opportunities for improvement and implement any necessary actions to meet customer requirements and enhance customer satisfaction. These shall include: a) improving products and services to meet requirements as well as to address future needs and expectations; b) correcting, preventing or reducing undesired effects; c) improving the performance and effectiveness of the quality management system
Defect prevention techniques • Elements of Agile processes • Uncertainty management • Audits
Agile Manifesto • Individuals and interaction over processes and tools • Working software over comprehensive documentation • Customer collaboration over contract negotiation • Responding to change over following a plan • That is, while there is value in the items to the right, we value the items to the left more.
Agile programming methodology • Short iterations • Frequent deliveries • Open communication • Close collaboration • Tight teaming • Simplicity • Refactoring • Continuous testing • Proactive management
Some agile ideas • Involve the customer – most important part first • Short delivery cycles – quick response • The idea behind agile development is that the whole team should be empowered to make decisions – the term “project manager” is sometimes not used. • EXPECT THE SYSTEM AND THE PROJECT TO CHANGE
Agile planning • “The customer has the right to see the overall plan” (Kent Beck). • The planning game – Release planning – Sprint planning (2-4 weeks) • Small teams • The focus is on a working system; every day, week, every second week, month, or even every second month. • User story focus. Feature focus. Functional focus. • Bi-weekly, weekly and/or daily (morning) planning meetings
Agile Sprints • Sprint planning – Velocity planning, – Identification and estimation of activities – Story point estimation, relative estimations – Planning poker (Wideband Delphi) – No detailed scheduling • Sprint follow-up – Task board – Daily scrum – Burndown chart (focus on the time left for finishing an activity)
Burndown charts
Kanban techniques • Developed by Toyota • Use lean and just-in-time concepts • The problem/challenge is to get things done! • Kanban simply described: – Limit the number of on-going tasks! – Use visual communication (flags, signs, post-it-notes) – Eliminate bottlenecks – focus on solving these problems – Continuously improve your working process – feedback. • Communicate visually: 1. Backlog items 2. On-going (in-progress) items 3. Finished items
Agile Retrospectives • "At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.” • A meeting in the end of the sprint • What happened during the sprint? • Each member of the team should answer – What worked well for us? – What did not work well for us? – What actions can we take to improve our process going forward?
Agile Retrospectives • Organized by the Scrum Master • Time-boxed • “Lessons learned” • What changes shall we make in the next iteration? • Retrospectives are team-driven – the team decides how the meetings shall be run • Built on honesty and trust • Stresses continuous improvement • Identify actions for improvement
Agile Retrospectives • “Everyone did the best job they could” – given what they knew at the time – their skills and abilities – the resources available – the situation at hand. • By some people considered as the most important Agile practice • Sometimes you search for the root cause, sometimes you don’t
Agile Retrospectives • Negative comments are not bad • Negative comments are necessary • Try to not make negative comments personalized (but this is hard) • Focus participation in the retrospectives • You need data in order to discuss what you did well • The team should vote on which items to focus on in the next sprint
Retrospectives – Do NOT • Focus on individual performance feedback – de-personalize • Provide the answer for inadequate technical skills or collaboration skills • Make “complaint sessions” of the retrospectives • List too many improvements • Keep the same retrospective process if people are “bored” – change and adapt the process
Uncertainty Management (PMBOK) • “The Uncertainty Performance Domain addresses activities and functions associated with risk and uncertainty”. • Uncertainty: A lack of understanding and awareness of issues, events, paths to follow, or solutions to pursue. • Ambiguity: A state of being unclear, having multiple options from which to choose • Complexity: A characteristic of a project that is difficult to manage. • Volatility: The possibility for rapid and unpredictable change.
Options for Responding to Uncertainty • Gather information – conduct research, engage experts • Prepare for multiple outcomes – prepare for the different outcomes • Set-based design – multiple designs, explore options • Build on resilience – adapt and respond quickly to unexpected changes
Options for Responding to Uncertainty, cont. • Progressive elaboration • Experiment • Prototyping • Simulation • Diverse perspectives • Build in redundancy • Alternatives analysis • Cost reserve
Uncertainty management • “Project management is uncertainty management” • Identify potential problems before they occur • Put preventive actions in place before unrecoverable harm occurs • Risk – possible event that may affect the project negatively (Old definition) • Risk: An uncertain event or condition that, if it occurs, has a positive or negative effect on one or more project objectives. (New definition, PMBOK)
Risk management • Problems – events that have and/or will affect the project negatively • Stages in risk management: – Risk identification; project, product and business risks – Risk analysis; probability and consequences, triggers – Risk planning; how to address risks – eliminate, reduce, ignore – Risk monitoring; controlling and updating the risks • In risk management you deal with both threats and opportunities! • Risk factor – it is common that likelihood*consequence is used, e.g., with the scales 1-5
Risk management, cont. Categories • Technology • People • Organizational • Tools • Requirements • Estimations Top ten risks according to Boehm (91) • Personnel Shortfalls • Unrealistic schedules/budgets • Developing the wrong software functions • Developing the wrong user interface • Gold plating • Continuing stream of requirements changes • Shortfalls in externally furnished components • Shortfalls in externally performed tasks • Real-time performance shortfalls • Straining computer science capabilities
Top ten risks according to codebots.com (2020) 1. Inaccurate estimations 2. Scope variations 3. End-user engagement 4. Stakeholder expectations 5. Poor quality code 6. Poor productivity 7. Inadequate risk management 8. Low stakeholder engagement 9. Inadequate human resources 10. Lack of ownership
Risks - Top seven list (Literature survey by Arnuphaptrairong 2011) 1. Misunderstanding of requirements 2. Lack of management commitment and support 3. Lack of adequate user involvement 4. Failure to gain user commitment 5. Failure to manage end user expectation 6. Changes to requirements 7. Lack of an effective project management methodology
Risk identification • Identify major threats • Should involve team members, stakeholders, etc • Risk types: – Technology – People – Organizational – Tools – Requirements – Estimations • Reduce the list to a manageable size
Risk analysis • Analyze and discuss the risks identified • Group risks • Merge risks • Make the risks specific • Analyze probability and consequence, discuss in workshops e.g. • Different people have different opinions – consolidation needed • Rank the risks, voting? • Focus on a limited number of risks • Risk factor – it is common that likelihood*consequence is used, e.g., with the scales 1-5
Risk analysis example Risk P - Probabili ty (1-5) C - Consequenc e (1-5) Risk factor P*C Consequence description Mitigation System test may be scheduled for a too short period 4 5 20 More faults in delivered product Reduce scope and/or allocate more resources Poor estimations for x module 3 3 9 All stories will not be implemented Continuously refine estimations Project will have low priority 1 5 5 Resources will be moved to other projects No mitigation – just monitor Poor competence in React framework 4 3 12 Poor quality and delays Schedule education and training
Risk planning • Focus on the top risks • Define strategy for each top risk. • For threats choose between: – Accept (but monitor) – Mitigate (reduce) – Avoid (remove) – Escalate (threat is outside the scope of the project) – Transfer (shifting ownership)
Risk planning • Define strategies for handling opportunities. • For opportunities choose between: – Exploit: ensure that it occurs – Escalate: it is outside the scope of the project – Share: shift ownership – Enhance: increase the probability of occurrence – Accept: accept but no action is planned
Risk monitoring • Check the status of the risks • Update and take care of risk changes • Probability changed? • Consequence changed? • New risks? Top risks? • Not only continuously within the team, but also include the stakeholders continuously
Risk management – common mistakes • Risk analysis is often done early in the project, and then never considered again • Risk analysis is done by the project manager, solely • Risks from the development is not tracked • Top risks are not selected • The risk is already a problem • Corrective actions are not executed • The organisation’s risk process is not followed
Problem management • A risk can happen – become a problem • If it is a problem – do not ignore it! • There is a high risk that the problem will generate new, worse problems • An existing problem can propagate into something worse • Initiate actions immediately for eliminating the problem. Prioritize this action.
Tips on Uncertainty and Problem Management • Report openly and honestly • Document so that you can go back in time • Prepare proposals for action • Get stakeholders and upper management acceptance for actions • Request help if needed • Continuously update and report status of uncertainties and problems!!!
Root Cause Analysis (RCA) process - example • What is the root cause of a problem, fault, customer dissatisfaction, etc.? • A workshop is to prefer – Introduction – Describe what happened – WHAT-analysis – consequences – WHY-analysis – causes – Action definition – avoiding – Summary – responsibility and follow-up
RCA Process guideline • Introduction – Presentation RCA workshop, present the planning – Handshake the problem description – Give an account of the course of events – Be sure that everyone understands the purpose and goal of this session • Describe what happened. – Collect and organize the facts surrounding the event to understand what happened. See: Guidance_for_RCA (note the error in the headline for Step 2, page 2)
RCA Process guideline • WHAT are the consequences? – Brainstorming (preferably round-table discussion). Both negative and positive consequences should be analyzed. – Prioritization (simple) – Hints concerning affected areas for WHAT: cost (internal/external), internal efficiency, customer satisfaction, employee motivation, product quality
RCA Process guideline, cont. • WHY did the fault occur? – Brainstorming (quiet – write on stickers). Limit the number to 7 notes. – Walk-through, the participants shall, one at a time, read their notes. Perform the grouping by letting each participant “suggests” the potential belonging for each issue – Ask the question WHY for every cause in order to identify the “real” root cause – If necessary: grouping (e.g. divide into teams who perform the grouping)
RCA Process guideline, cont. • WHY did the fault occur? – Estimation of the effect and realization difficulties, if this cause is solved (use the ”four-fielder” – in Swedish: fyrfältare) – Prioritization, everyone have e.g., 7 ”dots” – Examples of potential groups: Poor documentation, processes, competence, management, rules, communication, handovers, Milestone/TG passages, etc.
RCA Process guideline, cont. • Options for brainstorming: • WHERE (was (should) the fault (have been) detected? • WHEN (was (should) the fault (have been) detected?) • Define Actions – The goal is to minimize the chance of having the same problem again – Choose a suitable number of “Why” (use the prioritized list) – Divide into groups and let each group select one or two “Why” – Each group shall propose Let each group present their proposal, adjust if necessary Action Time plan Responsible
RCA Process guideline, cont. • Who will document and distribute the result from the workshop? • Who is responsible for tracking the actions? • Budget/payment?
RCA workshop agenda 1. Introduction 2. Failure/problem definition consensus 3. Course of events 4. Consequence analysis 5. Cause analysis 6. Cause prioritization/grading 7. Identification of response actions 8. Conclusion/summary
RCA brainstorming guidelines • Be specific, try to avoid being too general, example: “insufficient testing” is (perhaps) too general. • One sticker==one issue • Initials in order to identify who wrote it • Write legible, short sentences if possible
Audit process To evaluate: – Software elements – The processes for producing them – Projects – Entire quality programs – To check the compliance to ISO9001, TickITplus, CMMI – To find elements for improvement – To give an independent view of the status – Are you following the processes defined? – Can you find requested information
When to Audit • A project milestone, calendar date, or other criterion has been (or is to be) met • The audit is initiated by earlier plans • External parties, stakeholders, require an audit • A local organizational element has requested the audit
Audit plans • Which project processes to examine? • Which software to examine. Sampling? Selection criteria? Sample size? • Reporting requirements. Necessary improvements? Recommendations? Findings? Defects? • Required follow-up activities • Activities, elements, and procedures necessary to meet the scope of the audit
Audit plan, cont. • Criteria that provide the basis for determining compliance (provided as input) • Audit procedures and checklists • Audit personnel requirements (e.g., number, skills, experience, and responsibilities) • Organizations involved in the audit • Date, time, place, agenda, and intended auditees
Audits – overview meeting • Overview of existing agents (e.g., audit scope, plan, and related contracts) • Overview of production and processes being audited • Overview of the audit process, its objectives, and outputs • Expected contributions of the audited organization to the audit process (the number of people to be interviewed, meeting facilities, etc.) • Specific audit schedule
Audit preparation • Understand the organization • Understand the products and processes • Understand the objective audit criteria • Prepare for the audit report • Detail the audit plan
Audit preparation, cont. • Team orientation and training • Facilities for audit interviews • Materials, documents, and tools required by the audit procedures • The software elements to be audited (e.g., documents, computer files, personnel to be interviewed) • Scheduling interviews
Audits – criteria for completion, examples • Each element within the scope has been examined • Response to draft findings have been received • Findings have been presented • A formal report has been written and distributed • A recommendation plan has been presented • Follow-up actions have been identified, planned, executed, followed-up and completed. • Verification of effectiveness of taken actions
The Audit shall not be perceived as … • An eager attempt to find a lot of deficiencies • An occasion for the auditor to show off • Forced by upper management (or quality department) • An activity which takes up unnecessary time for the participants • A necessary evil that must be done • Something you can forget about after it has been done • The issues found are too heavy to solve
Audits – Common problems • Issues are not taken care of/solved • Issues are not linked to business goals à The focus is NOT on “how well”, but on “out of control” • Goals and objectives are neither defined nor followed-up • Only project managers are audited. ”Be where the action is” • The follow-up is too weak • Upper management have poor focus/commitment • Project Management, PM, has poor focus • The priority on solving issues are low • Solving the issues seems to be a burden for the organization • Work pressure results in low priority for preventive actions
Audits – Item ratings RED The issue is not under control. Several lacks/deficiencies have been identified. YELLOW The issue is partly under control. A few lacks/deficiencies have been identified. GREEN The issue is under control. No major/severe deficiencies could be identified. BLACK Not applicable, or the issue was not handled/discussed during the audit.
Experiences from Industrial Audits • Mostly under control: – Progress reporting – CM handling – Document handling – Risk analysis – Test environment – Change request handling – Sponsoring – Product handling – Service/Support handling – Quite often – continuous improvement, the heart of ISO9001
Experiences from Industry Audits • Needs improvement: • Using defined and tailored processes • Reviews – planning and execution • Communication • Follow-up of risks – risk management • Statement of compliance – “we know we can not fulfil the requirements but have not negotiated it” • Backwards compatibility • Hardware requirements and availability of hardware • Traceability • Goals and Measurements • Management commitment
More Defect Prevention Methods? Or just follow existing processes? • Training of Personnel • Pair Programming • Tools • Test Driven Development (TDD) • Short sprints • Structured Design (Object Oriented) • Requirements tracking • Demos • Pilots • 7 management tools • WBS • Estimating methods • Frequent feedback meetings • Etc.
PA2557_SQM_Lecture7 - Defect Prevention.pdf

More Related Content

PPTX
IT Quality Testing and the Defect Management Process
byYolanda Williams
 
PPT
March APLN: Agile development- Measure & Analyze by Garry Rowland
byConscires Agile Practices
 
PPTX
Software Testing defect management PPT slides
byNeerajEmanuel
 
PDF
Chap 4 Defect Management of the software .pdf
bySANGEETA185482
 
PPTX
Software Testing Chapter four notes.pptx
bySANGEETA185482
 
PPTX
Develop a Defect Prevention Strategy—or Else!
byTechWell
 
PPT
From defect reporting to defect prevention
byBestBrains
 
PDF
Agile adoption tales from the coalface
byNish Mahanty
 
IT Quality Testing and the Defect Management Process
byYolanda Williams
 
March APLN: Agile development- Measure & Analyze by Garry Rowland
byConscires Agile Practices
 
Software Testing defect management PPT slides
byNeerajEmanuel
 
Chap 4 Defect Management of the software .pdf
bySANGEETA185482
 
Software Testing Chapter four notes.pptx
bySANGEETA185482
 
Develop a Defect Prevention Strategy—or Else!
byTechWell
 
From defect reporting to defect prevention
byBestBrains
 
Agile adoption tales from the coalface
byNish Mahanty
 

Similar to PA2557_SQM_Lecture7 - Defect Prevention.pdf

KEY
Another Agile Intro
bySteve Hayes
 
PPTX
Top 10 Agile Metrics
byXBOSoft
 
PPT
From Defect Reporting To Defect Prevention
bySune Gynthersen
 
PDF
Defect Prevention & Predictive Analytics - XBOSoft Webinar
byXBOSoft
 
PDF
Agile practices for management
byIcalia Labs
 
PPT
Intro to Agile Practices and Values
byOpenSource Connections
 
PDF
William "RED" Davidson Presentation
byHyperdrive Agile Leadership (powered by Bratton & Company)
 
PPTX
7-SOFTWARE QUALITY ENGINERING-Video.pptx
bysagar222612
 
PPTX
SQA Lec_4.ppt learn sqa knowledge using this pptx
bytk71577
 
PPTX
Applying TQM and the Toyota Production System in Development of Software Arti...
byDave Litwiller
 
PDF
Agile india 2012 sonali bhasin
bySonali Bhasin
 
PPTX
ATC2013-Thiru and Abhishek-How to prevent Agile from becoming Fragile?
byIndia Scrum Enthusiasts Community
 
PDF
What is agile
byOrange and Bronze Software Labs
 
PDF
Caring about Code Quality
bySaltmarch Media
 
PPTX
Agile Development Product Delivery For Successful Organizations
byMarc Crudgington, MBA
 
PPTX
Trip 3 of PMO Company's journey to Agile & Scrum savvyness
byGetjan Lammers
 
PPTX
WinSmart Technologies
bybijunairk
 
PPT
Software engineering unit V-2 notes in the ppt format
byhodcsm5
 
PPT
02 processraison
bySaul Fernando Godinez Cruz
 
PDF
Moving from Testing to Quality by Jeff Van Fleet
byQA or the Highway
 
Another Agile Intro
bySteve Hayes
 
Top 10 Agile Metrics
byXBOSoft
 
From Defect Reporting To Defect Prevention
bySune Gynthersen
 
Defect Prevention & Predictive Analytics - XBOSoft Webinar
byXBOSoft
 
Agile practices for management
byIcalia Labs
 
Intro to Agile Practices and Values
byOpenSource Connections
 
William "RED" Davidson Presentation
byHyperdrive Agile Leadership (powered by Bratton & Company)
 
7-SOFTWARE QUALITY ENGINERING-Video.pptx
bysagar222612
 
SQA Lec_4.ppt learn sqa knowledge using this pptx
bytk71577
 
Applying TQM and the Toyota Production System in Development of Software Arti...
byDave Litwiller
 
Agile india 2012 sonali bhasin
bySonali Bhasin
 
ATC2013-Thiru and Abhishek-How to prevent Agile from becoming Fragile?
byIndia Scrum Enthusiasts Community
 
What is agile
byOrange and Bronze Software Labs
 
Caring about Code Quality
bySaltmarch Media
 
Agile Development Product Delivery For Successful Organizations
byMarc Crudgington, MBA
 
Trip 3 of PMO Company's journey to Agile & Scrum savvyness
byGetjan Lammers
 
WinSmart Technologies
bybijunairk
 
Software engineering unit V-2 notes in the ppt format
byhodcsm5
 
02 processraison
bySaul Fernando Godinez Cruz
 
Moving from Testing to Quality by Jeff Van Fleet
byQA or the Highway
 

More from hulk smash

PDF
PA2557_SQM_Lecture6 - Process improvement processes. GQM+Strategies.pdf
byhulk smash
 
PDF
PA2557_SQM_Lecture5 - Customer satisfaction, Product Management.pdf
byhulk smash
 
PDF
PA2557_SQM_Lecture1 - Course Introduction.pdf
byhulk smash
 
PDF
PA2557_SQM_Lecture2 - Quality Basics.pdf
byhulk smash
 
PDF
PA2557_SQM_Lecture3 - Process Improvement.pdf
byhulk smash
 
PDF
PA2557_SQM_Lecture4 - Process improvement and Process Maturity.pdf
byhulk smash
 
PA2557_SQM_Lecture6 - Process improvement processes. GQM+Strategies.pdf
byhulk smash
 
PA2557_SQM_Lecture5 - Customer satisfaction, Product Management.pdf
byhulk smash
 
PA2557_SQM_Lecture1 - Course Introduction.pdf
byhulk smash
 
PA2557_SQM_Lecture2 - Quality Basics.pdf
byhulk smash
 
PA2557_SQM_Lecture3 - Process Improvement.pdf
byhulk smash
 
PA2557_SQM_Lecture4 - Process improvement and Process Maturity.pdf
byhulk smash
 

Recently uploaded

PPTX
Salesforce_Food_Distribution_Project.pptx
byfbinsta3426
 
PPTX
Ultra_Detailed_Acute_Severe_Asthma_Paediatrics_PPT.pptx
byHimadrimohapatra1
 
DOCX
Case Frameworks for casing help MBB guid
byMaybeSomeplace
 
PPTX
dum dum jnttgkntrgknkntrkngjntrbj bj. 2.pptx
byshounakban2033
 
PDF
reStartEvents 12:11 TS:SCI & Above Employer Directory.pdf
byKen Fuller
 
PPT
Hydrosphere, oceanic structure,ocean depthetc
bykisankumar4241
 
PPTX
A case of snake bite in Dang-1_35020aad-f49e-4fae-8bb2-f0525282d34a.pptx
byanuRegmi3
 
PPTX
Poston Boston University appointment 2008
byRobert Poston
 
PPTX
Trained Codependent Leader-Follower Dynamic (Dark Theme).pptx
byKhalizan Halid
 
PDF
Hybrid Work, Future Skills & the Human Edge in Asia’s Workplace
byBase Camp
 
PDF
Psychotic Leadership - Signs, Causes, Impacts, Remedies
byKhalizan Halid
 
PPTX
WEPP MODEL for computing soil erosion .pptx
bysnehilchandrakar
 
PPTX
Evaluation and Benchmarking of LLM Agents: Full Slide Deck (2025).pptx
byroshinijoga
 
PPT
Introduction to Services Selection Board (SSB)
bykrishmirji
 
PDF
Latitude & Longitude ,timezone and international Time zone
bykisankumar4241
 
PDF
IGC 1 meetingg no 1+ 2 exam preparation document.pdf
bychatif3
 
PDF
Lecture 1 the 3rd line of defence ; Adaptive immunity.pdf
bylangkvin
 
PPTX
AI-Powered-Comment-Analysis-Platform.pptx
bydevinjon990
 
PDF
CURRICULUM VITAE THOMAS K. GIRARD, MA, CDP CONTACT info@thomaskgirard.com
byThomas GIRARD, MA, CDP
 
PDF
Promoting mental health and wellness among students.pdf
bysangwanmnk
 
Salesforce_Food_Distribution_Project.pptx
byfbinsta3426
 
Ultra_Detailed_Acute_Severe_Asthma_Paediatrics_PPT.pptx
byHimadrimohapatra1
 
Case Frameworks for casing help MBB guid
byMaybeSomeplace
 
dum dum jnttgkntrgknkntrkngjntrbj bj. 2.pptx
byshounakban2033
 
reStartEvents 12:11 TS:SCI & Above Employer Directory.pdf
byKen Fuller
 
Hydrosphere, oceanic structure,ocean depthetc
bykisankumar4241
 
A case of snake bite in Dang-1_35020aad-f49e-4fae-8bb2-f0525282d34a.pptx
byanuRegmi3
 
Poston Boston University appointment 2008
byRobert Poston
 
Trained Codependent Leader-Follower Dynamic (Dark Theme).pptx
byKhalizan Halid
 
Hybrid Work, Future Skills & the Human Edge in Asia’s Workplace
byBase Camp
 
Psychotic Leadership - Signs, Causes, Impacts, Remedies
byKhalizan Halid
 
WEPP MODEL for computing soil erosion .pptx
bysnehilchandrakar
 
Evaluation and Benchmarking of LLM Agents: Full Slide Deck (2025).pptx
byroshinijoga
 
Introduction to Services Selection Board (SSB)
bykrishmirji
 
Latitude & Longitude ,timezone and international Time zone
bykisankumar4241
 
IGC 1 meetingg no 1+ 2 exam preparation document.pdf
bychatif3
 
Lecture 1 the 3rd line of defence ; Adaptive immunity.pdf
bylangkvin
 
AI-Powered-Comment-Analysis-Platform.pptx
bydevinjon990
 
CURRICULUM VITAE THOMAS K. GIRARD, MA, CDP CONTACT info@thomaskgirard.com
byThomas GIRARD, MA, CDP
 
Promoting mental health and wellness among students.pdf
bysangwanmnk
 

PA2557_SQM_Lecture7 - Defect Prevention.pdf

  • 1.
  • 2.
  • 3.
    What is aDefect? • A problem which, if not removed, will cause software to stop running or produce incorrect results • Software defects correlates with quality characteristics such as customer satisfaction, correctness, reliability, etc.
  • 4.
    Error – Defect– Failure A person makes an error.. …that can cause a failure in operation …that creates a defect in the software…
  • 5.
    What is aDefect? – cont. • Error (mistake): a human action producing an incorrect result • Defect (bug, fault): a flaw in the system • Failure: a deviation of the system from its expected delivery
  • 6.
    What Is aDefect? – cont. • Distinction: a defect is a state and a fault is an event. • If a calculator program doesn't check the divisor for 0 before dividing by it, that's a defect in the program. • Each time we run the program and it crashes because it divides by zero, that's a fault in the calculations. • If there is no way to recover and get the calculations right anyway, it's also a failure in the system.
  • 7.
    What Is aDefect? – cont. Defect Fault Failure Coding Execution Result
  • 8.
    Defects and Quality •Two key aspects of quality – Defect density of software applications • Measures the number of errors or bugs found in artifacts divided by the size of the component or system • Requirements, design, source-code, user manuals, bad fixes or defects resulting from repairs – Defect removal efficiency of the operations • Ratio between bugs found during development and during the defined period of measurement, in operation
  • 9.
    • Different definitions forsoftware quality – Low level of defects – High reliability – User satisfaction – Quick answers – Rapid repairs for defects – Compliance to requirements Factors Affecting Software Quality • Different origins – Users – Stakeholders – Human factor – Hardware limitations – Requirements – Design – Source code – User manual – Bad fixes or repairs – Flawed test cases
  • 10.
    Factors Affecting SoftwareQuality • Root causes – Inadequate training – Inadequate cost estimation to fix defects – Excessive schedule pressure – Insufficient defect removal methods • Defect elimination strategies – Effective defect prevention – High defect removal efficiency – Accurate defect prediction – Accurate defect tracking Ø See for example Standish Group example presented at previous lectures
  • 11.
    Defects & SoftwareProcess Improvement
  • 12.
  • 13.
    Defect Detection Methods •Static test techniques (No execution of the system) – Reviews/Inspections/Walkthroughs (static techniques) • Requirements • Design • Code – Pair work/programming – Audits • Dynamic test techniques (Execute the system) – Unit – Integration – Function – Feature – System
  • 14.
  • 15.
    Defect prevention • Itis easy to understand, but a challenge to implement • Software systems complexity increases • Software systems are used more intensively • Software is designed by humans (still), and humans do mistakes • If we do not act, the number of defects will increase • Defect prevention is more of a mindset
  • 16.
    Why defect prevention? •Did you remember? – 57% of contracts experienced cost overruns – Cost overruns totaled £9.0 billion – The average percentage cost overrun is 30.5% – 33% of contracts suffered major delays – 30% of contracts were terminated
  • 17.
    Why defect prevention? •Fixing errors is a waste of time? • Some reports indicate that up to 50% of the development bill is spent on bug-fixing • Do you ever believe that the customers wants to pay for bug-fixing? • Defects can be life-threatening (space programs, pacemakers, nuclear plants, etc.) • Back in time, there are examples of a misplaced ; in a Fortran program, causing severe failures in space programs • In Ariane 5, 1996, a 64 bit floating point number relating to the horizontal velocity of the rocket with respect to the platform was converted to a 16 bit signed integer. The rocket cost was $7 billion and its cargo was worth $500 million.
  • 18.
    More famous softwareerrors • NASA’s Mars orbit (1998) – failed to convert from English units to meter, cost was estimated to $125 million • Heathrow terminal 5 opening 2008 – during the first 10 days 140000 bags failed to travel with their owners and 500 flights were cancelled. Cause: Did not consider when bags were removed manually. • Denver airport luggage problem 1995 – 16 months late, $560 million over budget. Cause: Scope creep. Corrective action: A new manual system was built instead.
  • 19.
    Defect prevention –principles (Humphrey) • Programmers must evaluate their own errors • Feedback of performance is essential • There is no single cure-all that will solve all problems • Defect prevention includes process improvement • Defect prevention takes time, i.e. costs money
  • 20.
    Steps to DefectPrevention (Humphrey) 1. Defect reporting 2. Root cause analysis 3. Action plan development 4. Action implementation 5. Performance tracking 6. Starting over.. • So.. We can use PDCA and/or GQM+Strategies for steps to Defect prevention
  • 21.
    ISO9001:2015, clause 10.1 10.Improvement 10.1 General The organization shall determine and select opportunities for improvement and implement any necessary actions to meet customer requirements and enhance customer satisfaction. These shall include: a) improving products and services to meet requirements as well as to address future needs and expectations; b) correcting, preventing or reducing undesired effects; c) improving the performance and effectiveness of the quality management system
  • 22.
    Defect prevention techniques •Elements of Agile processes • Uncertainty management • Audits
  • 23.
    Agile Manifesto • Individualsand interaction over processes and tools • Working software over comprehensive documentation • Customer collaboration over contract negotiation • Responding to change over following a plan • That is, while there is value in the items to the right, we value the items to the left more.
  • 24.
    Agile programming methodology • Shortiterations • Frequent deliveries • Open communication • Close collaboration • Tight teaming • Simplicity • Refactoring • Continuous testing • Proactive management
  • 25.
    Some agile ideas •Involve the customer – most important part first • Short delivery cycles – quick response • The idea behind agile development is that the whole team should be empowered to make decisions – the term “project manager” is sometimes not used. • EXPECT THE SYSTEM AND THE PROJECT TO CHANGE
  • 26.
    Agile planning • “Thecustomer has the right to see the overall plan” (Kent Beck). • The planning game – Release planning – Sprint planning (2-4 weeks) • Small teams • The focus is on a working system; every day, week, every second week, month, or even every second month. • User story focus. Feature focus. Functional focus. • Bi-weekly, weekly and/or daily (morning) planning meetings
  • 27.
    Agile Sprints • Sprintplanning – Velocity planning, – Identification and estimation of activities – Story point estimation, relative estimations – Planning poker (Wideband Delphi) – No detailed scheduling • Sprint follow-up – Task board – Daily scrum – Burndown chart (focus on the time left for finishing an activity)
  • 28.
  • 29.
    Kanban techniques • Developedby Toyota • Use lean and just-in-time concepts • The problem/challenge is to get things done! • Kanban simply described: – Limit the number of on-going tasks! – Use visual communication (flags, signs, post-it-notes) – Eliminate bottlenecks – focus on solving these problems – Continuously improve your working process – feedback. • Communicate visually: 1. Backlog items 2. On-going (in-progress) items 3. Finished items
  • 30.
    Agile Retrospectives • "Atregular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.” • A meeting in the end of the sprint • What happened during the sprint? • Each member of the team should answer – What worked well for us? – What did not work well for us? – What actions can we take to improve our process going forward?
  • 31.
    Agile Retrospectives • Organizedby the Scrum Master • Time-boxed • “Lessons learned” • What changes shall we make in the next iteration? • Retrospectives are team-driven – the team decides how the meetings shall be run • Built on honesty and trust • Stresses continuous improvement • Identify actions for improvement
  • 32.
    Agile Retrospectives • “Everyonedid the best job they could” – given what they knew at the time – their skills and abilities – the resources available – the situation at hand. • By some people considered as the most important Agile practice • Sometimes you search for the root cause, sometimes you don’t
  • 33.
    Agile Retrospectives • Negativecomments are not bad • Negative comments are necessary • Try to not make negative comments personalized (but this is hard) • Focus participation in the retrospectives • You need data in order to discuss what you did well • The team should vote on which items to focus on in the next sprint
  • 34.
    Retrospectives – DoNOT • Focus on individual performance feedback – de-personalize • Provide the answer for inadequate technical skills or collaboration skills • Make “complaint sessions” of the retrospectives • List too many improvements • Keep the same retrospective process if people are “bored” – change and adapt the process
  • 35.
    Uncertainty Management (PMBOK) •“The Uncertainty Performance Domain addresses activities and functions associated with risk and uncertainty”. • Uncertainty: A lack of understanding and awareness of issues, events, paths to follow, or solutions to pursue. • Ambiguity: A state of being unclear, having multiple options from which to choose • Complexity: A characteristic of a project that is difficult to manage. • Volatility: The possibility for rapid and unpredictable change.
  • 36.
    Options for Respondingto Uncertainty • Gather information – conduct research, engage experts • Prepare for multiple outcomes – prepare for the different outcomes • Set-based design – multiple designs, explore options • Build on resilience – adapt and respond quickly to unexpected changes
  • 37.
    Options for Respondingto Uncertainty, cont. • Progressive elaboration • Experiment • Prototyping • Simulation • Diverse perspectives • Build in redundancy • Alternatives analysis • Cost reserve
  • 38.
    Uncertainty management • “Projectmanagement is uncertainty management” • Identify potential problems before they occur • Put preventive actions in place before unrecoverable harm occurs • Risk – possible event that may affect the project negatively (Old definition) • Risk: An uncertain event or condition that, if it occurs, has a positive or negative effect on one or more project objectives. (New definition, PMBOK)
  • 39.
    Risk management • Problems– events that have and/or will affect the project negatively • Stages in risk management: – Risk identification; project, product and business risks – Risk analysis; probability and consequences, triggers – Risk planning; how to address risks – eliminate, reduce, ignore – Risk monitoring; controlling and updating the risks • In risk management you deal with both threats and opportunities! • Risk factor – it is common that likelihood*consequence is used, e.g., with the scales 1-5
  • 40.
    Risk management, cont. Categories •Technology • People • Organizational • Tools • Requirements • Estimations Top ten risks according to Boehm (91) • Personnel Shortfalls • Unrealistic schedules/budgets • Developing the wrong software functions • Developing the wrong user interface • Gold plating • Continuing stream of requirements changes • Shortfalls in externally furnished components • Shortfalls in externally performed tasks • Real-time performance shortfalls • Straining computer science capabilities
  • 41.
    Top ten risksaccording to codebots.com (2020) 1. Inaccurate estimations 2. Scope variations 3. End-user engagement 4. Stakeholder expectations 5. Poor quality code 6. Poor productivity 7. Inadequate risk management 8. Low stakeholder engagement 9. Inadequate human resources 10. Lack of ownership
  • 42.
    Risks - Topseven list (Literature survey by Arnuphaptrairong 2011) 1. Misunderstanding of requirements 2. Lack of management commitment and support 3. Lack of adequate user involvement 4. Failure to gain user commitment 5. Failure to manage end user expectation 6. Changes to requirements 7. Lack of an effective project management methodology
  • 43.
    Risk identification • Identifymajor threats • Should involve team members, stakeholders, etc • Risk types: – Technology – People – Organizational – Tools – Requirements – Estimations • Reduce the list to a manageable size
  • 44.
    Risk analysis • Analyzeand discuss the risks identified • Group risks • Merge risks • Make the risks specific • Analyze probability and consequence, discuss in workshops e.g. • Different people have different opinions – consolidation needed • Rank the risks, voting? • Focus on a limited number of risks • Risk factor – it is common that likelihood*consequence is used, e.g., with the scales 1-5
  • 45.
    Risk analysis example RiskP - Probabili ty (1-5) C - Consequenc e (1-5) Risk factor P*C Consequence description Mitigation System test may be scheduled for a too short period 4 5 20 More faults in delivered product Reduce scope and/or allocate more resources Poor estimations for x module 3 3 9 All stories will not be implemented Continuously refine estimations Project will have low priority 1 5 5 Resources will be moved to other projects No mitigation – just monitor Poor competence in React framework 4 3 12 Poor quality and delays Schedule education and training
  • 46.
    Risk planning • Focuson the top risks • Define strategy for each top risk. • For threats choose between: – Accept (but monitor) – Mitigate (reduce) – Avoid (remove) – Escalate (threat is outside the scope of the project) – Transfer (shifting ownership)
  • 47.
    Risk planning • Definestrategies for handling opportunities. • For opportunities choose between: – Exploit: ensure that it occurs – Escalate: it is outside the scope of the project – Share: shift ownership – Enhance: increase the probability of occurrence – Accept: accept but no action is planned
  • 48.
    Risk monitoring • Checkthe status of the risks • Update and take care of risk changes • Probability changed? • Consequence changed? • New risks? Top risks? • Not only continuously within the team, but also include the stakeholders continuously
  • 49.
    Risk management –common mistakes • Risk analysis is often done early in the project, and then never considered again • Risk analysis is done by the project manager, solely • Risks from the development is not tracked • Top risks are not selected • The risk is already a problem • Corrective actions are not executed • The organisation’s risk process is not followed
  • 51.
    Problem management • Arisk can happen – become a problem • If it is a problem – do not ignore it! • There is a high risk that the problem will generate new, worse problems • An existing problem can propagate into something worse • Initiate actions immediately for eliminating the problem. Prioritize this action.
  • 52.
    Tips on Uncertaintyand Problem Management • Report openly and honestly • Document so that you can go back in time • Prepare proposals for action • Get stakeholders and upper management acceptance for actions • Request help if needed • Continuously update and report status of uncertainties and problems!!!
  • 53.
    Root Cause Analysis(RCA) process - example • What is the root cause of a problem, fault, customer dissatisfaction, etc.? • A workshop is to prefer – Introduction – Describe what happened – WHAT-analysis – consequences – WHY-analysis – causes – Action definition – avoiding – Summary – responsibility and follow-up
  • 54.
    RCA Process guideline •Introduction – Presentation RCA workshop, present the planning – Handshake the problem description – Give an account of the course of events – Be sure that everyone understands the purpose and goal of this session • Describe what happened. – Collect and organize the facts surrounding the event to understand what happened. See: Guidance_for_RCA (note the error in the headline for Step 2, page 2)
  • 55.
    RCA Process guideline •WHAT are the consequences? – Brainstorming (preferably round-table discussion). Both negative and positive consequences should be analyzed. – Prioritization (simple) – Hints concerning affected areas for WHAT: cost (internal/external), internal efficiency, customer satisfaction, employee motivation, product quality
  • 56.
    RCA Process guideline,cont. • WHY did the fault occur? – Brainstorming (quiet – write on stickers). Limit the number to 7 notes. – Walk-through, the participants shall, one at a time, read their notes. Perform the grouping by letting each participant “suggests” the potential belonging for each issue – Ask the question WHY for every cause in order to identify the “real” root cause – If necessary: grouping (e.g. divide into teams who perform the grouping)
  • 57.
    RCA Process guideline,cont. • WHY did the fault occur? – Estimation of the effect and realization difficulties, if this cause is solved (use the ”four-fielder” – in Swedish: fyrfältare) – Prioritization, everyone have e.g., 7 ”dots” – Examples of potential groups: Poor documentation, processes, competence, management, rules, communication, handovers, Milestone/TG passages, etc.
  • 58.
    RCA Process guideline,cont. • Options for brainstorming: • WHERE (was (should) the fault (have been) detected? • WHEN (was (should) the fault (have been) detected?) • Define Actions – The goal is to minimize the chance of having the same problem again – Choose a suitable number of “Why” (use the prioritized list) – Divide into groups and let each group select one or two “Why” – Each group shall propose Let each group present their proposal, adjust if necessary Action Time plan Responsible
  • 59.
    RCA Process guideline,cont. • Who will document and distribute the result from the workshop? • Who is responsible for tracking the actions? • Budget/payment?
  • 60.
    RCA workshop agenda 1.Introduction 2. Failure/problem definition consensus 3. Course of events 4. Consequence analysis 5. Cause analysis 6. Cause prioritization/grading 7. Identification of response actions 8. Conclusion/summary
  • 61.
    RCA brainstorming guidelines •Be specific, try to avoid being too general, example: “insufficient testing” is (perhaps) too general. • One sticker==one issue • Initials in order to identify who wrote it • Write legible, short sentences if possible
  • 62.
    Audit process To evaluate: –Software elements – The processes for producing them – Projects – Entire quality programs – To check the compliance to ISO9001, TickITplus, CMMI – To find elements for improvement – To give an independent view of the status – Are you following the processes defined? – Can you find requested information
  • 63.
    When to Audit •A project milestone, calendar date, or other criterion has been (or is to be) met • The audit is initiated by earlier plans • External parties, stakeholders, require an audit • A local organizational element has requested the audit
  • 64.
    Audit plans • Whichproject processes to examine? • Which software to examine. Sampling? Selection criteria? Sample size? • Reporting requirements. Necessary improvements? Recommendations? Findings? Defects? • Required follow-up activities • Activities, elements, and procedures necessary to meet the scope of the audit
  • 65.
    Audit plan, cont. •Criteria that provide the basis for determining compliance (provided as input) • Audit procedures and checklists • Audit personnel requirements (e.g., number, skills, experience, and responsibilities) • Organizations involved in the audit • Date, time, place, agenda, and intended auditees
  • 66.
    Audits – overviewmeeting • Overview of existing agents (e.g., audit scope, plan, and related contracts) • Overview of production and processes being audited • Overview of the audit process, its objectives, and outputs • Expected contributions of the audited organization to the audit process (the number of people to be interviewed, meeting facilities, etc.) • Specific audit schedule
  • 67.
    Audit preparation • Understandthe organization • Understand the products and processes • Understand the objective audit criteria • Prepare for the audit report • Detail the audit plan
  • 68.
    Audit preparation, cont. •Team orientation and training • Facilities for audit interviews • Materials, documents, and tools required by the audit procedures • The software elements to be audited (e.g., documents, computer files, personnel to be interviewed) • Scheduling interviews
  • 69.
    Audits – criteriafor completion, examples • Each element within the scope has been examined • Response to draft findings have been received • Findings have been presented • A formal report has been written and distributed • A recommendation plan has been presented • Follow-up actions have been identified, planned, executed, followed-up and completed. • Verification of effectiveness of taken actions
  • 70.
    The Audit shallnot be perceived as … • An eager attempt to find a lot of deficiencies • An occasion for the auditor to show off • Forced by upper management (or quality department) • An activity which takes up unnecessary time for the participants • A necessary evil that must be done • Something you can forget about after it has been done • The issues found are too heavy to solve
  • 71.
    Audits – Commonproblems • Issues are not taken care of/solved • Issues are not linked to business goals à The focus is NOT on “how well”, but on “out of control” • Goals and objectives are neither defined nor followed-up • Only project managers are audited. ”Be where the action is” • The follow-up is too weak • Upper management have poor focus/commitment • Project Management, PM, has poor focus • The priority on solving issues are low • Solving the issues seems to be a burden for the organization • Work pressure results in low priority for preventive actions
  • 72.
    Audits – Itemratings RED The issue is not under control. Several lacks/deficiencies have been identified. YELLOW The issue is partly under control. A few lacks/deficiencies have been identified. GREEN The issue is under control. No major/severe deficiencies could be identified. BLACK Not applicable, or the issue was not handled/discussed during the audit.
  • 73.
    Experiences from IndustrialAudits • Mostly under control: – Progress reporting – CM handling – Document handling – Risk analysis – Test environment – Change request handling – Sponsoring – Product handling – Service/Support handling – Quite often – continuous improvement, the heart of ISO9001
  • 74.
    Experiences from IndustryAudits • Needs improvement: • Using defined and tailored processes • Reviews – planning and execution • Communication • Follow-up of risks – risk management • Statement of compliance – “we know we can not fulfil the requirements but have not negotiated it” • Backwards compatibility • Hardware requirements and availability of hardware • Traceability • Goals and Measurements • Management commitment
  • 75.
    More Defect PreventionMethods? Or just follow existing processes? • Training of Personnel • Pair Programming • Tools • Test Driven Development (TDD) • Short sprints • Structured Design (Object Oriented) • Requirements tracking • Demos • Pilots • 7 management tools • WBS • Estimating methods • Frequent feedback meetings • Etc.