Applying Continuous Verification and Validation
to achieve the right Quality in Systems delivery
Business
Risk
Technical
R...
Software and Systems Engineering | Rational

The V is dead! Long live the new V with continuous Verification
and Validatio...
Software and Systems Engineering | Rational

Bottom Line is…
 While delivering Quality is already a major challenge, the ...
Software and Systems Engineering | Rational

Agenda




4

© 2012 IBM Corporation
Software and Systems Engineering | Rational

In the News… Business Implications of a Failed missile test
Could result in m...
Software and Systems Engineering | Rational

Business Implications of Recalls
Not only recalls are very expensive, but the...
Software and Systems Engineering | Rational

So… What is Quality?
Exceeding end users needs, even when they keep changing....
Software and Systems Engineering | Rational

Agenda







8

© 2012 IBM Corporation
Software and Systems Engineering | Rational

Continuous Verification & Validation Process

End users needs, at least initi...
Software and Systems Engineering | Rational

Different perspectives
What all the stakeholders need

What the customer need...
Software and Systems Engineering | Rational

Verification vs. Validation Examples

While Verification is “Internal”, Valid...
Software and Systems Engineering | Rational

Verification vs. Validation

While Verification is “internal”, Validation is ...
Software and Systems Engineering | Rational

Verification vs. Validation
While Verification detects defects, Validation is...
Software and Systems Engineering | Rational

Verification vs. Validation
While Verification reduces Technical risk, Valida...
Software and Systems Engineering | Rational

Continuous Verification and Validation (V&V)
Compressing time to customers fe...
Software and Systems Engineering | Rational

Continuous Verification and Validation (V&V)
Compressing time to customers fe...
Software and Systems Engineering | Rational

Agenda







19

© 2012 IBM Corporation
Software and Systems Engineering | Rational

Collaboration through Reviews
Testing for early feedback from Internal and Ex...
Software and Systems Engineering | Rational

Reviews are collaboration points
Include the voice of the customer for on-goi...
Software and Systems Engineering | Rational

Reviews Impact
Reviews are ~2x more efficient in removing defects than Testin...
Software and Systems Engineering | Rational

Reviews Impact
Reviews are ~2x more efficient in removing defects than Testin...
Software and Systems Engineering | Rational

End Users Operational Scenarios
Use them as top level test cases to continuou...
Software and Systems Engineering | Rational

To leverage System Level Test Cases, you need…
Use Operational scenarios as T...
Software and Systems Engineering | Rational

Example User Scenario
Boat
loaded

Boat lifted
Boat on car

Ready to
sail

Bo...
Software and Systems Engineering | Rational

Requirements from Scenarios
Goal hierarchy

Boat
loaded

Boat lifted

traceab...
Software and Systems Engineering | Rational

Virtual Models

Enabling early Verification and Validation… before building a...
Software and Systems Engineering | Rational

Modeling has low potential of introducing defects
Models are best in removing...
Software and Systems Engineering | Rational

Effective and Efficient Testing
Single source of truth for all Quality relate...
Software and Systems Engineering | Rational

Overall Impact on Testing Effectiveness
Collaborative Test Planning, Traceabi...
Software and Systems Engineering | Rational

Agenda








32

© 2012 IBM Corporation
Software and Systems Engineering | Rational

Key Take-Aways
 Verification in Inbound; Validation is Outbound. Both needed...
Software and Systems Engineering | Rational

Continuous V&V – Reducing Technical and Business Risks
 Quality can be deliv...
Software and Systems Engineering | Rational

36
36

© 2012 IBM Corporation
Software and Systems Engineering | Rational

www.ibm.com/software/rational

© Copyright IBM Corporation 2013. All rights r...
Upcoming SlideShare
Loading in...5
×

Kerim Cakmak, Moshe Cohen -- Continuous Verification and Validation

3,685

Published on

Talk of Kerim Cakmak (IBM, Rational Systems Tiger, CEE) "Applying Continuous Verification and Validation to achieve the right Quality in Systems delivery" at 87th INCOSE Russian Chapter meeting, 12 Februaty 2014.

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,685
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
33
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide
  • Reality is that the definition of Quality is changing… low tolerance for inadequate quality, and even Tesla is now delivering updates over the air.To deliver quality, we need to take a holistic approach to Quality, where we continuously verify and validate our designs, throughout the whole development life cycle, and not just at the right side of the V, or even worse – at the top right side of the V.I will show you how to do it, so that you will be more predictable in your delivery schedule, you’ll deliver the right system and at a lower cost of quality. I must here that when I talk about Defect, I mean it in the ISO 9000 sense, which is non-fulfillment of intended usage requirements. It is being used in the general sense, covering product defect, design defect, manufacturing defect, an error, a fault or a failure.
  • Let’s look at the agenda.By now, you already know that you need to take a holistic approach to Quality. We will take a quick look at some recent examples of inadequate quality, talk about Cont V&V… then key capabilities that are needed to deploy it successfully, and then we will take a closer look at how the Rational Systems and Software Eng solution supports Cont V&V.Let’s look at some examples.
  • Bad quality can turn into penalties. In the government determines that the test failed due to quality issues at Raytheon, that could result in major financial consequences to Raytheon. Wall street is watching.Reminds me of a meeting with another contractor, a missile test failed and they were supposed to pay heavy penalties. Like here.The contractor was able to demonstrate full traceability from Requirements to Test plans to Test Cases, very detailed. Therefore they could analyze all their tests, back to Requirements, only to find that the DoD executed a test that was outside the tolerances that were specified in the Requirements. And indeed – the Test failed – as it should have!In this case, Traceability, and being able to demonstrate it, saved this contractor a lot of money, including not being at risk of loosing future projects awards.We never expected this to be a value prop, but to this contractor, this was a major one!
  • Bad quality hurts your stock valuation. Like in the serial Sony recalls. The Prius gas pedal costs Toyota $3B, only a third was for repair cost, 2/3rds were business related.And we start to see different corporate behaviors… GM is trying to pass on cost of recalls to their suppliers. Remains to be seen how suppliers will respond…
  • So, what’s a good quality?While there are many definitions to Quality, the common aspects to most of them is meeting or exceeding customers and end users needs and expectations. Look at the mp3 players… all met the specs… non is around today….And end users keep changing!
  • Let’s look at the agenda.By now, you already know that you need to take a holistic approach to Quality. We will take a quick look at some recent examples of inadequate quality, talk about Cont V&V… then key capabilities that are needed to deploy it successfully, and then we will take a closer look at how the Rational Systems and Software Eng solution supports Cont V&V.Let’s look at some examples.
  • Process quality assurance (automotive spice or CMMI) Processes are written according to some process framework. Does our product development lifecycle complies with these processes Cycling computer. Desinged for cyclers. Customer: Garmin, focused specifically on distance, pace, time. What about people doing cycling for loosing weight? Calories?
  • Types of Verification: Analysis,inspection, review, test, demo, COC (Certification of Conformance, Compliance)
  • Testing code that will eventually reside in a black box without the actual hardware is called Software-in-the-Loop (SIL). Installing software in an actual device and testing that device in a virtual environment is called Hardware-in-the-Loop (HIL). If human interaction is desired or required, the test also includes a Man-in-the-Loop (MIL) component.
  • Let’s look at the agenda.By now, you already know that you need to take a holistic approach to Quality. We will take a quick look at some recent examples of inadequate quality, talk about Cont V&V… then key capabilities that are needed to deploy it successfully, and then we will take a closer look at how the Rational Systems and Software Eng solution supports Cont V&V.Let’s look at some examples.
  • Food and Drug Administration
  • Let’s look at the agenda.By now, you already know that you need to take a holistic approach to Quality. We will take a quick look at some recent examples of inadequate quality, talk about Cont V&V… then key capabilities that are needed to deploy it successfully, and then we will take a closer look at how the Rational Systems and Software Eng solution supports Cont V&V.Let’s look at some examples.
  • End users needs, at least initially, are never precise or complete, therefore we need to help the end user to flash them out. Like if you were to build a new home, would you know up front the exact size of the home and how many rooms you need? No… but the architect will help you figure this out. Same here.As you go down the V, you will continuously do both Verification and Validation. Verification is where you verify that you are building the system right. Like the builder checking with the architect, or the city doing inspections that you build to code.But you will also do continuous Validation…. Not wait to the end, like the builder and/or the architect coming back to you to check with you as they make more decisions on your dream home.
  • Kerim Cakmak, Moshe Cohen -- Continuous Verification and Validation

    1. 1. Applying Continuous Verification and Validation to achieve the right Quality in Systems delivery Business Risk Technical Risk Kerim Cakmak, PhD IBM Rational Tiger Team CEE Moshe S. Cohen QM Market Manager and Evangelist © Copyright IBM Corporation 2013
    2. 2. Software and Systems Engineering | Rational The V is dead! Long live the new V with continuous Verification and Validation across the full V! Abstract: While we know how to reduce technical risks in Systems delivery through Verification activities, our business risk is on a steep rise. This is largely due to the fact that "Validation" is performed at the end of the development lifecycle, rather than throughout, and it is done against customers requirements rather than end-users needs. In this session we will discuss some new best practices around Verification and Validation that were observed through recent successful and even more unsuccessful projects, and show you how to continuously verify and validate your designs throughout the whole lifecycle and not only at the tail end of it. Not only will this improve the quality of your products and systems, but it will also allow you to converge earlier on the right requirements, compressing the time to end users feedback, and improve your time to market predictability. While it doesn't change the V model, it is a new V. 2 © 2012 IBM Corporation
    3. 3. Software and Systems Engineering | Rational Bottom Line is…  While delivering Quality is already a major challenge, the definition of Quality is changing… – Low tolerance for bad quality (real and perceived) – Expected short cycles with continuous improvements  To deliver Quality, we need a holistic approach to Quality – Continuously verify and validate our designs, throughout the whole lifecycle.  Business results: – Deliver the right system • Compressing time to customer’s feedback – Better time to market predictability • Earlier convergence on the right Requirements – Lower cost of Quality • Defect* Avoidance, Lower defect density, Higher defect removal efficiency * Defect, as defined in ISO 9000 (2008): The non-fulfillment of intended usage requirements. The term Defect will be used here in the general sense, regardless whether it is a product defect, design defect, manufacturing defect, an error, a fault or a failure. 3 © 2012 IBM Corporation
    4. 4. Software and Systems Engineering | Rational Agenda   4 © 2012 IBM Corporation
    5. 5. Software and Systems Engineering | Rational In the News… Business Implications of a Failed missile test Could result in major financial consequences, unless Raytheon succeeds in demonstrating that it is not its fault. 5 © 2012 IBM Corporation
    6. 6. Software and Systems Engineering | Rational Business Implications of Recalls Not only recalls are very expensive, but they also drive corporate behaviors.  Sony recalls: “Sony-related recalls are following one another, and that may ruin the company’s brand image,” said Keita Wakabayashi, an analyst at Mito Securities Co. with a “neutral plus” rating on the stock. Bloomberg, Oct 2011.  Toyota Prius gas pedal recall, 2010: ~$3B (CNNMoney.com) – Repair cost (quality related expenses): $1.12B – Legal cost (class-action lawsuit): $1.1B (WSJ, Dec 27, 2012) – Image cost (sales losses): $770M-$880M – “Toyota said Thursday that a software problem is causing problems with the brakes”… CNNMoney, Chris Isidore, senior writer, Feb 4, 2010.  GM presses suppliers for future recall costs (Automotive News, Aug 5, 2013) – GM has adopted a new purchasing contract that would allow it to recover the cost of safety recalls from its suppliers. – The new GM contract has open-ended implications, stating that the supplier's components "will not, at any time (including after expiration or termination of this contract), pose an unreasonable risk to consumer or vehicle safety." 6 © 2012 IBM Corporation
    7. 7. Software and Systems Engineering | Rational So… What is Quality? Exceeding end users needs, even when they keep changing. No single definition, most definition are very detailed and verbose… but the common theme is most definitions is:  Yesterday: Meeting the Spec.  Today: Meeting or exceeding customers and endusers needs and expectations – “End users needs” ≠ “Initial Requirements” “The test of innovation lies not in its novelty, its scientific content or its cleverness. It lies in success in the marketplace.” Peter Drucker. (Business Management Philosopher) 7 © 2012 IBM Corporation
    8. 8. Software and Systems Engineering | Rational Agenda     8 © 2012 IBM Corporation
    9. 9. Software and Systems Engineering | Rational Continuous Verification & Validation Process End users needs, at least initially, are imprecise and incomplete – This is Normal! Initial Elicitation of end-users needs Starts with early elicitation of end-users needs  Helping the customer/end user to flash out their needs (Requirements Elicitation) – Imagine you want to build a new home, do you know the exact size of the house? Notice that this results in Initial understanding of the needs, still partial and incomplete Continuously Verify… Continuously Validate… Continuously verify, throughout Development, that you are building the system right. • • As the builder checking with the architect. As the city inspections, making sure the house is being built in compliance with regulations. Continuously validate with the customer, as the design is getting more concrete, that you are building the right system. • As an architect and/or the builder validating assumptions they have made as your house is being built. Continuously improving understanding of customer/end-users needs Delivery © 2012 IBM Corporation
    10. 10. Software and Systems Engineering | Rational Different perspectives What all the stakeholders need What the customer needs What the customer thinks they need What the customer says they want Given this How do we get to this? © 2012 IBM Corporation
    11. 11. Software and Systems Engineering | Rational Verification vs. Validation Examples While Verification is “Internal”, Validation is “External”  Verification: The evaluation of whether or not a product, service or system complies with a regulation requirement, specification, or imposed condition. It is often an internal process. (IEEE) • Code should never run thru a divide by zero as it may cause a crash or unexpected results. • Did you implement the design pattern properly? • Does this block diagram represent the best architecture for the system?  Validation: The assurance that a product, service, or system meets the needs of the customer and other identified stakeholders. It often involves acceptance and suitability with external customers. (IEEE) • On a cycling computer… The customer asked for certain data fields to be displayed… but wouldn't the end user want to customize them? • Is this {…} the right sequence of voice commands for making a phone call while driving your car? • What's the right behavior of the wifi enabled home dialysis machine when the connection is intermittent? 11 © 2012 IBM Corporation
    12. 12. Software and Systems Engineering | Rational Verification vs. Validation While Verification is “internal”, Validation is “External”  Verification: Building the system right – Testing against requirements • “Internal” requirements – Utilizes Test, Simulations, Analysis, Inspections and Demonstrations Customers Needs – Assuring robust design and implementation Requirements Analysis System Testing – Compliance with industry regulations – We “know” how to do that… often done well!  Validation: Building the right system Systems Analysis Systems Design Integration Testing – Validating against your customers needs • As expected, imprecise and incomplete • Often negotiated with several teams • Often negotiated with people who may not be the end users – Do we know how to do that? Do we do it? 12 Detailed Design & Implementation Note that the V does not represent a development process but rather a flow of information as we decompose and recompose a system. © 2012 IBM Corporation
    13. 13. Software and Systems Engineering | Rational Verification vs. Validation While Verification detects defects, Validation is about avoiding them. Left side of the V Verification • Early defect removal Right side of the V • Expensive defect removal Customers Needs System Testing Requirements Analysis Validation 13 • Defect Avoidance • Early detection of potentially business or mission failures • Early Acceptance tests • Final Acceptance test Systems Analysis Systems Design Integration Testing Detailed Design & Implementation © 2012 IBM Corporation
    14. 14. Software and Systems Engineering | Rational Verification vs. Validation While Verification reduces Technical risk, Validation reduces Business or Mission risk  Verification: Reduces Technical risks – It is a Testing process… Business Risk – We “know” how to do that… often done well!  Validation: Reduces Business risks Technical Risk – Focuses on closing the loop with the end-user as often as possible • Can’t just blame the Requirements being wrong… • “Meeting the spec” is not a guarantee for business or mission success… – It is NOT a Testing process… but rather an on-going Discovery process • Important to “developers” • Critical to End Users – Helping users figuring out their own Requirements! – Do we know how to do that? Do we do it? 14 © 2012 IBM Corporation
    15. 15. Software and Systems Engineering | Rational Continuous Verification and Validation (V&V) Compressing time to customers feedback! Very Common: Customers Requirements Late Validation (Re-Evaluate) Long/Costly Process System Testing Requirements Analysis System Testing Requirements Analysis Systems Analysis Systems Design Integration Testing Systems Analysis Systems Design Requirements Analysis Integration Testing System Testing Systems Analysis Systems Design Integration Testing Detailed Design & Implementation Detailed Design & Implementation Detailed Design & Implementation Example:  AA and United approached Boeing for a new flight entertainment system  Job has been sub’ed to Sony Trans Com., who designed & delivered a state-of-the-art system.  End result: – Premium paying passengers had trouble operating the system – Flight attendants had trouble supporting the passengers  End-End result: Flight attendants shutting down the system before take off, claiming malfunction. © 2012 IBM Corporation
    16. 16. Software and Systems Engineering | Rational Continuous Verification and Validation (V&V) Compressing time to customers feedback! Late Validation (Re-Evaluate) Very Common: Long/Costly Process Customers Requirements System Testing Requirements Analysis System Testing Requirements Analysis Systems Analysis Systems Design Integration Testing Systems Analysis Systems Design Integration Testing Requirements Analysis System Testing Systems Analysis Systems Design Integration Testing Detailed Design & Implementation Detailed Design & Implementation Detailed Design & Implementation Continuous V&V per Feature: Use Scenarios? Validation Operational Scenarios? Validation Prototypes? Validation HIL/SIL? Validation Final? Validation  Customers Needs Verification 16 Verification Verification Verification Verification © 2012 IBM Corporation
    17. 17. Software and Systems Engineering | Rational Agenda      19 © 2012 IBM Corporation
    18. 18. Software and Systems Engineering | Rational Collaboration through Reviews Testing for early feedback from Internal and External stakeholders  Review process where peers and other stakeholders collaborate in order to identify potential errors and deviations from the spec.  Objectives include: – Detect wrong understanding, bad assumptions and defects close to when introduced (Functional review) – Identify potential improvements and/or risks (Functional review) – Verify compliance with regulations and conformance to standards (Quality review)  Audience: – External Reviews: With customers and/or end-users, often focused on Validation – Internal Reviews: With internal stakeholders (peers, management), often focused on Verification  Formal inspections (with defined roles & process) to ad-hoc team reviews and walkthroughs.  FDA Example: On Going Reviews  / Late Validation  20 © 2012 IBM Corporation
    19. 19. Software and Systems Engineering | Rational Reviews are collaboration points Include the voice of the customer for on-going validation  Applies to the full development life cycle and to all artifacts  Reviews are short, and per feature (best practice: focus on end user feature)  Examples: Review Internal/External Requirements elicitation Identifying Use Scenarios External Review What are the main functions/activities? Requirements Review Defining Operational Scenarios External Review How does the system operate? Architecture Design Review Is this the “best” architecture? Internal Review Design documents Initial Prototype Review Early feedback based on feature execution External review Low Fidelity prototype, early feedback. Advanced Prototype Review Production code in HIL/SIL environment External Review Hi Fidelity prototype, still early feedback. Test Plan Review Agreement across all stakeholders Internal and/or External Review Sunny & Rainy days scenarios Test Execution Review 21 Objective Discuss Quality status and next actions Internal and/or External Review Focused on high level operational scenarios © 2012 IBM Corporation
    20. 20. Software and Systems Engineering | Rational Reviews Impact Reviews are ~2x more efficient in removing defects than Testing (Not including Defect Avoidance) Reviews Testing Source: Capers Jones, 2012 22 © 2012 IBM Corporation
    21. 21. Software and Systems Engineering | Rational Reviews Impact Reviews are ~2x more efficient in removing defects than Testing (Not including Defect Avoidance) Reviews Testing 23 © 2012 IBM Corporation
    22. 22. Software and Systems Engineering | Rational End Users Operational Scenarios Use them as top level test cases to continuously drive all V&V activities  Operational Scenarios – Quasi formal but yet intuitive – Can be extended with sketches and Interaction Diagrams  Operational Scenarios are a Testable version of the requirements! – Can be converted into System level test cases  System level test cases: – Should be Validated (reviewed and approved) with the customer and/or end-user – To be used as a Golden Reference model throughout design and implementation for Verification purposes. Although diagrams shown here are SysML diagrams, concept applies to any system. 24 © 2012 IBM Corporation
    23. 23. Software and Systems Engineering | Rational To leverage System Level Test Cases, you need… Use Operational scenarios as Tests throughout the whole development Lifecycle  Validate your understanding of the needs by reviewing the expected scenarios with the stakeholders and getting them approved.  A Modeling environment that support Use Case analysis  A Quality Management process that support the conversion of Operational scenarios into test cases – Test cases for either Manual Testing or Automated Testing – May require internal review and approval  Traceability: Needs  Requirements  Operational Scenarios  Test Cases Operational Scenarios System Level Test Cases Real Prototype 25 Virtual © 2012 IBM Corporation
    24. 24. Software and Systems Engineering | Rational Example User Scenario Boat loaded Boat lifted Boat on car Ready to sail Boat unloaded sequential Boat rigged parallel To have sailed and survived Mast rigged Center-plate rigged Rudder rigged Gybed Subgoals Sailed normally Tacked alternate Overal l goal Sailed periodic Returned home Cruised Maneuvered alternate Ashore Recover from Capsize Righted exception Coast guard contacted © 2012 IBM Corporation
    25. 25. Software and Systems Engineering | Rational Requirements from Scenarios Goal hierarchy Boat loaded Boat lifted traceability Two people shall be able to lift the boat onto the roof of the average saloon car. Boat on car Ready to sail Boat unloaded sequential Boat rigged parallel To have sailed and survived Stakeholder requirements Mast rigged Center-plate rigged The sailor shall be able to perform a gybe. Rudder rigged Gybed Subgoals Sailed normally Tacked alternate Cruised Maneuvered Sailed Overall goal periodic Returned home alternate Ashore Recover from Capsize exception Righted Coast guard contacted The sailor shall be able to contact the coastguard when the boat is capsized. © 2012 IBM Corporation
    26. 26. Software and Systems Engineering | Rational Virtual Models Enabling early Verification and Validation… before building anything real.  Virtual models that can be used for early analysis and trade studies (functionality, behavior, architecture, structure, performance, reliability, safety, etc.) – Models abstract mechanics, electronics and/or software entities, operating independently or integrated – Different domains: • Mechanical/Control models (such as Mathworks, NI) • Graphical languages (such as SysML) • Textual language (such as SystemC for HW design)  Can be used for – The system being designed AND its TestBench (Plant models)  Use System level test cases to drive analysis – Execution, Simulation or Prototyping  Various configurations driving on going system analysis: • Eg. 1: Virtual sensors driving virtual HW+SW models • Eg. 2: Mechanical prototype driving virtual HW+SW models • Eg 3.: Mechanical prototype driving virtual HW+production SW 28 © 2012 IBM Corporation
    27. 27. Software and Systems Engineering | Rational Modeling has low potential of introducing defects Models are best in removing defects  Models are Executable – You can ask Questions and get consistent answers!  Models can be Analyzed Model Based: Low Defect Potential, Best Removal Efficiency – Tradeoff studies – Performance – Behaviors – Etc.  Leverage correct-by-construction Synthesis capabilities to guide the design and implementation process Source: Capers Jones, 2012 – Eg. 3D printers, C/C++ for SW, SystemC for HW – Enabling Co-Execution for V&V purposes 29 © 2012 IBM Corporation
    28. 28. Software and Systems Engineering | Rational Effective and Efficient Testing Single source of truth for all Quality related artifacts and processes  Central Quality Management Hub, acting as a single source of truth, enabling: – Collaborative Test Planning • Development and approval for Verification and Validation test plans – As well as specific plans as needed, such as Safety. • Traceability back to Requirements (why we need which test?) – Managing test execution (manual and automated) according to Test Plan • Who is running which test, when, why, pre-conditions, test lab/equipment reservation etc. – Integrated Defect Management • What went wrong, how to reproduce, defect resolution, reporting, etc. – Traceability across the full lifecycle • User Needs  Requirements  Test cases  Test Results  Defects  Resolutions  Other key capabilities – Collaboration around Rainy days scenarios – Test assets ReUsability – Reconciliation process between Requirements and Test Plans/Test Cases 30 – Actionable Analytics to drive process improvements © 2012 IBM Corporation
    29. 29. Software and Systems Engineering | Rational Overall Impact on Testing Effectiveness Collaborative Test Planning, Traceability, Test Execution and Analytics Impact of Quality Management on Process Efficiency 100% 85% 90% 76% 80% 70% 58% 60% 85% 87% Detect 3, release 7 75% 60% Detect 3, release 2 50% 40% 32% 30% 30% 20% 15% 10% 0% 1 QM Impact: 20% 2 3 40% 40% CMMI Levels 4 5 40% 10% W/O QM W QM W/O QM: Percentage of defects detected during Requirements review, design reviews, unit testing and Functional testing – Current practice W QM: Percentage of defects detected thru Requirements review, design reviews, unit testing and Functional testing – Improved practice 31 Source: Analysis of over 850 GBS projects and Industry data © 2012 IBM Corporation
    30. 30. Software and Systems Engineering | Rational Agenda       32 © 2012 IBM Corporation
    31. 31. Software and Systems Engineering | Rational Key Take-Aways  Verification in Inbound; Validation is Outbound. Both needed.  Verification is a Testing process. Validation is a Discovery process, helping the customer refine their understanding of their own needs.  Compressing time to feedback allows for change… and therefore reduces surprises.  Reduce business risk with Early and Frequent feedback from all stakeholders – Use Scenarios and Operational Scenarios – Validation Test Plan – Low Fidelity and Hi Fidelity prototypes  Operational scenarios = Testable Requirements = Golden reference – Use them for your Testing throughout the whole development process and not only at the end! Question:  Unit Testing and Integration Testing are both very important.  Which is more important?  Which one would you do first? 34 © 2012 IBM Corporation
    32. 32. Software and Systems Engineering | Rational Continuous V&V – Reducing Technical and Business Risks  Quality can be delivered by compressing the time to customer feedback with continuous Verification and Validation.  Both Technical and Business risks can be lowered  Business Implications: – Lower cost of Quality Business Risk Technical Risk • Defect Avoidance, Lower defect density, higher defect removal efficiency – Deliver the right system • Compressing time to customer feedback – Better time to market predictability • Earlier convergence on the right Requirements 35 © 2012 IBM Corporation
    33. 33. Software and Systems Engineering | Rational 36 36 © 2012 IBM Corporation
    34. 34. Software and Systems Engineering | Rational www.ibm.com/software/rational © Copyright IBM Corporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others. 37 © 2012 IBM Corporation
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×