Collaborative Quality Management


Published on

An introduction to managing quality

Published in: Business, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • The story behind the Rain Sensing Wiper The development of the first Rain Sensing Wiper illustrates how a classic failure to fully conceptualize a product's physical architecture resulted in the discovery of integration issues at servicing time, thus quickly leading to engineering change requests. The scenario revolves around the initial introduction of the Rain Sensing Wiper (RSW) feature in an automobile manufacturer's vehicle program. Before examining the reason of the failure of the RSW, let us briefly review the characteristics of the system. The RSW contains mechanical (optical mounting device), electronics (IR sensors and ECU), and software (computer vision algorithm) components that are procured by tier-one suppliers. These components are simply integrated by the manufacturer. The main parameters of the system are: (1) the optical and geometric specifications of the windshield, in particular its thickness and glass optical indexes, and (2) the ranges of operation of the electronic optical sensors. The detection software also has normal ranges of operation relative to these parameters, but in addition relies on data about the actual values of the parameters of the windshield. The fact that the RSW electronics and software specifications include ranges for the relevant windshield properties is important, because it allows more flexibility on the choice of the windshield itself. This is a critical design choice, the procurement and integration costs associated with the windshield being an order of magnitude greater than that of the RSW. For optimal performance, the actual values of the windshield properties should fall near the center of the normal operating ranges both for the RSW sensors and the software; however, acceptable operation should be guaranteed for the whole range. From a procurement standpoint, the windshield is simultaneously purchased from different suppliers. Depending on the year of production and where the product is manufactured, suppliers may modify the design of their windshields. Also, one or more changes of suppliers can occur during the production phase. In the failure scenario, which occurred during the first year of the RSW's introduction, a local windshield supplier provided a component whose characteristics were incompatible with the operating range of the sensor. Unfortunately, no requirement for calibrating the system properly (i.e., for verifying that sensor and windshield are compatible) had been captured for the RSW at that point. Thus cars were sent to customers with a non-functioning wiper system. Initial diagnostics designated the software as the culprit for the malfunction, since it was difficult for mechanics to test its behavior. The other components (ECU, sensor, and windshield) were functioning normally when tested independently. The failure mode for the RSW resided at the level of its sub-systems, which made it difficult for the manufacturer to discover it. After discovering the root cause, a new requirement was captured to ensure that new systems will be properly calibrated at the production stage.
  • IBM INTERNAL USE ONLY And the cost of failure is enormous. Here’s one example. People rely on the investment firm Moody’s for accurate assessment of securities. A bug in a software system caused Moody’s to incorrectly assess a safe credit rating to $4 Billion of risky securities. When this one little mistake was revealed, Moody’s lost $1.7 B USD in market value in a single day. This incident illustrates how business-critical software systems are today. One bug can literally mean life or death for the organization.
  • IBM INTERNAL USE ONLY There have been many notable cases of system outages a scalability issues that have made front page as mission-critical applications have moved beyond the closed doors of the IT shop. The impact is not limited to immediate revenue, but can extend to long term customer retention. Today’s interconnected and distributed applications carry a much larger risk of housing a significant performance problem that can go undetected until deployment into production. Performance Testing is a critical part of the QA strategy to identify and resolve these risks before they become a problem to your users.
  • In order to achieve business value and innovation, software must be managed like any other business asset – as an integrated set of repeatable activities focused on producing a desired business outcome… that is to say, as a business process . A process whereby key stakeholders… >Collaborate to achieve common goals by optimizing how people, across teams, organizations, companies and contents work >Automate to increase control and efficiency through integrated workflows that reduce manual tasks and human labor >And, report and analyze to continuously improve what is being done by measuring progress in real-time
  • OSLC is a key differentiator for us – Many of our existing products provide and continue to enhance their jazz story by supporting the OSLC specification – by doing so they become a productive member of a customers ALM solution. OSLC allows us to support a mix of open source technologies, other market leading offerings, business partners, home grown integrations, etc. The Jazz Team Server is our #1 implementation of OSLC providing an integration hub our customers can use. Most vendors point integrate everything with brittle interfaces. Our approach creates rich, consistent interfaces in ALM to improve the entire development lifecycle and protect existing, future investments.
  • >Higher quality software More thoroughly tested requirements, even as requirements change Loose coupling between the requirements and test cases reduces reliance on human communication to maintain the interlock >Software that better meets requirements Greater visibility of requirements detail by developers and architects Linkage created between requirements and the work items exposes more information immediately to business analysts, developers, architects >Shorter time to delivery Data association between testing and development Association created between test cases and defects enable improved, timely identification, resolution, and verification of defects >Lab Manager deployment triggered by build notification    When the change sets for a defect / workitem are delivered in a build, RQM will automatically receive notification from the build system. Provide the ability to configure automatic execution of lab management tasks based on the notification. This function will typically be used to deploy the application under test to one or more test environment. >RQM/ RTLM Execution Flow    RQM/ RTLM will provide a more optimized user experience as part of the test execution flow from TER to finding, reserving, settting up machine, to executing tests. As part of this solution, the reservation system will be enhanced to support blocking reservations to better handle reserving machines for test execution. >Common C/ALM UI Elements Common Banner Support Common Login Support Rich UI Hover Support >Linkage of RTC WorkItems to Test Plans/Test Cases linking between Test Plans/Test Cases to RTC workitems (plan items, stories, etc.) for the purpose of establishing the relationship to the code implementation of what the test case will test. >Support for BVT A user can configure a sequence of steps to run whenever a build is successfully completed. These steps include optional deployment/provisioning of the build, a Test Suite to verify the build, and an optional “clean-up” step to teardown the installed build. The user preconfigures all the data (parameters, machine targets) for the BVT steps. Prepopulation of Defect form when creating a defect    The default work item process template has been updated to include these custom fields: related requirement name, related test case, test environments, build id, verification point failure. These fields appears in the detail section of the defect form. >RQM/RRC Integration Integration provided in both RQM and RRC >Cross Server Work Item Integration with RTC The capability must continue to support the same reporting capabilities that exist in RQM today. Reports that relate test status to defect status must integrate the information from the various servers >Automatic Build Notification from RTC Build System When the change sets for a defect / workitem are delivered in a build, RQM will automatically receive notification from the RTC build system and change the status of the appropriate Test Execution Records to indicate that the tests are clear to run.
  • To give the big picture... This slide captures quality management end-to-end capabilities and those of the Jazz platform and the best practices that can be customized and be flexibly integrated into a process that is based on client needs… Test management, test lab management and a variety of domain-specific test automation capabilities supported by Jazz’s open platform helps extend our quality management offerings...
  • IBM INTERNAL USE ONLY > Rational addresses these challenges with Rational Quality Manager. Rational Quality Manager is a centralized test management system for business-driven quality software that addresses business risk, operational efficiency, and the ability to make confident release decisions. > RQM helps to mitigate business risk through: Distributed project team collaboration on a dynamic, customizable test plan over a Web 2.0 interface Process enforcement through user-defined roles/customized process workflows and work item assignments >Operational efficiency is accomplished through automation. For example: Reducing testing time through capabilities to help users track, schedule and execute tests on physical and virtual lab assets Automated assistance to optimize test environment coverage Accelerating manual testing processes so testers can do more in less time >Confident release decisions are supported by: Process monitoring with version histories and trend analysis Proactive risk management with automated, filtered and prioritize reporting Customizable dashboards that map pattern analysis to key productivity indicators for greater predictability > All this delivered on an open, extensible platform to interface with Rational and 3 rd party testing tools and add-ons
  • Now that we are delivering on our vision, we are gaining the accolades of analyst firms, like Gartner, who track and evaluate vendors' ability to deliver on integrated ALM solution. And as you can see here, IBM is the only vendor -- of twenty vendors evaluated -- who was rated as Strong Positive, because of its current market strengths, breadth of portfolio, and the solid architectural foundation of the Jazz platform. Strong Positive Is viewed as a provider of strategic products, services or solutions: Customers: Continue with planned investments. Potential customers: Consider this vendor a strong choice for strategic investments.
  • A single quality contract for the entire software delivery team represents the ‘glue’ <sts> that formalizes the business’ goals/objectives and entrance/exit criteria, defines the different team roles and responsibilities, and guides team members’ tasks, handoffs/review/approvals/and signoffs… one cohesive plan to get to end of job that all stakeholders and team members are focused on and collaboratively working toward completing. A plan is not a prescription, it is an evolving, moving target
  • Now, before we go any further, let me provide a little context and clarification around the term “Test Plan”. Most people when they hear the term think of 1 of 2 things. Either they visualize traditional word docs – where the plan is documented (but not the actual tests being run), or they visualize some sort of tree view that captures test cases (which ignores the business objectives and is effectively just a task list). In Rational Quality Manager – the test plan is a hybrid of these two types. It documents the traditional business objectives you’d capture in a word doc, as well as the dynamic content – like requirements and test cases you’d capture in a tree view. This provides the best of both worlds, capturing everything that matters to test teams.
  • The biggest hidden QM cost is associated with defect duplications. If we are able to control and minimize the number of duplicate defects that are captured we have an opportunity to not only drive down costs but also reduce the time team members are performing duplicate work. Duplicate Defect Identification capability within IBM Rational Quality Manager, scans existing defects associated with a variety of related artifacts and presents a list of suspect duplicates to to minimize duplicate team efforts.
  • Too often we still find projects testing products or systems as built, instead of verifying that those products and systems actually meet the goals laid out for them in the requirements. To achieve this, tests must be based on requirements – so furthering our approach to delivering the right capabilities to the right users in the right tools
  • >Plan Tests Early< Plan tests for each requirement as the requirement is written. Considering testing at the same time as capturing requirements improves the way the requirement is expressed by encouraging consideration of how the requirement is quantified. For every requirement, the question should be asked: How will we know if this requirement will be, or has been, satisfied? Note the use of both future and past tenses in this question. Early tests, such as design inspections, check that, when the product is built according to the design, it will meet requirements. Later tests check that requirements have been satisfied by what has been built. The above question should lead to a qualification strategy for each requirement. Becauseo f the broad range of qualification activities available, every requirement will give rise to multiple tests covering all stages of development. For instance, an early opportunity for qualification is to check the design against a requirement. Are the necessary elements present in the design to ensure that the requirement will be met?In addition, there will be one or more system tests to ensure that what is finally built really does meet the requirement. The set of selected tests form the qualification strategy. The question should also lead to the identification of qualification criteria for every test. These formalize what is considered a successful outcome for each test. Qualification criteria vary with the nature of the requirement and the planned test. >Conduct Tests Early< Perform tests as early as possible in the development process. “ A stitch in time saves nine.” Defects propagate themselves from stage to stage, and the cost of correcting them geometrically increases throughout the development process. Because of this, development cost and schedule can be dramatically reduced by identifying and correcting defects as early as possible, thus minimizing expensive rework in later stages. Examples of early testing are reviews, inspections, and walkthroughs. >Relate Tests to Requirements< Trace tests back to the requirements they are design to check. Tracing is a means of documenting the relationships between artifacts during development. The most common form of tracing is the satisfaction relationship between layers of requirements. However, the qualification relationship between requirements and the tests that show that they have been met is also important to capture. Tracing enables two kinds of analysis to be carried out: Coverage analysis is used to ensure, for example, that every requirement has at least one qualification action planned and eventually executed. It can also be used to ensure that every qualification action has an associated requirement and, therefore, provides benefit. Impact analysis is used to determine the test cases that may need to be revisited when requirements change. Before accepting a change to a requirement, the cost of redefining and re-executing its associated qualification actions is taken into account. Similarly, when a test fails, the impact of the failure can be viewed in terms of the requirements. Impact analysis uses a combination of the satisfaction and qualification relationships to determine potential rework. For instance, if a component test fails, it not only affects the associated component requirements (through the qualification relationship) but also the sub-system requirements that those component requirements are supposed to satisfy (through the satisfaction relationship) and, in turn, the satisfied system and customer requirements. Testers understand that testing at every level is necessary. For instance, a sub-system requirement cannot be fully tested by just testing its components; the sub-system (the integration of those components) still needs to be tested. >Relate Defects to Requirements< Trace defects back to the requirements that they show are not satisfied. When a test reveals unexpected behavior, or that test criteria are not met, defects arise. There are three possible causes of defects: An error in the definition of the test. An error in the expression of the requirement or in its test case. A genuine defect in the product. In every case, a defect is identified by tracing it to the requirements that are shown not to be satisfied. This is a good discipline because a defect is defined as a departure from requirements. Since each test may be traced to multiple requirements, analysis of each defect is needed to determine precisely which requirements are affected. Measure Progress against Requirements Set targets and measure the progress of testing in terms of those requirements that are shown to be satisfied or are not satisfied. One of the most difficult judgments to make during development is deciding when enough testing has been done. Limited resources are available, and the test manager needs to know where to apply effort most effectively. When testing is isolated from requirements, information about the relative importance of different aspects of the system is not available to inform the test manager, making it hard to allocate resources and difficult to assess the impact of failure. With the appropriate traceability in place, the relative importance of each test and the success and failure of tests can be related to the requirements affected at the desired level—customer requirements, system requirements, etc. Targets for completed testing can be set in terms of requirements. The test manager knows where to apply resources, and reports can be generated that show the progress of testing in requirements terms.
  • >Clean look : More space for relevant and most needed information   Better view for graphics - Should be able to compress all statements and see graphic one at a time, or expand all graphics inline or thumbnail etc. More real estate by making keyword view collapsible Every Manual test should have an optional associate VP (also known as expected result).  The concept we currently know as VP should be wrapped into each step. >Lesser keystrokes/Better/Faster navigation Users should be able to traverse through a manual test script using the arrow keys Add Preference for allowing ability to apply verdict/result to ALL step types Manual test execution: quickly jump ahead to a Step and mark it "Failed" Provide a warning message if you add defect associations in testscripts and cancel script without applying(click curve down arrow) to commit the step Manual Test Execution - Apply all faster if we avoid animation of the cursor
  • IBM INTERNAL USE ONLY Rational’s Quality Management solution delivers a comprehensive approach to managing the quality risk in your software development efforts:
  • IBM INTERNAL USE ONLY Rational’s Quality Management solution delivers a comprehensive approach to managing the quality risk in your software development efforts:
  • IBM INTERNAL USE ONLY Test lab Management Find physical and virtual lab assets Schedule test run Auto-provision lab assets Capture analytics for optimization Server requirements and configuration coverage for future tests and production
  • The third and final quality management subtheme focuses on making informed, confident decisions - our QM offerings achieve this with real-time metrics and measures built-in and delivered to you via dashboards and reports so you have full visibility into the quality status of your project Patterns of success: Deliver real time, fact-based reporting How can we track progress against goals? “ Organizations exercising world-class performance management practices enjoy a 2.4 times market returns of typical companies” BusinessWeek Study: The Payoff of Pervasive Performance Management When you think about your own transformation, do you have these types of questions? Are you executing development across state boundaries, countries or perhaps even continents? Do you have culture issues where your culture inherently does not allow you to achieve agile software delivery operations or perhaps an agile business model? Are you executing across a complex software supply chain, across in‑house developed applications, outsourced applications and perhaps packaged applications? Are you utilizing your human resources and talents to the best of your advantage? Are you applying your human talents to solve those most complex business challenges rather than applying them to repetitive tasks that could be automated? I would say that if you have some of these questions, then you probably need to ask yourself, where do I get started with this transformation journey? Sources Top 2/3's and 77% quotes Business Week and Business Objects study: "The Fact Gap: The Disconnect Between Data and Decisions“ The middle quote on "revenue 75% below..." "How companies make good decisions: McKinsey Global Survey“ Speaker Notes This slide + 4 more slides. This slide sets up the 3 main themes for executing with agility to make course corrections as needed to improve project predictability… The 3 slides that follow drill down on the capabilities that make this possible and then this will finish with a summary slide. This subtheme is about This solution offers you three primary capabilities: The first is about The second capability The third capability
  • Cognos – separate install and setup on separate “server”
  • VM Logix Surgient - BSD Group – Bi-directional integration between IBM Rational Quality Manager and HP Mercury Quality Center; Extend RQM context-based collaboration from legacy test assets Device Anywhere - Access to mobile application testing on more than 1,500 real handsets across the globe, while employing RQM to control execution of tests and keep track of results. Users can employ DeviceAnywhere’s test automation layer – available through its DeviceAnywhere Pro solution – to create and automate scripts on handsets, then utilize RQM to control the execution tests, keep track of results, and manage change requests. Weblayers - WebLayers expands the value of their automated SOA governance support for Rational customers with a planned Connector to Rational Quality Manager. Features like automating review / update of test plans based on policies and expanding the RQM Dashboard with Policy metrics, will enable better test quality and productivity across the services design-time lifecycle. Worksoft’s Certify execution tool adapter for RQM enables access to automated testing solutions for functional testing of SAP and other enterprise business applications. Finaris RapidRep execution tool adapter for RQM enables the RapidRep Test Suite RapidRep validates data, such as relational databases, or structured files like CSV or XML.
  • IBM Internal Use Only
  • Better, faster requirements lead to less project rework, faster time to market, and better business outcomes. Effectively engaging a wide range of customers and other stakeholders in the requirements process helps to uncover customer needs and define good requirements. In this Webinar, you will learn how Strongback Consulting and Rational Requirements Composer use A Single Source of Truth to help teams define and use requirements effectively across the project lifecycle. Companies who push quality to the end of the development lifecycle typically experience major project delays, software that is delivered and implemented with serious flaws and are on occasion forced to abandon a project completely. Learn how Strongback and Rational Quality Manager can help you deliver quality throughout the product and application lifecycle – from concept to launch to retirement. This integrated, lifecycle approach helps make quality a shared responsibility – not siloed and disconnected, enabling organizations to achieve greater consistency, efficiency and predictability in their delivered software solutions.
  • Standard closing slide which includes appropriate legal disclaimer to be included in all external presentations. Learn more links: IBM Rational software: Rational launch announcements: Rational Software Delivery Platform: Accelerate change and delivery: Deliver enduring quality: Enable enterprise modernization: Ensure Web site security and compliance: Improve project success: Manage architecture: Manage evolving requirements: Small and midsized business: Targeted solutions: Rational trial downloads: Leading Innovation Web site: developerWorks Rational: IBM Rational TV: IBM Rational Business Partners: IBM Rational Case Studies:
  • Collaborative Quality Management

    1. 1. Greater confidence. Improved business performance. Delivering software quality as a strategic competency IBM Rational Quality Manager
    2. 2. Agenda <ul><ul><li>Challenges </li></ul></ul><ul><li>Executive summary & the Jazz vision </li></ul><ul><li>Rational Quality Management </li></ul><ul><li>Customer results </li></ul>
    3. 3. Rain Sensing Wiper: Example of a System Design Failure Individual Systems Worked, B ut Failed When Integrated <ul><li>Windshield provided by local supplier </li></ul><ul><ul><li>Incompatible with the operation range of the sensor </li></ul></ul><ul><ul><li>No captured requirement for proper system calibration </li></ul></ul><ul><ul><li>(i.e., verifying sensor and </li></ul></ul><ul><ul><li>windshield compatibility) </li></ul></ul><ul><ul><li>Cars were sent to customers with non-functioning wiper system </li></ul></ul><ul><li>Initial diagnostics designated software as culprit for malfunction </li></ul><ul><ul><li>Mechanics couldn’t test software behavior </li></ul></ul><ul><ul><li>Other components (electronic control unit, sensor, and windshield) </li></ul></ul><ul><ul><li>functioned normally when tested independently </li></ul></ul><ul><ul><li>Failure was not of individual components, but in the interaction at a system level </li></ul></ul>
    4. 4. The Cost of Poor Software Functionality Moody’s Investment Services’ $1.7 billion dollar defect <ul><li>Software bug incorrectly assigns positive safe credit ratings to $4 Billion of high risk complex debt securities </li></ul><ul><li>Moody’s market value tumbles $1.7 Billion in one day </li></ul>
    5. 5. The Cost of Poor Software Performance A failed business-critical application can have a major impact on business
    6. 6. Applying automation where it matters the most Test Suites covering Many critical Requirements i.e. high business value. Test Suites No. of Requirements Test Suites covering All critical Requirements “20% of Test cases covering 80% of Business value” (Barry Boehm) Test Suites with high Requirements coverage Test Suites with low contribution “ 80% of the defects come from 20% of the modules” (Barry Boehm) => Most changed modules are often most defect-prone (Mockus, French)
    7. 7. Agenda <ul><ul><li>Challenges </li></ul></ul><ul><li>Executive summary & the Jazz vision </li></ul><ul><li>Rational Quality Management </li></ul><ul><li>Customer results </li></ul>
    8. 8. Executive summary Help customers continuously improve their ability to achieve desired business outcomes and deliver enduring software and product quality <ul><li>Cut risk and cost </li></ul><ul><ul><li>Collaborate seamlessly to reduce rework and the cost of bugs with integrated processes aligned to business goals </li></ul></ul><ul><li>Speed time to market </li></ul><ul><ul><li>Automation reduces error-prone tasks and improve consistency and operational efficiency </li></ul></ul><ul><li>Make confident decisions </li></ul><ul><ul><li>Manage to business objectives. Anywhere. In real-time with automated data collection and analysis using customized quality reports and dashboards </li></ul></ul>Delivering greater value from your investments in software
    9. 9. Successful businesses will manage software and systems delivery as a robust business process Repeatable activities producing a desired business outcome Achieve common goals by optimizing how people work Increase control and efficiency by integrating workflows Collaborate Continuously improve by measuring progress in real time Report Automate
    10. 10. Enable ALM across your organization Team Concert Collaborative Software Development and Software Delivery Agility Collaborative SCM, Change Management, Agile Planning, Dashboards / Metrics, Lifecycle Process Management Quality Manager Lifecycle Quality Management Coordinate quality assurance plans, processes and resources Requirements Composer Elicit, capture, elaborate, discuss and review requirements Business Expert and Stakeholder Collaboration Storage Collaboration Query Discovery Administration: Users, projects, process Best Practice Processes OPEN SERVICES Presentation: Mashups Future IBM Capabilities Product & Project Management Application Lifecycle Management Engineering & Software Tools Business Planning & Alignment Your existing capabilities 3 rd -Party Jazz Capabilities Compliance & Security Existing / Future Investments ClearCase, ClearQuest, RequisitePro.. Software Delivery Platform Products Other owned Vendor Products COTS, Business Management, etc Custom Developed Open Services for Lifecycle Collaboration Open Services Standards based framework
    11. 11. Integration framework for ALM protecting existing and future customer investments Open Services for Lifecycle Collaboration Open Services for Lifecycle Collaboration c Existing Rational Offerings New Rational/ IBM Offerings Business Partner Offerings Consume Consume Consume Consume Requirements Management Services Change Management Services Quality Management Services Architecture Management Services Publishing Services Publish Consume Publish Publish Publish Publish c Existing Rational Offerings New Rational/ IBM Offerings Business Partner Offerings Consume Consume Consume Consume Requirements Management Services Change Management Services Quality Management Services Architecture Management Services Publishing Services Publish Consume Publish Publish Publish Publish c Existing Rational Offerings New Rational/ IBM Offerings Business Partner Offerings
    12. 12. Lower the cost of delivering quality solutions Orchestrate across teams with ALM integration for maximum transparency and traceability of assets REST API Quality Management REST API Requirements Definition & Management Change & Configuration Management REST API Tester <ul><li>Coordinate analysts, developers and testers </li></ul><ul><ul><li>Collaborate in context and track status </li></ul></ul><ul><li>Align teams on project plans </li></ul><ul><ul><li>Which requirements are in the development plan? </li></ul></ul><ul><li>Track builds and the impact of defects </li></ul><ul><ul><li>What build is ready for test? </li></ul></ul><ul><ul><li>What defects are blocking test from proceeding? </li></ul></ul><ul><li>Track status with C/ALM dashboard mashups </li></ul><ul><ul><li>Transparency across teams and artifacts </li></ul></ul>Architect Developer Association Changes; notification Tester finds defects, Defect status Defects closed Requirement linked to work item Analyst
    13. 13. Centralized test management hub & full lifecycle support Collaboration Presentation: Mashups Discovery Query Storage Administration: Users, projects, process Best Practice Processes IBM Collaborative Application Lifecycle Management Quality Dashboard Test Management Create Plan Build Tests Manage Test Lab Execute Tests Report Results Requirements Management Defect Management Functional Testing Performance Testing Web Service Quality Code Quality Security and Compliance Open Platform homegrown Open Lifecycle Service Integrations Java System z, i SAP .NET Test Data Quality
    14. 14. <ul><ul><li>Stakeholder and team coordination reduces mistakes </li></ul></ul><ul><ul><li>Risk identification and management leads to educated prioritization decisions </li></ul></ul><ul><ul><li>Test traceability linked to business requirements improves customer satisfaction </li></ul></ul><ul><ul><li>Running tests earlier leads to reduced repair costs </li></ul></ul><ul><ul><li>Running more tests in less time improves coverage </li></ul></ul><ul><ul><li>Reducing manual labor leads to fewer testing errors </li></ul></ul><ul><ul><li>Lab configuration automation improves efficiency and asset utilization </li></ul></ul><ul><ul><li>Real-time dashboards enable proactive risk management </li></ul></ul><ul><ul><li>Customizable reports facilitate ongoing process improvement </li></ul></ul>IBM Rational Quality Manager A central hub for business-driven software quality Mitigate business risk with collaboration Improve operational efficiency with automation Make confident decisions with effortless reporting IBM Rational Quality Manager
    15. 15. Gartner MarketScope for Application Life Cycle Management IBM Rational earns “Strong Positive” – the highest possible rating IBM earns “ Strong Positive”―the highest possible rating The MarketScope is copyrighted November 11, 2010 by Gartner, Inc. and is reused with permission. The MarketScope is an evaluation of a marketplace at and for a specific time period. It depicts Gartner's analysis of how certain vendors measure against criteria for that marketplace, as defined by Gartner. Gartner does not endorse any vendor, product or service depicted in the MarketScope, and does not advise technology users to select only those vendors with the highest rating. Gartner disclaims all warranties, express or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. The MarketScope graphic was published by Gartner, Inc. as part of a larger research note and should be evaluated in the context of the entire report. The Gartner report is available upon request from IBM. Efficient coordination and automation of the delivery process requires new, collaborative approaches to the planning, measurement, execution, control and reporting of activities. These new approaches are what differentiate current application life cycle management (ALM) tools, and what make ALM processes vital to leading-edge development activities. ALM is what enables sustainable agile practices. ALM creates a management framework providing consistent, auditable records of the decisions and activities of agile teams.
    16. 16. Agenda <ul><ul><li>Challenges </li></ul></ul><ul><li>Executive summary & the Jazz vision </li></ul><ul><li>Rational Quality Management </li></ul><ul><li>Customer results </li></ul>
    17. 17. Customer Speak! Cut risk and cost Collaborate seamlessly to reduce rework and the cost of bugs with integrated processes aligned to business goals Lower the cost of delivering quality solutions Orchestrate across teams with ALM integration for maximum transparency and traceability of assets Unify the team through real-time, in-context collaboration A single, dynamic quality contract provides clear and accountable direction * Source: IBM “ Testing consumes 20% to 40+% of the average software application life cycle effort”* I just got a budget cut, what testing should I eliminate? What impact will it have on application production quality? “ Some large projects have found that 41% of all defects have their origin in bad requirements.”* Avoid disruption and achieve better business stability and project delivery predictability Achieve quality objectives by understanding and controlling sources of risk
    18. 18. Customer Speak! Speed time to market Automation reduces human error and improve operational efficiency Leverage investments in existing tooling and infrastructure Consolidate and reduce costs with support for heterogeneous environments and business partner offerings Optimize lab efficiency and asset utilization Save on test lab overhead, infrastructure and duration costs Ramp up productive teams day-1 Productivity boosters automate repetitive, mundane tasks for rapid ROI <ul><li>Sources: *IBM, ** Standish Group </li></ul>“ Testers are spending 40% of their time building test environments”* “ Testing can account for 30% to 50% of maintenance efforts”* “ We should reuse requirements and test artifacts and save time and money”
    19. 19. Unify the team through real-time, in-context collaboration A single, dynamic quality contract provides clear and accountable direction <ul><li>Unify the entire team with a shared view of quality assets </li></ul><ul><ul><li>Central location for assets (e.g., business objectives requirements, resources, platform, exit criteria) </li></ul></ul><ul><li>Comprehensive dynamic planning and updates </li></ul><ul><ul><li>Integrated process workflow, not artifacts drives team activities, hand-offs, reviews/approvals and sign-offs </li></ul></ul><ul><ul><li>Know what others are doing and what others expect of you </li></ul></ul><ul><ul><li>Task management for individuals and team </li></ul></ul>
    20. 20. Requirements driven testing Knowing what to test <ul><li>Requirements tracking built into the test management tooling </li></ul><ul><li>Customizable attributes enable you to track what is important to your team </li></ul><ul><li>Real-time impact analysis of requirements changes </li></ul><ul><li>Traceability of test results to user needs </li></ul>Know you are testing the right things
    21. 21. Comprehensive dynamic planning and updates Process flow, not artifacts drives team activities <ul><li>Live dynamic documentation </li></ul><ul><li>Defines test process </li></ul><ul><li>and strategy </li></ul><ul><li>Defines responsibilities </li></ul><ul><li>Activity based versus hierarchy </li></ul><ul><li>Business level reporting against quality objectives </li></ul>Word based Test Plan Other Test Plans Rational Quality Manager Plan
    22. 22. Collaborative risk based testing Risk management and prioritization <ul><li>Risk assessments captured in Test Plan and Test Cases </li></ul><ul><li>Collaboration planning of risk mitigation strategy </li></ul><ul><li>Test Case will contain a risk failure score and a risk priority score </li></ul><ul><li>Documented risk related decisions </li></ul>Base project decisions on qualitative risk analysis
    23. 23. Test Configurations 4 languages 4 browser types 5 databases 5 application servers 400 Combinations! Pairwise Optimizations Test the right 20 combinations Test Platform Environment Management Configuration awareness <ul><li>Focus your environment coverage </li></ul><ul><li>Document your environment coverage </li></ul><ul><li>Gain agreement across the project </li></ul>Test the right cases instead of everything. Plan optimal execution Test coverage optimization Focus resources on testing the right combinations
    24. 24. Source: GBS Test Practices study, 2005-2008, over 846 projects Reduce project cost and time from duplicates rework Duplicate defect detection Improve visibility and tracking of defects Typical Scenario Defects 13, 9, 7, 6, 4, 2 11, 7, 2 12, 9, 7, 4, 2 13, 11, 9, 8, 2 Ideal Scenario Defects 13, 9, 7, 6, 4, 2 11 12, 9 8
    25. 25. Tests based on requirements ensure deliverables meet customer expectations Quality is conformance to requirements DOORS Requirements Management Test Design Test Execution RequisitePro Requirements Composer Test Planning Test Status Process Automation and Increased Focus The test team is working against the right set of requirements Integrated Requirements Management Quality Manager
    26. 26. Integrating DOORS with IBM Rational Quality Manager “ Tests based on requirements ensure deliverables meet customer expectations” <ul><li>1. Plan Tests Early </li></ul><ul><li>Plan tests for each requirement as the requirement is written. </li></ul><ul><li>2. Conduct Tests Early </li></ul><ul><li>Perform tests as early as possible in the development process. </li></ul><ul><li>3. Relate Tests to Requirements </li></ul><ul><li>Trace tests back to the requirements they are design to check. </li></ul><ul><li>4. Relate Defects to Requirements </li></ul><ul><li>Trace defects back to the requirements that they show are not satisfied. </li></ul><ul><li>5. Measure Progress against Requirements </li></ul><ul><li>Set targets and measure the progress of testing in terms of those requirements that are shown to be satisfied or are not satisfied. </li></ul>DOORS Rational Quality Manager Requirements Management Test Status Test Planning Test Execution Test Design
    27. 27. Integrated Manual test authoring and execution <ul><li>Manual test author and execute </li></ul><ul><ul><li>Step by step capture and execution of manual tests </li></ul></ul><ul><ul><li>Assisted data entry </li></ul></ul><ul><ul><li>Keyword support for integrated manual and automated testing </li></ul></ul><ul><ul><li>Rich defect capture during execution, including screenshot and attachments </li></ul></ul><ul><ul><li>Simple intuitive interface for quick test execution </li></ul></ul>Maximizing efficiency of manual testing
    28. 28. <ul><li>Increase repeatability through automated test playback </li></ul><ul><li>Test more critical functions faster with automation </li></ul><ul><li>Automatically deploy your test environment and schedule the execution of your test Suites </li></ul><ul><li>Track and communicate progress and regressions throughout the testing lifecycle </li></ul>Integrated Functional and Regression test execution Accelerate test execution and deepen test coverage through automated test execution
    29. 29. <ul><li>Capture and track Business SLAs in your test plans </li></ul><ul><li>Create realistic user workloads that exercise the key business transactions </li></ul><ul><li>Deploy the environment and schedule execution of your workload </li></ul><ul><li>Identify the root cause of performance problems </li></ul><ul><li>Communicate results and areas of non-compliance to the stakeholders </li></ul>Integrated Performance test execution Comprehensive test planning, authoring, and analysis to identify and manage the risk of application performance failures Test Authoring and Control Load Generators Rational Performance Tester System Under Test Web Server App Server dB Server
    30. 30. Test lab automation and management Insight and control over the test lab <ul><li>Manage </li></ul><ul><ul><li>Verify that I have the resources required to fulfill my test plan </li></ul></ul><ul><li>Deploy </li></ul><ul><ul><li>Deliver the configurations my teams require for test </li></ul></ul><ul><li>Optimize </li></ul><ul><ul><li>Analyze patterns to minimize cost and maximize utilization </li></ul></ul>Work smarter, save on test lab overhead, infrastructure and duration costs 40% Lab Setup 60% Test Time 20% Setup 80% Test Time Free Cycles Non-automated tasks or steps (50% of setup effort) Application setup with RTLM (30-50% reduction) System setup with TPM (15-20% reduction) Ad-hoc or varying workflows and setup tasks 50% reduction in effort for test system deployment 20% gain Before lab automation With lab automation Source: IBM
    31. 31. Customer Speak! Make confident decisions Manage to business objectives. Anywhere. In real-time with automated data collection and analysis for custom quality reports and dashboards Confidently deliver incremental quality improvements Manage, measure and improve quality software delivery capability with a proven, repeatable approach Achieve project quality objectives each and every time Complete traceability across quality assets Make informed decisions and proactive change with real-time analysis and actionable reporting Measure and manage quality, project and team status performance and results “ We can do better, but don’t know what's not working, how bad it is, or where to start.” “ 2/3 of executives make more than half of their decisions based on ‘gut feel’ rather than verifiable information”* “ 77% of managers are aware of bad decisions made due to lack of access to accurate information”* *Source: Business Week
    32. 32. Assess and measure against Organizational policies <ul><li>Assessing status </li></ul><ul><ul><li>Standard Objectives </li></ul></ul><ul><ul><li>Reuse across Test Plans </li></ul></ul><ul><ul><li>All working toward same objectives </li></ul></ul><ul><ul><li>Measures against business objectives </li></ul></ul>Drive continuous and measured improvement
    33. 33. Make confident decisions with effortless reporting Closed Loop Analysis & Reporting <ul><li>Customizable reports and dashboards </li></ul><ul><ul><li>Reduce escalating cost of information gathering </li></ul></ul><ul><ul><li>Reduce risk by identifying trends before they become issues </li></ul></ul><ul><ul><li>Raise enterprise visibility and transparency to reduce costs and risk </li></ul></ul><ul><ul><li>Measures the effectiveness of processes and practices to improve organizational and business outcomes </li></ul></ul>Make the right decisions at the right time
    34. 34. Make informed decisions and proactively change with real-time analysis and actionable reporting Measure and manage quality, project and team status performance and results <ul><li>Measure quality process and project outcomes </li></ul><ul><ul><li>Real-time intelligence based on IT industry best-practice metrics, dashboards and models </li></ul></ul><ul><li>Inform quality decisions and drill into issues </li></ul><ul><ul><li>Alerts and automated analysis focuses owner to take action on root causes </li></ul></ul><ul><ul><li>52 out-of-the-box, customizable Cognos test management reports </li></ul></ul><ul><li>Take real-time action on relevant quality and project data </li></ul><ul><ul><li>Proven business intelligence backbone automates collection and analysis to improve lifecycle productivity </li></ul></ul>A customized report Cognos report in the dashboard
    35. 35. Rational Quality Manager Open Ecosystem Today Quality Manager Managing mobile applications testing across a global handset test environment Supporting bidirectional integration with the Jira change management system Enabling greater quality and productivity with automated SOA governance support Accelerating test cycles with virtual machine management and execution <ul><li>Requirements </li></ul><ul><li>Rational Requirements Composer </li></ul><ul><li>Rational ReqPro </li></ul><ul><li>Rational DOORS </li></ul><ul><li>Builds, WorkItems and Defects </li></ul><ul><li>Rational ClearQuest </li></ul><ul><li>Rational Team Concert </li></ul><ul><li>Rational BuildForge </li></ul>Quick Test Professional LoadRunner <ul><li>Provisioning </li></ul><ul><li>Tivoli Provisioning Manager (TPM) </li></ul><ul><li>Tivoli Service Request Manager </li></ul><ul><li>TADDM </li></ul><ul><li>Reporting </li></ul><ul><li>Rational Insight </li></ul>RapidRep Certify TMAP Process <ul><li>Automated Testing </li></ul><ul><li>Rational Functional Tester </li></ul><ul><li>Rational Performance Tester </li></ul><ul><li>Rational Service Tester for SOA Quality </li></ul><ul><li>Rational AppScan Tester Edition </li></ul><ul><li>Test RealTime </li></ul><ul><li>Rational Robot </li></ul><ul><li>Rational Rhapsody </li></ul>
    36. 36. Agenda <ul><ul><li>Challenges </li></ul></ul><ul><li>Executive summary & the Jazz vision </li></ul><ul><li>Rational Quality Management </li></ul><ul><li>Customer results </li></ul>
    37. 37. Customer Industry: Computer Services Solution Client Situation <ul><li>Defined and implemented a common set of IBM Rational tools to support end-to-end integration, including requirements gathering, configuration and change management, and defect management. Defined a common workflow to streamline processes. Established goals to drive tool adoption and metrics to measure productivity. Three key pilots drove process improvements and efficiencies: </li></ul><ul><li>Automation of regression testing </li></ul><ul><li>Increased team productivity and collaboration </li></ul><ul><li>Web scanning and remediation </li></ul><ul><li>Benefits </li></ul><ul><li>Realized over US$3 million in productivity savings </li></ul><ul><li>90% of developers, 80% of architects and project managers, 50% of testers using the Rational tools </li></ul><ul><li>31 migrations completed from third party tools to Rational, saving on license costs </li></ul><ul><li>Displaced shareware and homegrown tools created common processes across projects, reusable assets </li></ul><ul><li>Organization faced three main challenges: speed, quality and increased time to value needing to: </li></ul><ul><li>Deliver software projects to operations on time and within budget with greater reliability. </li></ul><ul><li>Ensure business critical applications met end-user requirements and performance criteria, while meeting regulatory standards. </li></ul><ul><li>Leverage a common development environment, reduce duplicate tools and processes, increase efficiencies and realize benefits earlier. </li></ul><ul><li>Rational Software </li></ul><ul><li>Rational AppScan Enterprise Edition, Rational Asset Manager, Rational ClearCase, Rational ClearQuest, Rational ClearQuest and Functional Testing, Rational Developer for System z, Rational Functional Tester, Rational Performance Tester, Rational Portfolio Manager, Rational Quality Manager Standard, Rational RequisitePro, Rational Software Architect, Rational Team Concert Standard </li></ul>Over US$3m productivity savings
    38. 38. Customer Industry: Telcommunications Solution Client Situation <ul><li>Organization’s project portfolio was consistently growing in size which caused internal pressures to deliver its projects faster while decreasing with necessary internal resources. However, due to the complexity of many of the projects, its siloed IT environment constricted such improvements. Moving forward, the organization needed to adopt new software technologies to create a fully integrated project delivery system that could deliver projects faster, on or under budget, and with fewer internal staff resources. </li></ul><ul><li>Software </li></ul><ul><li>Rational ClearCase, Rational ClearQuest, Rational Quality Manager, Rational RequisitePro </li></ul><ul><li>Adopting a comprehensive suite of IBM Rational software solutions provided the company with complete management traceability for software lifecycles and integrated the development environment to allow for simplified resource sharing between projects and teams. Software disciplines included configuration management, change management enterprise test management, and requirements management software. </li></ul><ul><li>Benefits </li></ul><ul><li>Able to complete development projects faster and within, or under, budget. In 2003, company completed only 36 projects. In 2006, it completed 67 projects with cost savings of US$3.7 million. </li></ul><ul><li>Reduced average project cycle time by 76 days while increasing overall software quality. </li></ul><ul><li>Improved relationship with its stakeholders by: delivering project-based benefits faster with improved use of its IT budget and better focus on deliver high business value. </li></ul>Reduced average project cycle time by 76 days
    39. 39. Customer Industry: Healthcare Solution Rational DOORS + Rational Quality Manager The company deployed DOORS in 2005 and has been using the tool to manage their project requirements effectively. Using consultative selling methods, the sales team identified their need to generate dynamic reports for FDA compliance which showed requirements and test information on the same page and avoid manual processes that used to take days to produce. Rational DOORS Web Access To increase visibility of requirements across the organization and avoid installation and training costs, the company also decided to use DOORS Web Access. Allowing other teams (testers, developers and business stakeholders) to review requirements and provide feedback during the requirements management process. Client Situation <ul><li>A global leader in the development, manufacturing and marketing of innovative medical products sells a comprehensive range of products in more than 60 countries worldwide through direct offices and a network of distributors. Challenges: </li></ul><ul><li>Reduce time spent in creating reports to demonstrate compliance with FDA </li></ul><ul><li>Establish an audit capability that has credibility and increases consistency </li></ul><ul><li>Improve collaboration between requirements analysts, developers and test teams across geographical boundaries </li></ul><ul><li>Be able to view end to end traceability on a single window </li></ul>12 month ROI
    40. 40. Previously in this series….. <ul><li>Collaborate, Integrate, Automate -- Revolutionize Your Distributed Development Environment </li></ul><ul><li>When: Wednesday, June 15, 2011 at 2:00 p.m. EDT </li></ul><ul><li> </li></ul>A Single Source of Truth -- Aligning Business Priorities and IT Project Deliverables When: Wednesday, June 22, 2011 at 2:00 p.m. EDT
    41. 41. What's Next <ul><li>Review the whole of CLM on our blog site and previous webcasts </li></ul><ul><li>Review & Download RQM or the full CLM at </li></ul><ul><ul><li> </li></ul></ul><ul><li>Have us do a deep dive with you and your team </li></ul><ul><li>Contact us for more details </li></ul><ul><ul><li>Kenny: [email_address] </li></ul></ul><ul><ul><li>386-232-8746 </li></ul></ul>
    42. 42. <ul><li>Learn more at: </li></ul><ul><li>IBM Rational software </li></ul><ul><li>Rational launch announcements </li></ul><ul><li>Rational Software Delivery Platform </li></ul><ul><li>Accelerate change & delivery </li></ul><ul><li>Deliver enduring quality </li></ul><ul><li>Enable enterprise modernization </li></ul><ul><li>Rational trial downloads </li></ul><ul><li>developerWorks Rational </li></ul><ul><li>Leading Innovation </li></ul><ul><li>IBM Rational TV </li></ul><ul><li>IBM Business Partners </li></ul><ul><li>IBM Rational Case Studies </li></ul><ul><li>Ensure Web security & compliance </li></ul><ul><li>Improve project success </li></ul><ul><li>Manage architecture </li></ul><ul><li>Manage evolving requirements </li></ul><ul><li>Small & midsized business </li></ul><ul><li>Targeted solutions </li></ul>