Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deployment of a test management solution for a defence project using an integrated tool chain


Published on

The presentation shows how a test management solution has been established for a defence project in compliance with a set of applicable standards using an integrated IBM Rational tool chain consisting of Rational Quality Manager for test management, IBM Rational DOORS for requirement management, IBM Rational Team Concert for defect management, IBM Rational Publishing Engine for automatic generation of project deliverables and last - but not least - IBM Rational Insight for trend and status reporting.

Published in: Engineering, Technology, Business
  • Be the first to comment

  • Be the first to like this

Deployment of a test management solution for a defence project using an integrated tool chain

  1. 1. © 2014 IBM Corporation Deployment of a test management solution for a defence project using an integrated tool chain Code 1183 Dr. Einar Karlsen (IBM, Co-Speaker) Thomas Schöning (Airbus DS)
  2. 2. 1 Introduction Dr. Einar Karlsen (IBM) -Technical Solution Architect Thomas Schöning (Airbus Defence and Space) - Integration and Test Manager Thomas Südbröcker (IBM) - Technical Sales and Solutions + Community of Practices Leader Quality Management (Germany)
  3. 3. 2 • Subcontractor in a NATO Project with requirement based development, test and formal verification of a complex system entity • Find a proper tooling solution for test and verification process using the IBM Rational tooling chain (Jazz Platform) – Support for relevant standards ISO/IEE 15288, ISO/IEEE 12207, ISO 9001, ISO/IEC 9126 – Support for relevant process: Waterfall (V-Model) with incremental SW Development – Integration with IBM Rational DOORS to accommodate for requirement traceability – Automatic document generation for requirements, design and test artefacts (plans, procedures, reports) – Insight into status and trend of the testing efforts (metrics) Setting the Scene – Situation
  4. 4. 3 • Challenge – Unknown effort to adequately install, integrate and customize the whole tooling chain to perform modelling, implementation, and test management – Appropriate test management support (planning, execution, reporting, document generation, etc) according to required process? • Improvements – Decrease of costs, risks, inconsistencies and time-to-market through automatic generation of project deliverables – Collection of metrics to measure test design progress, requirements coverage – Online/Offline determination of test results while performing the test Setting the Scene – Risk and Opportunity
  5. 5. 4 • Profound orientation for configuration and customization of IBM Rational Tools for a tool-aided test management solution (planning, execution, reporting, documentation) for similar test activities 1. Start early to implement test management prototyping to better estimate the effort to adapt the tools to the project needs 2. Invite technical professionals to have guidance and avoid pitfalls 3. Perform team training 4. Look at the deliverables before customizing your tool chain Setting the Scene – Position and Benefits
  6. 6. 5 • Introduction – Programme Overview – Test and Evaluation Strategy • Test management – Solution Overview – Solution Architecture – Document Generation – Metrics and Dashboards • Summary Agenda
  7. 7. 6 • Confidential, multinational NATO C4ISR Project (C4ISR=Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance) – Unmanned Air, Command and Control Entities – Transportable and Mobile Ground Stations for data exception, processing, and archiving facilities – Communication links to other NATO and interoperable national ground stations, and to NATO and national Command and Control systems – Exchange of standardized data via different interfaces and communication lines • Airbus DS responsible for development, test and formal verification of a system entity consisting of several Subsystems Programme Overview
  8. 8. Test and Evaluation Strategy Verification Development Testing Entity Subsystems CSCIsHWCIs ServicesComponents Development Testing Development TestingTest Test Test Verification Integration
  9. 9. 8 • Requirements of System Entity, Subsystems, Computer Software Configuration Items (CSCIs), and Hardware Configuration Items (HWCIs) are kept in DOORS database with traceable entries • Testing/Verification is classified either to be – Informal Tests • conduct informal tests throughout the system development – Subcontractor Development Tests • For CSCI, HWCI, HW/CSCI groupings, Subsystem level, Entities and Integration testing – Formal Verification • Verification of allocated customer requirements based on proof of derived requirements (Entity, Subsystem, HW/CSCI groupings) Test and Evaluation Strategy
  10. 10. Test Management – Solution Overview 9
  11. 11. 10 Test Management – Solution Overview • Test Plan – consists of a number of Test Cases used to proof requirements – is linked with one or more Requirements Modules (Test Plan views) coverage and traceability (verification level) • Test Case – is linked to one or more particular requirements belonging to the requirements view(s) proofing requirements on different levels • Test Script – keeps the Test Procedure with single steps for test execution • Test Case Execution Record/Test Case Result – container to keep the Test Case results depending on iteration (integration, dry-run, acceptance) and test environment • PTR (=Problem Trouble Report) – keeps the defects occurred during Test Case execution
  12. 12. Test Management – Solution Architecture 11 IBM Rational DOORS IBM Rational Insight IBM Rational Publishing Engine REST, ETL REST, ETL IBM Rational Quality Manager IBM Rational Team Concert REST, ETL Dashboards, Status and trend reports Test Plans, Procedures and Reports (Project Deliverables) RESTREST REST OSLC links OSLC links
  13. 13. 12 Test Management – Test Document Generation
  14. 14. Test Document Generation – Master Test List
  15. 15. Test Document Generation – Test Case Details
  16. 16. 15 • Metrics: What you can measure that can be done! • Metric types: Project, Product, Process, and Personnel • Definition of metrics for monitoring and control test planning, design, development and execution – test planning • Percentage of requirements having a test case – test design (analysis and concept) • Percentage of test case with different status (draft, ready for review, approved) – test development • Percentage of test scripts with different status (draft, ready for review, approved) – test execution • Percentage of planned test cases being executed (passed, passed with condition, failed, blocked, error) Test Management – Metrics and Dashboards
  17. 17. 16 Metrics and Dashboards – Test Case Design
  18. 18. 17 Metrics and Dashboards – Test Case Execution
  19. 19. 18 • Tooling Solution (Jazz platform) for test management process with combined DT&E strategy (Informal, Development Tests, Formal Verification) applied for system entity (NATO Project) • Test Management Solution (Overview and Architecture) – Test Plans linked with requirements view – Test Cases linked with requirements – Test Execution Records keeping the results for each Iteration – Definition of timelines, iterations to support test planning and execution • Test Document Generation – Tooling adaption for artefact templates (Test Plan, Test Case) – Examples (Master Test List, Test Case Details) • Metrics and Dashboards – metrics for monitoring and control test planning, design, development and execution Summary
  20. 20. Articles on developerWorks • RQM Template authoring articles on developer Works. – Using IBM Rational Publishing Engine to generate compliance documents: Report generation for Rational Quality Manager, introducing REST service URLs, importing schemas, and basic template authoring – Using IBM Rational Publishing Engine to generate compliance documents: Part 2. Report generation for Rational Quality Manager, showing how to extract information from grid sections and report on test results and related defects
  21. 21. Thank You! Your Feedback is Important! Access the Innovate agenda tool to complete your session surveys from your smartphone, laptop or conference kiosk.
  22. 22. 21 • Test Plan: – Artifact Templates (keep Test Cases and TCERs only) – Artifact Categories (manufacturer, part number, type) to distinguish between different plans (system, software,etc) • Test Case: – Artifact Template has been customized to the documentation format needs (Test Plan, Test Procedure) – Artifact Categories (test area, test method, test event, test type) and RQM section (description, test objective, requirement links, estimation) are used to appropriately keep Test Case data • Timelines (Development and Production Phase) • Iteration Types (Milestone, Software Test, System Test) • Iteration – directly corresponds with incremental software development (integration tests, dry run, acceptance test) – support generation of TCER Test Management – Tooling adaption
  23. 23. 22 • Administration of project requirements and test cases as well as the creation of bi-directional dependencies between them • Planning test effort and milestones • Defining test goals, acceptance and quality criteria • Deriving test cases derived from requirement specifications • Estimation of the effort of the test activities • Definition of a time plan for each test iteration with finish dates of key activities • Collection of different test environments and the generation of test configurations • Creation and administration of test cases (manual and automated tests) • Recording test results, creating defect and evaluating reports • Support review and release process for the test plan and individual test cases. IBM Rational Tools – Usage