Using Defect Reports to Build Requirements Knowledge in Product Lines2nd International Workshop on Managing Requirements Knowledge (MaRK’09)Robyn LutzFlight Software Systems Engineering & Architectures Jet Propulsion Lab/Caltech & Iowa State UniversityNicolas RouquetteFlight Software Systems Engineering & Architectures Jet Propulsion Lab/CaltechPasadena, CA nicolas.rouquette@jpl.nasa.govrlutz@cs.iastate.eduThe research described in this paper was carried out in part at the Jet Propulsion Laboratory, California Institute of Technology ,and funded by NASA’s OSMA Software Assurance Research Program.

Problem OverviewGoal: use defect analysis to incrementally improve the requirements knowledge for a product lineApproach: use information in defect reports from previous product-line members to improve the quality of future members and reduce their requirements-related defectsChallenge: can we improve propagation of the requirements knowledge exposed by past defects to future products in the product line?Courtesy NASA/JPL-CaltechCourtesy NASA/JPL-Caltech2Courtesy Classroom ClipartCourtesy NASA/JPL-CaltechLutz_Rouquette_MaRK'09

Results OverviewOur analysis of defect reports from testing of flight software product line implicated undocumented, tacit requirements informationWe describe 4 types of new requirements knowledge found in the defect reports We are adapting two mechanisms, new to product lines, to better preserve and convey defect-derived requirements knowledge to upcoming product-line members3Lutz_Rouquette_MaRK'09

Some Related WorkKnowledge sharing: Savolainen and Sajaniemi; Maalej and Happel; Kruchten, Lago & van VlietTacit knowledge:Grunbacher & Briggs; Stone & SawyerRationale management:Dutoit and Paech; Lago & van VlietProduct Lines: Jirapanthong and Zisman; Trew; Cho, Lee & Kang; Pohl, Bockle & van der Linden; Weiss & LaiDefect analysis for improvement:Chillarege et al.; Dalal et al.; Ostrand & Weyuker; Mohagheghi, et al.; Lausen & Vinter; Fenton & Ohlsson; Leszak, Perry & Stoll; Lutz and Mikulski; Shull et al. Power of anecdotes: Petroski, Herschbach, Bayer, Rasmussen, DvoraktheTo the best of our knowledge, the work reported here is the first attempt to systematically use defect analysis to incrementally improve the requirements knowledge for a product line4Lutz_Rouquette_MaRK'09

Application DomainProduct line of flight software for spacecraft, previously used on seven missions managed by JPL, with two more missions in development Example of a commonality: all products have software fault protection to respond to power loss Example of a variability: the choice of reaction wheels used to control spacecraft attitude (i.e., position) differs among the products5Lutz_Rouquette_MaRK'09

ApproachProduct Lines offer an opportunity for incremental requirements knowledge and quality improvement.6Lutz_Rouquette_MaRK'09Defect analysis of earlier PL members, together with forward propagation of results, provides a way to achieve this.

TasksHow to capture relevant product-line requirements knowledge from the defect reportsUsed Orthogonal Defect Classification to identify defect patternsODC previously used on spacecraft anomalies [RE’03, TSE’04]How to pro-actively communicate the new requirements-related information to developers of future product-line membersPreserve: Specify the information so as to enable retrieval of the defects-learned requirements knowledge by future developers (a “pull” mechanism)Convey: Also need a “push” mechanism so that the requirements knowledge that could have prevented the defects in previous product-line applications will be remembered in future product-line applications 7Lutz_Rouquette_MaRK'09

1. Defect-derived Requirements KnowledgeNewly discovered requirements, e.g., complicated interface information surfaced during testingUnexpected requirements dependencies, e.g., incorrect assumptions about relationships among variabilitiesTacit* requirements rationales (insufficiently documented or incorrect), e.g., hardware idiosyncrasy known, but not by testersMisunderstood requirements, e.g., software behavior surprised testers, often due to ambiguous or partial documentation *Tacit knowledge: “knowledge that is essential but not documented” [Kruchten, Lago & van Vliet]Lutz_Rouquette_MaRK'098

2a). Preserve Defect-derived Tacit Requirements KnowledgeExtended Feature Model:FMs are extended with assumption specifications [Lago & van Vliet, ICSE’05]9Lutz_Rouquette_MaRK'09

2b). Convey Defect-derived Tacit Requirements KnowledgePLA-DPs (Product Line Analysis-Defect Paradigms)* are stories of past PL anomalies, packaged in the product-line repository & told to future PL projects to “remember old lessons learned”“During testing on MRO the power software got confused [defect due to erroneous assumption] when the Reaction Wheel (to aim the spacecraft) was turned off by an uplinked software command & right back on by the fault-protection software. It turned out that there was a race condition which revealed a new software requirement. This is why [rationale] the software sometimes disables the schedule-verification function when the switch is being commanded.”Courtesy Classroom Clipart*Modeled on Petroski’s Design Paradigms10Lutz_Rouquette_MaRK'09

ConclusionDefect reports from past product-line members are a rich, and insufficiently tapped, source of tacit requirements knowledge for future product-line membersFuture work will expand scale (to include more members of the product line) and continue investigating how to better preserve and convey defect-derived requirements knowledge feature models extended with assumption specifications (formal)structured anecdotes of paradigmatic product-line defects (informal)11Lutz_Rouquette_MaRK'09