Software Reuse: Metrics and ModelsWILLIAM FRAKESVirginia TechandCAROL TERRYINCODE Corporation            As organizations ...
416       •       W. Frakes and C. Terry CONTENTS                                               well as quality and produc...
Software Reuse: Metrics and Models                    •       417                       Table 1.   Reusable Aspects of Sof...
418       •       W. Frakes and C. Terry                                   Table 2.    Types of Software Reuse(R Յ 1). (No...
Software Reuse: Metrics and Models                    •       419                 Table 3.   Barnes’ and Bollinger’s econo...
420       •       W. Frakes and C. Terry          Table 4.   Costs and payoff threshold values for reusable components [Fa...
Software Reuse: Metrics and Models                   •       421                        Table 5.   Observable Data [Poulin...
422       •       W. Frakes and C. Terry                 Table 6.   Characteristics of SEL subsystems [Agresti and Evanco ...
Software Reuse: Metrics and Models                   •       423   Table 7.   Mean Development Time and [95% Confidence In...
424       •       W. Frakes and C. Terry (rated from extensively modified to un-           productivity. Gaffney and Durek...
Software Reuse: Metrics and Models                  •       425                    Table 9.   Hudson and Koltun Reuse Matu...
426       •       W. Frakes and C. Terryoritize goals and build successive stages                  reuse level metrics tha...
Software Reuse: Metrics and Models                  •       427M ϭ the number of items not from an               T ϭ total...
428       •       W. Frakes and C. Terry(3)   total reuse level;                                 aged, with similar defini...
Software Reuse: Metrics and Models                  •       4294.3 Reuse Predictions for Life Cycle                No Atte...
430       •       W. Frakes and C. Terry                                    Figure 2.      Reuse library system.This data ...
Software Reuse: Metrics and Models                 •       431   Indexing costs include the cost of cre-   can be measured...
432       •       W. Frakes and C. Terry                       Table 10.   Summary of Software Reuse Metrics and ModelsACM...
Software Reuse: Metrics and Models                     •       433Table A1.   Definitions of Reuse Types                  ...
434       •       W. Frakes and C. Terry ment/contractor, commercial, aca-                 overlap in meaning. For example...
Software Reuse: Metrics and Models                    •       435    An empirical study of software design practices.   FR...
Upcoming SlideShare
Loading in...5
×

Art06

239

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
239
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Art06

  1. 1. Software Reuse: Metrics and ModelsWILLIAM FRAKESVirginia TechandCAROL TERRYINCODE Corporation As organizations implement systematic software reuse programs to improve productivity and quality, they must be able to measure their progress and identify the most effective reuse strategies. This is done with reuse metrics and models. In this article we survey metrics and models of software reuse and reusability, and provide a classification structure that will help users select them. Six types of metrics and models are reviewed: cost-benefit models, maturity assessment models, amount of reuse metrics, failure modes models, reusability assessment models, and reuse library metrics. Categories and Subject Descriptors: D.2.8 [Software Engineering]: Metrics; D.2.m [Software Engineering]: Miscellaneous—reusable software; K.6.0 [Management of Computing and Information Systems]: General—economics; K.6.4 [Management of Computing and Information Systems]: System Management—quality assurance General Terms: Economics, Measurement, Performance Additional Key Words and Phrases: Cost-benefit analysis, maturity assessment, reuse level, object-oriented, software reuse failure modes model, reusability assessment, reuse library metrics, software, reuse, reusability, models, economics, quality, productivity, definitions1. INTRODUCTION Isoda 1994]. Organizations implement- ing systematic software reuse programsSoftware reuse, the use of existing soft- must be able to measure their progressware artifacts or knowledge to create and identify the most effective reusenew software, is a key method for signif- strategies.icantly improving software quality and In this article we survey metrics andproductivity. Reusability is the degree models of software reuse and reusabil-to which a thing can be reused. To ity. A metric is a quantitative indicatorachieve significant payoffs a reuse pro- of an attribute of a thing. A model spec-gram must be systematic [Frakes and ifies relationships among metrics. InAuthors’ addresses: W. Frakes, Virginia Tech, Computer Science Department, 2990 Telestar Court,Falls Church, VA 20165; email: ͗frakes@sarvis.cs.vt.edu͘; C. Terry, INCODE Corp., 1100 S. Main St.,Blacksburg VA 24060; email: ͗cterry@bev.net͘.Permission to make digital / hard copy of part or all of this work for personal or classroom use is grantedwithout fee provided that the copies are not made or distributed for profit or commercial advantage, thecopyright notice, the title of the publication, and its date appear, and notice is given that copying is bypermission of the ACM, Inc. To copy otherwise, to republish, to post on servers, or to redistribute tolists, requires prior specific permission and / or a fee.© 1996 ACM 0360-0300/96/0600–0415 $03.50 ACM Computing Surveys, Vol. 28, No. 2, June 1996
  2. 2. 416 • W. Frakes and C. Terry CONTENTS well as quality and productivity payoff. Maturity assessment models categorize reuse programs by how advanced they are in implementing systematic reuse. 1. INTRODUCTION Amount of reuse metrics are used to as- 1.1 Types of Reuse sess and monitor a reuse improvement 2. COST BENEFIT ANALYSIS 2.1 Cost/Productivity Models effort by tracking percentages of reuse for 2.2 Quality of Investment life cycle objects. Failure modes analysis 2.3 Business Reuse Metrics is used to identify and order the impedi- 2.4 Relation of Reuse to Quality and Productivity ments to reuse in a given organization. 3. MATURITY ASSESSMENT 3.1 Koltun and Hudson Reuse Maturity Model Reusability metrics indicate the likeli- 3.2 SPC Reuse Capability Model hood that an artifact is reusable. Reuse 4. AMOUNT OF REUSE library metrics are used to manage and 4.1 Reuse Level 4.2 Reuse Metrics for Object-Oriented Systems track usage of a reuse repository. Organi- 4.3 Reuse Predictions for Life Cycle Objects zations often encounter the need for 5. SOFTWARE REUSE FAILURE MODES MODEL these metrics and models in the order 6. REUSABILITY ASSESSMENT presented. 7. REUSE LIBRARY METRICS 8. SUMMARY Software reuse can apply to any life APPENDIX 1: DEFINITIONS OF TYPES OF REUSE cycle product, not only to fragments of REFERENCES source code. This means that developers can pursue reuse of requirements docu- ments, system specifications, design structures, and any other developmentFigure 1, reuse models and metrics are artifact [Barnes and Bollinger 1991].categorized into types: (1) reuse cost- Jones [1993] identifies ten potentiallybenefits models, (2) maturity assess- reusable aspects of software projects asment, (3) amount of reuse, (4) failure shown in Table 1.modes, (5) reusability, and (6) reuse li- In addition to these life cycle prod-brary metrics. Reuse cost-benefits models ucts, processes (such as the waterfallinclude economic cost/benefit analysis as model of software development and the Figure 1. Categorization of reuse metrics and models.ACM Computing Surveys, Vol. 28, No. 2, June 1996
  3. 3. Software Reuse: Metrics and Models • 417 Table 1. Reusable Aspects of Software ProjectsSEI capability maturity model), and 2. COST BENEFIT ANALYSISknowledge are also potentially reusable. As organizations contemplate system- atic software reuse, the first question1.1 Types of Reuse that will arise will probably concernClear definitions of types of reuse are costs and benefits. Organizations willnecessary prerequisites to measure- need to justify the cost and time in-ment. Table 2 provides a faceted classi- volved in systematic reuse by estimat-fication of reuse definitions gathered ing these costs and potential payoffs.from the literature. Each column speci- Cost benefit analysis models includefies a facet, with the facet name in bold. economic cost-benefit models and qual-Below each facet are its corresponding ity and productivity payoff analyses.terms. (See Appendix 1 for detailed def- Several reuse cost-benefit modelsinitions of the terms in the table.) have been reported. None of these mod-Terms in parentheses are synonyms. els are derived from data, nor have theyDevelopment scope refers to whether the been validated with data. Instead, thereusable components are from a source models allow a user to simulate theexternal or internal to a project. Modifi- tradeoffs between important economiccation refers to how much a reusable parameters such as cost and productiv-asset is changed. Approach refers to dif- ity. These are estimated by setting arbi-ferent technical methods for implement- trary values for cost and productivitying reuse. Domain scope refers to measures of systems without reuse, andwhether reuse occurs within a family of then estimating these parameters forsystems or between families of systems. systems with reuse. There is consider-Management refers to the degree to able commonality among the models, aswhich reuse is done systematically. Re- described in the following.used entity refers to the type of thereused object. 2.1 Cost/Productivity Models Reuse in an organization can be de-fined by selecting appropriate facet- Gaffney and Durek [1989] propose twoterm pairs from this table. For example, cost and productivity models for soft-an organization might choose to do both ware reuse. The simple model shows theinternal and external code reuse, allow- cost of reusing software components. Theing only black box modification as part cost-of-development model builds uponof a compositional approach. They the simple model by representing the costmight choose to focus on reuse within of developing reusable components.their domain (vertical) and to pursue a The simple model works as follows.systematic management approach. Let C be the cost of software develop- The following sections discuss in detail ment for a given product relative to alleach of the six types of reuse metrics and new code (for which C ϭ 1). R is themodels. proportion of reused code in the product ACM Computing Surveys, Vol. 28, No. 2, June 1996
  4. 4. 418 • W. Frakes and C. Terry Table 2. Types of Software Reuse(R Յ 1). (Note that R is a type of reuse The cost of development model in-level; this topic is discussed in detail in cludes the cost of reusing software andSection 4.) b is the cost relative to that the cost of developing reusable compo-for all new code, of incorporating the nents as follows. Let E represent thereused code into the new product ( b ϭ 1 cost of developing a reusable componentfor all new code). The relative cost for relative to the cost of producing a com-software development is: ponent that is not reusable. E is ex- pected to be Ͼ1 because creating a com-[(relative cost of all new code) ponent for reuse generally requires extra effort. Let n be the number of uses ‫ ͑ ء‬proportion of new code ͒ ] over which the code development cost will be amortized. The new value for C ϩ ͓͑ relative cost of reused software) (cost) incorporates these measures: ‫ ͑ ء‬proportion of reused software͒ ]. C ϭ ͑ b ϩ ͑ E/n ͒ Ϫ 1 ͒ R ϩ 1 .The equation for this is: Other models help a user estimate the C ϭ ͑ 1 ͒͑ 1 Ϫ R ͒ ϩ ͑ b ͒͑ R ͒ effect of reuse on software quality (num- ber of errors) and on software develop- ϭ ͓͑ b Ϫ 1 ͒ R ͔ ϩ 1 , ment schedules. Using reusable software generally results in higher overall devel-and the corresponding relative opment productivity; however, the costsproductivity is, of building reusable components must be P ϭ 1/C ϭ 1/ ͑͑ b Ϫ 1 ͒ R ϩ 1 ͒ . recovered through many reuses. Some empirical estimates of the relative cost ofb must be Ͻ 1 for reuse to be cost producing reusable components and foreffective. The size of b depends on the cost recovery via multiple reuses are dis-life cycle phase of the reusable compo- cussed in the next section.nent. If only the source code is reused, Applications of Cost/Productivitythen one must go through the require- Models. Margono and Rhoads [1993]ments, design, and testing to complete applied the cost of development modeldevelopment of the reusable component. to assess the economic benefits of a re-In this case, Gaffney and Durek esti- use effort on a large-scale Ada projectmate b ϭ 0.85. If requirements, design, (the United States Federal Aviation Ad-and code are reused as well, then only ministration’s Advanced Automationthe testing phase must be done and b is System (FAA/AAS)). They applied theestimated to be 0.08. model to various types of software cate-ACM Computing Surveys, Vol. 28, No. 2, June 1996
  5. 5. Software Reuse: Metrics and Models • 419 Table 3. Barnes’ and Bollinger’s economic investment modelgorized by the source (local, commercial, listed in order of increasing complexity.or public) and mode of reuse (reused (The quoted definitions are from Boochwith or without change). The equation for [1987].)C in the reuse economics model was mod-ified to reflect acquisition, development, Monolithic:and integration costs. Results show that “Denotes that the structure is alwaysthe cost of developing for reuse is often treated as a single unit and that itstwice the cost of developing an equivalent individual parts cannot be manipu-nonreusable component. lated. Monolithic components do not Favaro [1991] utilized the model from permit structural sharing; stack,Barnes et al. [1988] to analyze the eco- string, queue, deque, ring, map, set,nomics of reuse. (Note: the Barnes et al. and bag components are monolithic.”[1988] model is the same as that of Polylithic:Gaffney and Durek [1989].) The rele- “Denotes that the structure is notvant variables and formulas are shown atomic and that its individual parts canin Table 3. be manipulated. Polylithic components Favaro’s research team estimated the permit structural sharing; lists, trees,quantities R and b for an Ada-based and graph components are polylithic.”development project. They found it diffi- Graph:cult to estimate R because it was un- “A collection of nodes and arcs (whichclear whether to measure source code or are ordered pairs of nodes).”relative size of the load modules. The Menu, mask:parameter b was even more difficult End-products of the project. They wereto estimate because it was unclear developed as generalized, reusable com-whether cost should be measured as the ponents and so were included in theamount of real-time necessary to install study.the component in the application andwhether the cost of learning should be Table 4 shows values reported by Fa-included. varo for E, the relative cost of creating a Favaro classified the Booch compo- reusable component, b, the integrationnents according to their relative com- cost of a reusable component, and N0,plexity. The following categories are the payoff threshold value. ACM Computing Surveys, Vol. 28, No. 2, June 1996
  6. 6. 420 • W. Frakes and C. Terry Table 4. Costs and payoff threshold values for reusable components [Favaro 1991] The overall costs of reusable compo- producer activities and consumer activi-nents relative to nonreusable compo- ties. Producer activities are reuse invest-nents is E ϩ b. b is expected to be less ments, or costs incurred while makingthan 1.0, since it should be cheaper to one or more work products easier to reuseintegrate a reusable component than to by others. Consumer activities are reusecreate the component from scratch. E is benefits, or measures in dollars of howgreater than or equal to 1.0, showing much the earlier reuse investment helpedcosts of developing reusable components or hurt the effectiveness of an activity.are higher than costs of developing non- The total reuse benefit can then be foundreusable components. Favaro found that by estimating the reuse benefit for allthe cost of reusability increased with subsequent activities that profit from thethe complexity of the component. Mono- reuse investment.lithic components were so simple there The quality of investment (Q) is thewas no extra cost to develop them as ratio of reuse benefits (B) to reuse invest-reusable components. In contrast, the ments (R): Q ϭ B/R. If Q is less than onecost of the mask component more than for a reuse effort, then that effort resulteddoubled as it was generalized. The inte- in a net financial loss. If Q is greater thangration cost b was also high in complex one, then the investment provided a goodapplications. The values for N0 show return. Three major strategies are identi-that the monolithic and polylithic com- fied for increasing Q: increase the level ofponent costs are amortized after only reuse, reduce the average cost of reuse,two reuses. However, the graph compo- and reduce the investment needed tonent must be used approximately five achieve a given reuse benefit.times before its costs are recovered, andthe most complex form of the mask will 2.3 Business Reuse Metricsrequire thirteen reuses for amortiza-tion. In summary, costs rise quickly Poulin et al. [1993] present a set of met-with component size and complexity. rics used by IBM to estimate the effort saved by reuse. The study weighs poten-2.2 Quality of Investment tial benefits against the expenditures of time and resources required to identifyBarnes and Bollinger [1991] examined and integrate reusable software into athe cost and risk features of software product. Although the measures used arereuse and suggested an analytical ap- similar to the Cost/Productivity Modelsproach for making good reuse invest- [Gaffney and Durek 1989] already dis-ments. Reuse activities are divided into cussed, metrics are named from a busi-ACM Computing Surveys, Vol. 28, No. 2, June 1996
  7. 7. Software Reuse: Metrics and Models • 421 Table 5. Observable Data [Poulin et al. 1993]ness perspective, and they provide a finer avoidance is calculated as the sum ofbreakdown for some calculations. For ex- cost avoidance in the developmentample, cost is broken down into develop- and maintenance activities.ment costs and maintenance costs. The metrics are derived from a set of Development cost avoidancedata elements defined in Table 5. ϭ RSI ϫ 0.8 ϫ ͑ new code cost͒ Given the preceding data, the follow-ing metrics are defined: Service cost avoidance—Reuse percent: reflects how much of the product can be attributed to re- ϭ RSI ϫ ͑ error rate͒ use. (This is equivalent to R in the ϫ ͑ new code cost͒ Gaffney and Durek model.) Poulin et al. distinguish the reuse percent of a Reuse cost avoidance product, the reuse percent of a prod- uct release, and the reuse percent for ϭ Development cost avoidance the entire organization. ϩ Service cost avoidance . Product Reuse percent —Reuse value added: a productivity in- dex that differs from the previous def- ϭ RSI/(RSI ϩ SSI) ϫ 100 percent . initions of relative productivity by in-—Reuse cost avoidance: measures re- cluding in the definition of reused duced total product costs as a result of code source code that is reused within reuse. (This is equivalent to 1 Ϫ RC in the product and source code that is the Gaffney and Durek model.) Poulin reused by others. (This is quite simi- et al. estimate that the financial ben- lar to internal and external reuse, efit attributable to reuse during the discussed in Section 4.1.) development phase is 80 percent of the cost of developing new code, de- Reuse value added rived from studies showing that the ϭ ͑ SSI ϩ RSI ϩ SIRBO͒ /SSI . cost of integrating an existing soft- ware element is 20 percent of the cost —Additional development cost: in- of new development (b in the Gaffney creased product costs as a result of and Durek model). This study also developing reusable software (same acknowledges savings that are real- as E in the Barnes and Bollinger mod- ized in the maintenance phase as re- el). This study estimates the cost of used software generally contains additional effort at 50 percent of the fewer errors; the total reuse cost cost of new development. ACM Computing Surveys, Vol. 28, No. 2, June 1996
  8. 8. 422 • W. Frakes and C. Terry Table 6. Characteristics of SEL subsystems [Agresti and Evanco 1992] Additional Development Cost slight modification, Յ 25% of lines changed) were between 26% and 28%. ϭ ͑ relative cost of reuse Ϫ 1 ͒ Defect densities were between 3.0 and 5.5 total defects per KSLOC. Table 6 ϫ code written for reuse by others shows four sample rows summarizing ϫ new code cost . the project characteristics of the sub- systems showing that a high level ofRelative cost of writing for reuse is the reuse correlates with a low defect den-cost of writing reusable code relative to sity (size is in KSLOC units).the cost of writing code for 1-time use The Reusability-Oriented Parallel pro-(estimated at 1.5). Code written for re- gramming Environment (ROPE) [Browneuse by others is the kloc of code written et al. 1990] is a software componentfor reuse by the initiating project. reuse system that helps a designer find and understand components. ROPE is2.4 Relation of Reuse to Quality and integrated with a development environ- Productivity ment called CODE (Computation-Orient-Because systematic software reuse is ed Display Environment), which supportsuncommon, empirical evidence relating construction of parallel programs using asoftware reuse to quality and productiv- declarative and hierarchical graph modelity is limited. However, several re- of computation. ROPE supports reuse ofsearchers have accumulated and pub- both design and code components, focus-lished statistics that support the notion ing on the key issues of reusability: find-that software reuse improves quality ing components, then understanding,and productivity. modifying, and combining them. An ex- Agresti and Evanco [1992] conducted periment was conducted to investigatea study to predict defect density, a soft- user productivity and software quality forware quality measurement, based on the CODE programming environment,characteristics of Ada designs. They with and without ROPE.used 16 subsystems from the Software The experimental design includedEngineering Laboratory (SEL) of NASA metrics such as fraction of code in aGoddard Space Flight Center. The SEL program consisting of reused compo-project database provided data on the nents, development time, and errorextent of reuse and subsystem identifi- rates. Reuse rates were reported as “ex-cation for each compilation unit, and tremely high” for the 43 programs writ-reported defects and nondefect modifi- ten using ROPE, with a mean reusecations. Collectively, approximately 149 rate for a total program (code and de-KSLOC (kilo-source lines of code) were sign) of 79%. The researchers used totalanalyzed. The project database showed development time to measure the effectthat the reuse ratios (fraction of compi- of reusability on productivity. Table 7lation units reused verbatim or with shows the development time in hoursACM Computing Surveys, Vol. 28, No. 2, June 1996
  9. 9. Software Reuse: Metrics and Models • 423 Table 7. Mean Development Time and [95% Confidence Intervals in Hours] [Browne et al. 1990] Table 8. Mean Number of Errors and 95% Confidence Intervals [Browne et al. 1990]for subjects programming in the CODE tween the measures of reuse rate, devel-environment and those programming in opment time, and decreases in numberCODE and ROPE. The data reveal that of errors.ROPE had a significant effect on devel- Card et al. [1986] conducted an em-opment time for all the experimental pirical study of software design prac-programs. tices in a FORTRAN-based scientific Error rates were used to measure computing environment. The goals ofquality. Compile errors, execution er- the analysis were to identify the typesrors, and logic errors were all counted. of software that are reused and to quan-The results are shown in Table 8. The tify the benefits of software reuse. Theuse of ROPE reduced error rates, but results were as follows.the data are less clear than those forproductivity. The researchers attribute —The modules that were reused with-this to the difficulty of collecting the out modification tended to be smalldata and to the lack of distinction be- and simple, exhibiting a relativelytween design and code errors. low decision rate. In summary, the CODE/ROPE exper- —Extensively modified modules tendediment showed a high correlation be- to be the largest of all reused software ACM Computing Surveys, Vol. 28, No. 2, June 1996
  10. 10. 424 • W. Frakes and C. Terry (rated from extensively modified to un- productivity. Gaffney and Durek be- changed) in terms of the number of lieve that the costs of building reus- executable statements. able parts must be shared across many—98 percent of the modules reused users to achieve higher payoffs from without modification were fault free software reuse. and 82 percent of them were in the lowest cost per executable statement 3. MATURITY ASSESSMENT category.—These results were consistent with a Reuse maturity models support an as- previous Software Engineering Labo- sessment of how advanced reuse pro- ratory study [Card et al. 1982] which grams are in implementing systematic showed that reusing a line of code reuse, using an ordinal scale of reuse amounts to only 20 percent of the cost phases. They are similar to the Capabil- of developing it new. ity Maturity Model developed at the Software Engineering Institute (SEI) at Matsumura [Frakes 1991] described Carnegie Mellon University [Humphreyan implementation of a reuse program 1989]. A maturity model is at the core ofat Toshiba. Results of the reuse pro- planned reuse, helping organizationsgram showed a 60 percent ratio of re- understand their past, current, and fu-used components and a decrease in er- ture goals for reuse activities. Severalrors by 20 to 30 percent. Managers felt reuse maturity models have been devel-that the reuse program would be profit- oped and used, though they have notable if a component were reused at least been validated.three times. A study of reuse in the object-orientedenvironment was conducted by Chen 3.1 Koltun and Hudson Reuse Maturityand Lee [1993]. They developed an envi- Modelronment, based on an object-orientedapproach, to design, manufacture, and Koltun and Hudson [1991] developeduse reusable Cϩϩ components. A con- the reuse maturity model shown in Tabletrolled experiment was conducted to 9. Columns indicate phases of reuse ma-substantiate the reuse approach in turity, assumed to improve along an ordi-terms of software productivity and qual- nal scale from 1 (Initial/Chaotic) to 5ity. Results showed improvements in (Ingrained). Rows correspond to dimen-software productivity of 30 to 90 percent sions of reuse maturity such as Motiva-measured in lines of code developed per tion/Culture and Planning for Reuse. Forhour (LOC/hr). each of the ten dimensions of reuse, the The Cost/Productivity Model by amount of organizational involvementGaffney and Durek [1989] specified the and commitment increases as an organi-effect of reuse on software quality (num- zation progresses from initial/chaotic re-ber of errors) and on software develop- use to ingrained reuse. Ingrained reusement schedules. Results suggested that incorporates fully automated supporttradeoffs can occur between the propor- tools and accurate reuse measurement totion of reuse and the costs of developing track progress.and using reusable components. In a To use this model, an organizationstudy of the latent error content of a will assess its reuse maturity beforesoftware product, the relative error con- beginning a reuse improvement pro-tent decreased for each additional use of gram by identifying its placement onthe software but leveled off between each dimension. (In our experience,three and four uses. The models show most organizations are between Initial/that the number of uses of the reus- Chaotic and Monitored at the start ofable software components directly cor- the program.) The organization willrelates to the development product then use the model to guide activitiesACM Computing Surveys, Vol. 28, No. 2, June 1996
  11. 11. Software Reuse: Metrics and Models • 425 Table 9. Hudson and Koltun Reuse Maturity Modelthat must be performed to achieve set of categorized critical success factorshigher levels of reuse maturity. Once an (stated as goals) that an organizationorganization achieves Ingrained reuse, can use to assess the present state of itsreuse becomes part of the business rou- reuse practice. The factors are orga-tine and will no longer be recognized as nized into four primary groups: man-a distinct discipline. agement, application development, as- set development, and process and3.2 SPC Reuse Capability Model technology factors. For example, in theThe reuse capability model developed by group “Application Development Fac-the Software Productivity Consortium tors/Asset Awareness and Accessibility”[Davis 1993] has two components: an as- is the goal “Developers are aware of andsessment model and an implementation reuse assets that are specifically ac-model. quired/developed for their application.” The assessment model consists of a The implementation model helps pri- ACM Computing Surveys, Vol. 28, No. 2, June 1996
  12. 12. 426 • W. Frakes and C. Terryoritize goals and build successive stages reuse level metrics that include factorsof implementation. The SPC identifies such as abstraction level of the life cyclefour stages in the risk-reduction growth objects and formal definitions of inter-implementation model: nal and external reuse. Work is also underway [Bieman and Karunanithi(1) Opportunistic: The reuse strategy 1993; Chidamber and Kemerer 1994] to is developed on the project level. define metrics specifically for object-ori- Specialized reuse tools are used and ented systems. The following sections reusable assets are identified. discuss this work in detail. To date,(2) Integrated: A standard reuse little work has been done on amount of strategy is defined and integrated reuse metrics for generative reuse, al- into the corporation’s software de- though Biggerstaff [1992] implicitly de- velopment process. The reuse pro- fines such a metric by reporting a ratio gram is fully supported by manage- of generated source lines to specification ment and staff. Reuse assets are effort for the Genesis system—40 categorized. KSLOC for 30 minutes specification(3) Leveraged: The reuse strategy ex- effort. pands over the entire life cycle and is specialized for each product line. Reuse performance is measured and 4.1 Reuse Level weaknesses of the program identified. The basic dependent variable in soft-(4) Anticipating: New business ven- ware reuse improvement efforts is the tures take advantage of the reuse level of reuse [Frakes 1993]. Reuse level capabilities and reusable assets. measurement assumes that a system is High payoff assets are identified. composed of parts at different levels of The reuse technology is driven by abstraction. The levels of abstraction customers’ needs. must be defined to measure reuse. ForThe SPC continues to evolve the model. example, a C-based system is composedIt has been used in pilot applications by of modules (.c files) that contain func-several organizations, but no formal tions, and functions that contain lines ofvalidation has been done. code. The reuse level of a C-based system, then, can be expressed in terms of mod-4. AMOUNT OF REUSE ules, functions, or lines of code. A soft- ware component (lower level item) mayAmount of reuse metrics are used to be internal or external. An internal lowerassess and monitor a reuse improve- level component is one developed for thement effort by tracking percentages of higher level component. An externalreuse of life cycle objects over time. In lower level component is used by thegeneral, the metric is: higher level component, but was created for a different item or for general use. amount of life cycle object reused The following quantities can be calcu- . lated given a higher level item com- total size of life cycle object posed of lower level items:A common form of this metric is basedon lines of code as follows: L ϭ the total number of lower level items in the higher level item.lines of reused code in system or module E ϭ the number of lower level items . total lines of code in system or module from an external repository in the higher level item.Frakes [1990], Terry [1993], and Frakes I ϭ the number of lower level itemsand Terry [1994] extend the basic in the higher level item that are“amount of reuse” metric by defining not from an external repository.ACM Computing Surveys, Vol. 28, No. 2, June 1996
  13. 13. Software Reuse: Metrics and Models • 427M ϭ the number of items not from an T ϭ total number of lower level external repository that are used items in the higher level item, more than once. both internal and external.These counts are of unique items Internal Reuse Level: IU/T(types), not tokens (references). External Reuse Level: EU/T Given these quantities, the followingreuse level metrics are defined [Frakes Total Reuse Level: (IU ϩ EU)/T.1990]: The reuse frequency metric is based onExternal Reuse Level: E/L references (tokens) to reused components rather than counting components onlyInternal Reuse Level: M/L once as was done for reuse level. TheTotal Reuse Level: variables and reuse frequency metrics are: External Reuse Level IUF ϭ number of references in the ϩ Internal Reuse Level. higher level item to reused internal lower level items.Internal, external, and total reuse lev- EUF ϭ number of references in theels will assume values between 0 and 1. higher level item to reusedMore reuse occurs as the reuse level external lower level items.value approaches 1. A reuse level of 0 TF ϭ total number of references toindicates no reuse. lower level items in the higher The user must provide information to level item, both internal andcalculate these reuse measures. The user external.must define the abstraction hierarchy, adefinition of external repositories, and a Internal Reuse Frequency: IUF/TFdefinition of the “uses” relationship. For External Reuse Frquency: EUF/TFeach part in the parts-based approach, wemust know the name of the part, source Total Reuse Frequency:of the part (internal or external), level ofabstraction, and amount of usage. (IUF ϩ EUF)/TF. Terry [1993] extended this formal Program size is often used as a mea-model by adding a variable reuse sure of complexity. The complexitythreshold level that had been arbitrarily weighting for internal reuse is the sumset to one in the original model. The of the sizes of all reused internal lowervariables and reuse level metrics are: level items divided by the sum of the sizes of all internal lower level itemsITL ϭ internal threshold level, the within the higher level item. An exam- maximum number of times an ple size weighting for internal reuse in internal item can be used a C system is the ratio of the size (cal- before it is reused. culated in number of lines of noncom-ETL ϭ external threshold level, the mentary source code) of reused internal maximum number of times an functions to the size of all internal func- external item can be used tions in the system. before it is reused. The software tool rl calculates reuse IU ϭ number of internal lower level level and frequency for C code. Given a items that are used more than set of C files, rl reports the following ITL. information: EU ϭ number of external lower level items that are used more than (1) internal reuse level; ETL. (2) external reuse level; ACM Computing Surveys, Vol. 28, No. 2, June 1996
  14. 14. 428 • W. Frakes and C. Terry(3) total reuse level; aged, with similar definitions to the(4) internal reuse frequency; server perspective.(5) external reuse frequency; Measurable system reuse attributes include:(6) total reuse frequency;(7) complexity (size) weighting for —% of new system source text imported internal functions. from the library;The user may specify an internal and —% of new system classes imported ver-external threshold level. The software batim from the library;allows multiple definitions of higher —% of new system classes derivedlevel and lower level abstractions. The from library classes and the averageallowed higher level abstractions are % of the leveraged classes that aresystem, file, or function. The lower imported;level abstractions are function and —average number of verbatim andNCSL (Non-Commentary Source Lines leveraged clients for servers, andof code). servers for clients; —average number of verbatim and le- veraged indirect clients for servers,4.2 Reuse Metrics for Object-Oriented and indirect servers for clients; Systems —average length and number of pathsBieman [1992] and Bieman and Karu- between indirect servers and clientsnanithi [1993] have proposed reuse met- for verbatim and leveraged reuse.rics for object-oriented systems. Bieman Bieman and Karunanithi [1993] de-[1992] identifies three perspectives from scribe a prototype tool that is underwhich to view reuse: server, client, and development to collect the proposedsystem. The server perspective is the measures from Ada programs. Thisperspective of the library or a particular work recognizes the differences betweenlibrary component. The analysis focuses object-oriented systems and proceduralon how the entity is reused by the cli- systems, and exploits those differencesents. From the client perspective, the through unique measurements.goal is knowing how a particular pro- Chidamber and Kemerer [1994] pro-gram entity reuses other program enti- pose a metrics suite for object-orientedties. The system perspective is a view of design. The paper defines the followingreuse in the overall system, including metrics based on measurement theory:servers and clients. The server reuse profile of a class (1) Weighted methods per class;characterizes how the class is reused by (2) Depth of inheritance tree;the client classes. The verbatim server (3) Number of children;reuse in an object-oriented system is (4) Coupling between object classes;basically the same as in procedural sys- andtems, using object-oriented terminology. (5) Responses for a class.Leveraged server reuse is supportedthrough inheritance. A client can reuse Among these, the most significant to re-the server either by extension, adding use is the metric depth of inheritance tree.methods to the server, or by overload, This metric calculates the length of inher-redefining methods. (Note that McGre- itance hierarchies. Shallow hierarchiesgor and Sykes [1992] offer good defini- forsake reusability for the simplicity oftions of the object-oriented terminology understanding, thus reducing the extentused in this section.) of method reuse within an application. The client reuse profile characterizes Chidamber and Kemerer assert that thishow a new class reuses existing library metrics suite can help manage reuse op-classes. It too can be verbatim or lever- portunities by measuring inheritance.ACM Computing Surveys, Vol. 28, No. 2, June 1996
  15. 15. Software Reuse: Metrics and Models • 4294.3 Reuse Predictions for Life Cycle No Attempt to Reuse Objects Part Does Not ExistFrakes and Fox [1995] present models Part Is Not Availablethat allow the prediction of reuse levels Part Is Not Foundfor one life cycle object based on reuse Part Is Not Understoodlevels for other life cycle objects. Such Part Is Not Validmodels can be used to estimate reuselevels of later life cycle objects such as Part Can Not Be Integratedcode from earlier ones such as require- Each failure mode has failure causesments. The authors define both temporal associated with it. “No Attempt to Re-and nontemporal models, and discuss use,” has among its failure causes, formethods for tailoring the models for a example, resource constraints, no incen-specific organization. The models are tive to reuse, and lack of education. Tobased on reuse data collected from 113 use the model, an organization gathersrespondents from 29 organizations, pri- data on reuse failure modes and causes,marily in the US. The study found that and then uses this information to prior-there were significant correlations be- itize its reuse improvement activities.tween the reuse levels of life cycle objects,and that prediction models improved asdata from more life cycle objects were 6. REUSABILITY ASSESSMENTadded to the models. Another important reuse measurement area concerns the estimation of reus-5. SOFTWARE REUSE FAILURE MODES ability for a component. Such metrics MODEL are potentially useful in two key areasImplementing systematic reuse is diffi- of reuse: reuse design and reengineer-cult, involving both technical and non- ing for reuse. The essential question is,technical factors. Failure modes analysis are there measurable attributes of aprovides an approach to measuring and component that indicate its potentialimproving a reuse process based on a reusability? If so, then these attributesmodel of the ways a reuse process can will be goals for reuse design and re-fail. The reuse failure modes model re- engineering. One of the difficulties inported by Frakes and Fox [1996] can be this area is that attributes of reusabil-used to evaluate the quality of a system- ity are often specific to given types ofatic reuse program, to determine reuse reusable components, and to the lan-impediments in an organization and to guages in which they are implemented.devise an improvement strategy for a sys- In this section we review work in thistematic reuse program. area. Given the many factors that may af- In a study of NASA software, Selbyfect reuse success, how does an organi- [1989] identified several module at-zation decide which ones to address in tributes that distinguished black-boxits reuse improvement program? This reuse modules from others in his sam-question can be answered by finding out ple. The attributes included:why reuse is not taking place in theorganization. This can be done by con- —Fewer module calls per source line;sidering reuse failure modes—that is, —Fewer I/O parameters per source line;the ways that reuse can fail. —Fewer read/write statements per line; The reuse failure modes model hasseven failure modes corresponding to —Higher comment to code ratios;the steps a software engineer will need —More utility function calls per sourceto complete in order to reuse a compo- line;nent. The failure modes are: —Fewer source lines. ACM Computing Surveys, Vol. 28, No. 2, June 1996
  16. 16. 430 • W. Frakes and C. Terry Figure 2. Reuse library system.This data suggest that modules possess- ponents. Potentially reusable softwareing these attributes will be more reus- is identified, and a method to measureable. distances from that ideal is defined. By Basili et al. [1990] reported on two measuring the amount of change neces-reuse studies of systems written in Ada. sary to convert an existing program intoThe first study defined measures of data one composed of maximally reusablebindings to characterize and identify re- components, an indication of the reus-usable components. Data binding is the ability of the program can be obtained.sharing of global data between programelements [Hutchins and Basili 1985]. 7. REUSE LIBRARY METRICSBasili et al. [1990] proposed a method toidentify data bindings within a pro- As can be seen in Figure 2, a reusegram. After identification of the bind- library is a repository for storing reus-ings, a cluster analysis computes and able assets, plus an interface for search-weights coupling strengths. A coupling ing the repository. Library assets can beis based on references to variables and obtained from existing systems throughparameters (data bindings). Aliasing, or reengineering, designed and built fromreferencing, is not taken into account; scratch, or purchased. Components thenonly one level of data bindings is consid- are usually certified, a process for as-ered. The cluster analysis identifies suring that they have desired attributeswhich modules are strongly coupled and such as testing of a certain type.may not be good candidates for reuse, The components are then classified soand which modules are found to be inde- that users can effectively search forpendent of others and are potentially them. The most common classificationreusable. A similar use of module cou- schemes are enumerated, faceted, andpling information to identify potential free text indexing [Frakes and Gandelobjects in C code has been reported by 1990]. The evaluation criteria for index-Dunn and Knight [1991]. ing schemes of reuse libraries are: costs, The second study defines an abstract searching effectiveness, support for un-measurement of reusability of Ada com- derstanding, and efficiency.ACM Computing Surveys, Vol. 28, No. 2, June 1996
  17. 17. Software Reuse: Metrics and Models • 431 Indexing costs include the cost of cre- can be measured by the number of bytesating a classification scheme, maintain- needed to store the collection. Indexinging the classification scheme, and up- overhead ratios can be calculated bydating the database for the scheme. dividing the sum of the size of the rawThese concerns are negligible for a data, and indexing files by the size ofsmall collection, but become more im- the indexing files. Retrieval speed isportant as the collection grows. Each of usually calculated by measuring the timethese costs can be measured in terms of it takes the system to execute a search ofdollars or effort. These costs can be nor- a given query on a given database.malized by dividing total costs by the Quality of assets is another importantnumber of components handled by the aspect of a reuse library. There is con-library. siderable anecdotal evidence that this is Searching effectiveness addresses the most important factor in determin-how well the classification methods help ing successful use of a reuse library.users locate reusable components, and Frakes and Nejmeh [1987] proposed theis usually measured with recall and pre- following metrics as indicators of thecision. Recall is the number of relevant quality of assets in a reuse library.items retrieved divided by the numberof relevant items in the database. The —Time in Use: the module should havedenominator is generally not known and been used in one or more systems thatmust be estimated using sampling have been released to the field for amethods. Precision is the number of rel- period of three months.evant items retrieved divided by the —Reuse Statistics: the extent to whichtotal of items retrieved. Another effec- the module has been successfully re-tiveness measure is overlap, which re- used by others is perhaps the bestports the percentage of relevant docu- indicator of module quality.ments retrieved jointly by two methods.Frakes and Pole [1994], in a study of —Reuse Reviews: favorable reviewsrepresentation methods for reusable from those that have used the modulesoftware, reported no statistically sig- are a good indication that the modulenificant difference in terms of recall and is of higher quality.precision between enumerated, faceted, —Complexity: overly complex modulesfree text keyword, and attribute value may not be easy to modify or main-classification schemes. They reported tain.high overlap measures ranging from .72 —Inspection: the module should haveto .85 for all pairs of the classification been inspected.methods. —Testing: the modules should have To reuse a component, a software en- been thoroughly tested at the unitgineer must not only find it, but also level with statement coverage of 100understand it. Understanding metrics percent and branch coverage of atmeasure how well a classification least 80 percent.method helps users understand reus-able components. Frakes and Pole Another class of reuse library metrics is[1994] used a 7-point ordinal scale (7 ϭ used to measure usage of a reuse librarybest) to measure support for under- system. The following list was suppliedstanding. They reported no significant by the ASSET system, an on-line com-difference between the classification mercial reuse library system.methods using this metric. In order to be useful, a reuse library —Time on-line (system availability):must also be efficient. Library efficiency this is a measure of the number ofdeals with nonfunctional requirements hours the system is available for use.such as memory usage, indexing file —Account demographics: assigns userssize, and retrieval speed. Memory usage to the following categories: Govern- ACM Computing Surveys, Vol. 28, No. 2, June 1996
  18. 18. 432 • W. Frakes and C. Terry Table 10. Summary of Software Reuse Metrics and ModelsACM Computing Surveys, Vol. 28, No. 2, June 1996
  19. 19. Software Reuse: Metrics and Models • 433Table A1. Definitions of Reuse Types ACM Computing Surveys, Vol. 28, No. 2, June 1996
  20. 20. 434 • W. Frakes and C. Terry ment/contractor, commercial, aca- overlap in meaning. For example, the ta- demia, NonUS; ble terms public and external both de-—Type of library function performed: scribe the part of a product that was searches, browses, extracts; constructed externally; private and inter-—Asset distribution: whether electronic nal describe the part of a product that or via a human librarian; was not constructed externally but was developed and reused within a single—Number of subscriber accounts; product. The terms verbatim and black-—Available assets by type: interopera- box both describe reuse without modifica- tion, supplier listings, local compo- tion; leveraged and white-box describe re- nents; use with modification. The final four—Number of extractions by collection: terms in the table describe levels of reuse number of assets extracted by collec- that can occur in the object-oriented par- tion, number of assets extracted by adigm. (References in descriptions in Ta- evaluation level; ble A1 are provided for further informa-—Listing of assets by domain; tion. They do not necessarily indicate first—Request for services by: Telnet Log- use of the term.) ins, modem Logins, FTP, World Wide Web. REFERENCES These metrics can provide good man-agement information for a library sys- AGRESTI, W. AND EVANCO, W. 1992. Projectingtem. They can be used to demonstrate software defects in analyzing Ada designs. IEEE Trans. Softw. Eng. 18, 11, 988 –997.the value of the library to management BARNES, B. ET AL. 1988. A framework and eco-as well as to provide information for nomic foundation for software reuse. In IEEEcontinuous quality improvement. Tutorial: Software Reuse—Emerging Technol- ogy, W. Tracz, Ed. IEEE Computer Society8. SUMMARY Press, Washiington, D.C. BARNES, B. AND BOLLINGER, T. 1991. MakingA reuse program must be planned, delib- software reuse cost effective. IEEE Softw. 1,erate, and systematic if it is to give large 13–24.payoffs. As organizations implement sys- BASILI, V. R., ROMBACH, H. D., BAILEY, J., ANDtematic software reuse programs in an DELIS, A. 1990. Ada reusability and mea- surement. Computer Science Tech. Rep. Se-effort to improve productivity and qual- ries, University of Maryland, May.ity, they must be able to measure their BIEMAN, J. 1992. Deriving measures of soft-progress and identify the most effective ware reuse in object-oriented systems. Inreuse strategies. In this article we sur- BCS-FACS Workshop on Formal Aspects ofveyed metrics and models of software re- Measurement, Springer-Verlag.use. A metric is a quantitative indicator BIEMAN, J. AND KARUNANITHI, S. 1993. Candi date reuse metrics for object-oriented and Adaof an attribute. A model specifies relation- software. In Proceedings of IEEE-CS Firstships between metrics. International Software Metrics Symposium. Table 10 presents a summary of the BIGGERSTAFF, T. 1992. An assessment and anal-reuse metrics and models discussed. As ysis of software reuse. In Advances in Com-is typical in an emerging discipline such puters, Marshall Yovits, Ed. Academic Press,as systematic software reuse, many of New York.these metrics and models still lack for- BOOCH, G. 1987. Software Componenets with Ada. Benjamin/Cummings, Menlo Park, CA.mal validation. Despite this, they are BROWNE, J., LEE, T., AND WERTH, J. 1990. Ex-being used and are being found useful perimental evaluation of a reusability-ori-in industrial practice. ented parallel programming environment. IEEE Trans. Softw. Eng. 16, 2, 111–120.Appendix 1: Definitions of Types of Reuse CARD, D., MCGARRY, F., PAGE, G., ET AL. 1982. The software engineering laboratory. NASA/The terms in Table A1 describe various GSFC, 2.reuse issues. Some terms in the table CARD, D., CHURCH, V., AND AGRESTI, W. 1986.ACM Computing Surveys, Vol. 28, No. 2, June 1996
  21. 21. Software Reuse: Metrics and Models • 435 An empirical study of software design practices. FRAKES, W. AND POLE, T. 1994. An empirical IEEE Trans. Softw. Eng. 12, 2, 264 –270. study of representation methods for reusableCHEN, D. AND LEE, P. 1993. On the study of software components. IEEE Trans. Softw. software reuse: using reusable C ϩ ϩ compo- Eng. 20, 8, 617– 630. nents. J. Syst. Softw. 20, 1, 19 –36. FRAKES, W. AND TERRY, C. 1994. Reuse levelCHIDAMBER, S. AND KEMERER, C. 1994. A met- metrics. In Proceedings of the Third Interna- rics suite for object-oriented design. IEEE tional Conference on Software Reuse (Rio de Trans. Softw. Eng. 20, 6, 476 – 493. Janeiro), W. Frakes, Ed., IEEE Computer Sci-DAVIS, T. 1993. The reuse capability model: a ence Press, Los Alamitos, CA, 139 –148. basis for improving an organization’s reuse GAFFNEY, J. E. AND DUREK, T. A. 1989. Soft- capability. In Proceedings of the Second Inter- ware reuse— key to enhanced productivity: national Workshop on Software Reusability some quantitative models. Inf. Softw. Tech- (Herndon, VA). nol. 31, 5, 258 –267.DUNN, M. F. AND KNIGHT, J. C. 1991. Software HUMPHREY, W. 1989. Managing the Software reuse in an industrial setting: A case study. Process. Addison-Wesley, Reading, MA. In Proceedings of the Thirteenth International Conference on Software Engineering, IEEE HUTCHINS, D. H. AND BASILI, V. 1985. System Computer Society Press, Austin, TX, 329 – structure analysis: Clustering with data bind- 338. ings. IEEE Trans. Softw. Eng. 11, 8, 749 –757.FAVARO, J. 1991. What price reusability? A case JONES, C. 1993. Software return on investment study. Ada Lett. (Spring), 115–124. preliminary analysis. Software ProductivityFENTON, N. 1991. Software Metrics, A Rigorous Research, Inc. Approach. Chapman & Hall, London. KOLTUN, P. AND HUDSON, A. 1991. A reuse ma-FRAKES, W. 1993. Software reuse as industrial turity model. In Fourth Annual Workshop on experiment. Am. Program. (Sept.), 27–33. Software Reuse (Herndon, VA).FRAKES, W. (Moderator). 1991. Software reuse: LILLIE. 1995. Personal communication. is it delivering? In Proceedings of the Thir- MARGONO, T. AND RHOADS, T. 1993. Software teenth International Conference on Software reuse economics: cost-benefit analysis on a Engineering (Los Alamitos, CA). IEEE Com- large-scale Ada project. In International Con- puter Society Press, Los Alamitos, CA. ference on Software Engineering ACM, NewFRAKES, W. 1990. An empirical framework for York. software reuse research. In Proceedings of the MCGREGOR, J. AND SYKES, D. 1992. Object-Ori- Third Workshop on Tools and Methods for ented Software Development: Engineering Reuse (Syracuse, NY). Software for Reuse. Van Nostrand Reinhold,FRAKES, W. AND FOX, C. 1995. Modeling reuse New York. across the software lifecycle. J. Syst. Softw. OGUSH, M. 1992. A software reuse lexicon. 30, 3, 295–301. Crosstalk (Dec.).FRAKES, W. AND FOX. C. 1996. Quality improve- POULIN, J. S., CARUSO, J. M., AND HANCOCK, ment using a software reuse failure modes D. R. 1993. The business case for software model. IEEE Trans. Softw. Eng. 24, 4 (April), 274 –279. reuse. IBM Syst. J. 32, 4, 567–594.FRAKES, W. AND GANDEL, P. 1990. Representing PRIETO-DIAZ, R. 1993. Status report: Software reusable software. Inf. Softw. Technol. 32, 10, reusability. IEEE Softw. (May), 61– 66. 653– 664. SELBY, R. W. 1989. Quantitative studies of soft-FRAKES, W. AND ISODA, S. 1994. Success factors ware reuse. In Software Reusability, Volume of systematic reuse. IEEE Softw. 11, 5, 14 –19. II, T. J. Biggerstaff and A. J. Perlis, Eds.,FRAKES, W. B. AND NEJMEH, B. A. 1987. Addison-Wesley, Reading, MA. Software reuse through information retrieval. TERRY, C. 1993. Analysis and implementation In Proceedings of the Twentieth Annual Ha- of software reuse measurement. Virginia waii International Conference on Systems Sci- Polytechnic Institute and State University, ences. Kona, Jan., 530 –535. Master’s Project and Report.Received April 1994; revised October 1995; accepted November 1995 ACM Computing Surveys, Vol. 28, No. 2, June 1996
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×