Software Project Management Tools Draft 5.0 - July 18, 2008 ...
Upcoming SlideShare
Loading in...5
×
 

Software Project Management Tools Draft 5.0 - July 18, 2008 ...

on

  • 1,172 views

 

Statistics

Views

Total Views
1,172
Views on SlideShare
1,172
Embed Views
0

Actions

Likes
1
Downloads
28
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Software Project Management Tools Draft 5.0 - July 18, 2008 ... Software Project Management Tools Draft 5.0 - July 18, 2008 ... Document Transcript

  • Software Project Management Tools Draft 5.0 - July 18, 2008 Capers Jones Keywords: Software project management, software sizing, software cost estimating, software schedule planning, software quality estimating, software risk analysis, software progress tracking, software litigation. Abstract Software projects have been notoriously difficult to manage well. Cost overruns, schedule delays, and outright cancellations of software projects have been common occurrences for more than 50 years. Poor quality when delivered remains a chronic issue. Although software project management tools are not a panacea, when used by capable managers they do simplify the complexity of software project management and raise the odds of successful completions of large software projects. Software project management tools have many divisions and subcategories. However they can be broadly classified into two major areas of focus: 1) Tools for planning and estimating software projects before they begin; 2) Tools for tracking and reporting software project status and costs while projects are underway. Beneath this division are 12 topical areas and scores of project management activities. Capers Jones Chief Scientist Emeritus, Software Productivity Research LLC Web: http://www.spr.com Email: cjonesiii@cs.com Copyright © 2008 by Capers Jones. All rights reserved. 1
  • Software Project Management Tools The Tasks of Software Project Managers This article discusses the tasks of project management and the available tools that support those tasks. From the frequent failure of large software projects and from analysis of the depositions and testimony of litigation for software failures, it is apparent that software project managers are often not well trained in managerial tasks, and often fail to utilize appropriate tools to support critical tasks such as estimating and progress tracking. Before discussing the tools of software project management, it is appropriate to consider the many tasks that software project mangers are likely to confront in carrying out their jobs. The two major divisions of software project management include planning and estimating before projects start, and tracking status and expenses while software projects are underway. Beneath this broad division there are a dozen major functional areas and scores of individual tasks, as shown below: 1) Software Project Sizing a. Size in function points b. Size in lines of code c. Size in data base contents d. Numbers of screens e. Numbers of reports f. Numbers of interfaces g. Numbers of files h. Prediction of requirements creep i. Prediction of requirements churn 2) Software Project Schedule Planning a. Project schedules b. Activity schedules 2
  • c. Task schedules d. Critical paths e. Team assignments f. Individual assignments g. Requirements creep h. Requirements churn i. Requirements slip 3) Software Project Cost Estimating a. Estimating method selection b. Estimating tool selection c. Project cost estimating d. Activity cost estimating e. Task cost estimating f. Project earned value calculation g. Creeping requirements cost prediction h. Churned requirements cost prediction i. Slipped requirements cost prediction j. Complexity estimating 4) Software Project Risk Estimating a. Schedule delays b. Cost overruns c. Cancellation d. Poor quality e. Poor customer satisfaction 3
  • f. Litigation for breach of contract g. Loss of key personnel h. Potential security problems 5) Software Project Quality Estimating a. Defect potentials b. Defect removal efficiency c. Bad fix injections d. Error-prone modules e. Bad test cases f. Cyclomatic and essential complexity g. Number of defect removal activities h. Number of test stages i. Number of test cases per stage 6) Software Project Outsource Contract Analysis a. Requirements scope in contract b. Requirements changes in contract c. Quality terms in contract d. Incentives for early completion e. Penalties for late completion f. Penalties for poor quality g. Required tracking and reporting 7) Software Project Commercial Off-the-Shelf (COTS) Acquisition a. COTS Acquisition Analysis b. Due diligence prior to major acquisitions 4
  • c. Warranty provisions d. Installation schedules e. Installation defects f. Learning curves g. Loss of performance during learning 8) Software Personnel Management a. Specialist selection b. Hiring of new employees c. Appraisals of existing employees d. Promotions e. Demotions f. Terminations g. Awards 9) Software Project Measurement, Tracking, and Control a. Financial tracking b. Earned value tracking c. Balanced score card tracking d. Goal-question metric tracking e. Milestone tracking f. Issue and “red flag” tracking g. Quality and defect tracking h. Change request tracking i. Requirements creep tracking j. Requirements churn tracking 5
  • k. Complexity measurement l. Benchmark productivity data m. Benchmark quality data n. Coordination with other managers o. Coordination with supply chain managers 10) Software Technology Acquisition a. Project planning tools b. Project estimating tools c. Project tracking tools d. Defect tracking tools e. Complexity analysis tools f. Development methodologies g. Maintenance and enhancement methodologies h. Quality control methodologies i. Methodology management tools j. Development workbenches k. Maintenance workbenches l. Reverse engineering tools m. Reengineering tools n. Programming languages 11) Software Mergers and Acquisitions a. Due diligence for purchaser b. Analysis of tools in company to be acquired c. Technology conversion after merger 6
  • d. Organization planning e. Reorganization after mergers 12) Software Litigation Support a. Breach of contract litigation b. Intellectual property and patent litigation c. Employee confidentiality litigation d. Preparation of materials on plaintiff side e. Preparation of materials on defendant side f. Depositions if required g. Testimony if required As of 2008 automated tools are available for many but not all of the tasks that software project managers are likely to perform. For common tasks such as schedule planning and cost estimating there are dozens of available tools. For less common tasks such as litigation support or due diligence prior to a merger or acquisition, few tools are available. Project Management Knowledge Acquisition Academic training in the area of software project management is not as sophisticated as academic training in basic software engineering or even in other forms of engineering management. For example there is a shortage of academic training even for important topics such as quality estimation, risk analysis, and milestone tracking. For more unusual topics, such as litigation support or commercial off the shelf software (COTS) acquisition academic training hardly exists at all. The lack of academic training for software project managers is compensated for in some degree by training and “bodies of knowledge” or BOK provided by a number of professional associations. Among the large associations that provide useful information to software project managers can be found: Academy of Management (AOM) www.aomonline.org American Management Association (AMA) www.amanet.org American Society for Quality (ASQ) www.asq.org IEEE Computer Society www.computer.org 7
  • Information Technology Metrics and Productivity Institute (ITMPI) www.itmpi.org International Function Point Users Group (IFPUG) www.ifpug.org International Project Management Association (IPMA) www.ipma.org International Organization for Standards (ISO) www.iso.org Project Management Association of Japan www.pmaj.or.jp International Software Benchmark Standards Group (ISBSG) www.isbsg.org Project Management Institute (PMI) www.pmi.org Software Engineering Institute (SEI) www.sei.org Society of Project Management (SPM) www.spm-japan.jp There are also scores of national and local organizations that also provide information and seminars on software project management topics. Examples include the Australian Institute of Project Management & Australian Software Metrics Association & Software Quality Assurance (ASMA-SQA), the Brazilian Association for Project Management, the Netherlands Software Metrics Association (NESMA), and dozens of local chapters of the Software Process Improvement Network (SPIN) in the United States, Europe, South America, and Asia. Several web sites contain useful links to the project management literature. Among the commercial links can be found the previously mentioned Information Technology Metrics and Productivity Institute ITMPI) portal (www.ITMPI.org). Among academic links one of the most complete is that of Dave W. Farthing of the University of Glamorgan in the United Kingdom (www.comp.glam.ac.uk). This interesting portal has links to dozens of project management sites and the publishers of scores of project management books. From discussions with project managers in large corporations, in-house training of managers in companies such as IBM, Microsoft, AT&T, Google, and CISCO is actually the most effective source of information for software project managers. Topics such as quality estimation, measurement, and risk analysis are often better covered via in-house training than by universities. However this form of training is only available for project managers in rather large corporations. Small companies must depend upon commercial training or upon professional associations, plus a few universities that do have curricula in software management topics. Training in specific project management tools is also available from the vendors themselves. For example fairly sophisticated training in software cost estimating is available from the vendors of 8
  • commercial estimating tools such as KnowledgePlan, Price-S, SLIM, and SEER. Computer Aid Inc. and its subsidiary the Information Systems Metrics and Productivity Institute (ITMPI) also provide training in a number of software management topics. Overall, training of software project managers is a topic that needs to be improved. Quality control, economic analysis, estimation, and measurement are all topics where additional training is needed for entry-level managers just starting their careers. The History of Software Project Management Tools Software project management is of course only one of many different forms of project management, such as home construction projects, electrical engineering projects, or civil engineering projects such as building roads and bridges. Software project management does have some distinguishing attributes that seem to differ from other and more established forms of project management: 1. Because software is abstract and invisible to the eye, sizing software projects is more difficult than for many other kinds of project. 2. Because software engineering is not highly automated, manual effort by skilled knowledge workers plays a more important role in software projects than in many other kinds of project. 3. Because of the large volume of bugs or defects in all software deliverables, quality estimation and quality control are more important for software projects than for many other kinds of project. 4. For large software projects in the 10,000 function point range or larger, outright failure, major cost overruns, and major schedule delays occur more frequently than for many other kinds of project. 5. Although changing requirements can occur for any kind of project, they occur with unusual frequency for software projects. Sometimes more than 50% of the delivered features of software projects are not identified until the project is already in process. Software requirements changes have been measured to occur at a rate of between 1% and 2% per calendar month. Long before the computer era and long before software itself, project management was starting to be studied as a technology that needed formalization. Among the pioneers who dealt with project management issues include Henry Gantt whose famous Gantt charts circa 1917 facilitated schedule planning. Later in the 1950’s Willard Fazar’s “program evaluation and review technique” (PERT) added rigor to isolating and identifying the critical paths which complex projects needed to follow. On the quality side, W. Edwards Deming revolutionized the statistical 9
  • analysis of quality. Deming’s work was initially adopted in Japan circa 1950 and later came to the United States when it became obvious that quality was a major factor in successful competition. Other pioneers in related topics include Walter Shewart (one of Deming’s teachers), Phil Crosby of ITT, Joseph Juran, and Frederick Taylor. In terms of software project management tools as opposed to general-purpose project management tools, some of the pioneers include A.J. Albrecht and the invention of function point metrics circa 1973; Dr. Barry Boehm and the development of the famous “constructive cost model” or COCOMO circa 1981; Watts Humphrey and the development of the capability maturity model (CMM) circa 1983; and Peter Hill and the foundation of the International Software Benchmarking Standards Group (ISBSG) circa 1997. A more recent pioneer is Tony Salvaggio of Computer Aid, Inc. This company is pioneering a flexible form of project management tool that can add “cartridges” at will for specific project management functions. The phrase “project management tools” has been applied to a large family of tools whose primary purpose is sophisticated scheduling for projects with hundreds or even thousands of overlapping and partially interdependent tasks. These tools are able to drop down to very detailed task levels, and can even handle the schedules of individual workers. However for software, there are many other critical areas that also need the support of tools: quality prediction and measurement; requirements changes, and the earned-value of ongoing projects. Software projects require additional capabilities to be under full management control. There are a host of managerial functions that standard project management tools don’t deal with in depth, such as sizing with function point metrics, contract management, personnel management, assessments, quality estimating, defect tracking, and the like. The family of project management tools originated in the 1950’s with the U.S. Navy and Air Force. Weapons systems and other complex projects were becoming so large that manual planning and control were no longer feasible. Since computers were starting to become more powerful, it was natural to want to use computer power as an aid for project managers. Project management tools are an automated form of several management aids developed by the Navy for controlling large and complex weapons systems: the “program evaluation and review technique” (PERT), critical path analysis, resource leveling, and the classic Gantt charts. Such general-purpose tools can of course be used for software projects, but in addition specialized tools are needed that support the unique requirements of software projects. Project management tools did not originate for software, but rather for handling very complex scheduling situations where hundreds or even thousands of tasks need to be determined and sequenced, and where dependencies such as the completion of a task might affect the start of subsequent tasks. Project management tools are general-purpose in nature, and can be applied to any kind of project: building an aircraft, constructing an office building, or a software project. 10
  • General-purpose project management tools have no built-in expertise regarding software, as do specialized software cost estimating tools. For example, if you wish to explore the quality and cost impact of an object-oriented programming language such as Objective C a standard project management tool is not the right choice. By contrast, many software cost estimating tools have built-in tables of programming languages and will automatically adjust the estimate based on which language is selected for the application. Since software cost estimating tools originated about 10 years after commercial project management tools, the developers of software cost estimating tools seldom tried to replicate project management functions such as construction of detailed PERT diagrams or critical path analysis. Instead, the cost estimation tools would export data so that the final schedule could be adjusted via the project management tool. Thus bi-directional interfaces between software cost estimating tools and generic project management tools are now standard features in the commercial estimation market. The latest versions of software project management tools integrate cost and schedule estimation, quality estimation, risk analysis, and many other features. The Computer Aid “application insight” tool is an example of a general-purpose tool with the ability to be extended in many directions via plug-in “cartridges.” Usage Patterns of Software Project Management Tools Observations made from depositions and testimony during software litigation for breach of contract reveals that software projects that end up in court for outright failure or major overruns usually do not utilize many project management tools. Estimates were often informal and tracking seriously deficient. By contrast, data gathered while performing assessments and benchmarks of successful software projects that combined high productivity and high quality indicate that many specialized project management tools were utilized for both predictions before the projects started and for tracking and measurement while they were underway. Table 1 shows the approximate sizes of project management tool features measured in function points. The table also illustrates the patterns of tool usage noted on “lagging,” “average,” and “leading” projects. Information on the lagging projects is taken from information discovered during litigation. Information on the average and leading project is taken from benchmark and assessment studies by the author. 11
  • Table 1: Patterns of Project Management Tool Usage (Tool size expressed in terms of function point metrics) Predictive Tools Lagging Average Leading Project sizing (function points) 750 750 Project sizing (lines of code) 250 250 250 Project sizing (requirements changes) 250 Project schedule estimating 1,000 1,500 3,000 Project cost estimating 250 500 3,000 Project quality estimating 2,000 Project risk estimating 2,000 Project value estimating 1,500 Departmental budget planning 2,000 2,000 2,000 SUBTOTAL 3,500 5,000 14,750 Measurement and Tracking Tools Lagging Average Leading Earned Value Tracking 2,000 3,000 Project cost tracking 2,000 2,000 2,000 Project milestone tracking 1,000 1,500 Project defect tracking 1,500 2,000 Resource tracking 1,000 1,000 2,000 Cost variance reporting 1,000 2,000 2,000 Productivity measurements 1,000 12
  • Project benchmark reporting 1,000 Project quality measurement 1,000 SUBTOTAL 4,000 9,500 15,500 TOTAL 7,500 14,500 30,250 Table 1 illustrates two important aspects of software project management tool usage: 1) Leading or sophisticated project managers use more kinds of tools than do managers on lagging and average projects; 2) Leading or sophisticated project managers use more of the features or select more powerful tools than do project managers on lagging or average projects. The very significant use of project management tools on leading projects results in one overwhelming advantage: “No surprises.” The number of on-time projects in the leading set is far greater than in the lagging set, and all measurement attributes (quality, schedules, productivity, etc.) are also significantly better. Software project management tools are far from being a panacea, but advanced knowledge of software project size, potential numbers of defects or bugs, and schedules derived from similar historical projects can go a long way towards yielding a successful conclusion and eliminating the rather high odds of failure associated with large software projects. For large software projects in large corporations, the project will typically be supported by a “Project Office.” IBM was a pioneer in the utilization of project offices and first deployed them on versions of the main IBM operating system in the early 1970’s. Typically a project office will be formed for a major software application in the 100,000 function point size range. Applications of this large size have hundreds of development personnel and dozens of subordinate project managers under at least three higher levels of management. A typical project office for such a large system would contain perhaps half a dozen staff to as many as 10 personnel equipped with a variety of estimating and planning tools, plus historical data from similar projects. The project office would assist the individual project managers in cost estimating, quality estimating, schedule estimating and other predictive tasks. They would also monitor rates of progress, bug reports, earned value, costs, and other status data as the project was underway. Project offices tend to raise the percentage of successful large projects, but failures can still occur. For example one major cancelled project occurred primarily because the head of the project office and the head of the development group were political rivals and therefore did not cooperate well. Incidentally, in large companies and large projects, personal disputes and dislikes are frequent contributors to software disasters. This topic is difficult to study because none of the managers involved are likely to be candid. 13
  • Recent Evolution of Software Project Management Tools Project management tools, like other forms of commercial software, continue to evolve and add new features. The following paragraphs summarize some of the recent observations of project management tool capabilities noted in 2008. Software Project Sizing: Sizing software projects has been a difficult activity for project managers. Sizing in terms of “lines of code” primarily involved either comparison to similar projects or guess work. Adding to the difficulty, there are more than 700 programming languages in existence and some of these lack any rules for counting code. Further many software applications utilize several programming languages at the same time. More important, for large software projects the costs and schedules associated with producing paper documents and finding and repairing defects costs more than the code itself and takes more time during the schedule. These non-coding tasks cannot be estimated or measured using lines of code metrics. Function point metrics have proven to be effective for sizing non-coding work and also programming, so function points are a good choice for life-cycle economic analysis of software projects. Sizing in terms of function point metrics can be done with accuracy from requirements, but by the time requirements are available several iterations of cost estimates are normally required. Also, function point analysis has been somewhat slow and expensive. Certified function point analysts count function points only at a rate of perhaps 400 function points per day. Thus the long schedules and high costs of counting function point metrics have been a barrier As of 2008 several new forms of high-speed function point tools have been announced. Relativity Technologies has announced a tool for sizing legacy applications called “Function Point Analyzer” or FPA by parsing source code in selected languages. The assertion is that this new tool provides counts with an accuracy equivalent to normal function point analysis, but is able to accomplish this in a matter of minutes rather than a matter of days or weeks. Several other forms of high-speed function point sizing are also under development. Other high- speed function point methods include: 1) pattern matching against historical projects; 2) Automated prediction of function points from requirements and specifications; 3) data mining of legacy source code to recreate synthetic specifications; 4) automated source code analysis to improve mathematical conversion from source code statements to equivalent function point metrics. The last method, of mathematical conversion between source code and function points has actually existed since 1975. The common name for this method is “backfiring.” The first ratios of source code to function points were developed by A.J. Albrecht and his colleagues at IBM during the development of the original function point metric. As of 2008 published ratios are available for more than 700 programming languages and dialects. However the accuracy of 14
  • backfiring is low due to variations in code counting methods and variations in individual programming styles. The backfire method is most effective when based on automated counts of logical source code statements, as opposed to counts of physical lines. A problem with function point sizing as of 2008 is the fact that there are several different kinds of function point metrics. The function point metric defined by the International Function Point Users Group (IFPUG) is the most widely used. But there are also COSMIC function points, Mark II function Points, web-object points, Story points, and several national variants from the Netherlands and Australia, plus usage of mathematical conversion from lines of code to function points called “backfiring.” All of these variants produce different size predications, and as of 2008 there are no proven conversion rules from one variation to another. Another new sizing topic getting underway in 2008 deals with the impact of technical and architectural factors on application size. While user-requested functions are measured with function point metrics, part of the size of applications is also due to architectural and technical factors and the “layers” that the application will contain. Other sizing topics are waiting more research and more empirical data: sizing the volume data bases and sizing value other than financial value (i.e. customer satisfaction; enterprise prestige) are two topics that need more research. Software Project Schedule Planning: There are dozens of commercial tools, in-house tools, and even open-source freeware tools that can perform critical path analysis of software projects, as well as other kinds of projects. Examples of such tools include Microsoft Project, Artemis Views, and TimeLine. However for software projects schedule planning can also be performed by specialized software estimating tools such as COCOMO, KnowledgePlan, Price-S, SEER, and SLIM in alphabetical order. There are even newer forms of tools about to be released circa 2008 that can handle schedules. For example the Application Insight tool (AI) by Computer Aid Inc. integrates schedule planning, risk analysis, cost estimates, and tracking of resources, defects, and costs. This tool also supports interfaces with other tools, and even allows plugging in “cartridges” of specialized functions such as quality prediction or function point analysis. Overall a detailed work-breakdown structure that reaches the level of specific tasks and individual assignments is the most effective. This level of granularity can only be provided by automated tools: manual methods are too labor-intensive and also to difficult to update. As of 2008, schedule estimating remains a difficult challenge for software project managers. Above 10,000 function points in size, about two thirds of all software projects run late, exceed their budgets, or are cancelled because the delays and overruns have degraded the return on investment to a negative number. 15
  • Software Cost Estimating: Software cost estimation can be traced back to the first IBM software cost estimating tool in 1973 and also to the first commercial estimating tool which was the Price-S tool originally marketed by the RCA corporation. These pioneering tools were soon followed by other commercial software cost estimating tools such as Checkpoint, COCOMO, Estimacs, ISBSG, SEER, SLIM, SoftCost, SPQR/20 and dozens of others. These specialized software cost estimating tools were often superior to manual cost estimates, and they also included schedule and quality estimates as well. Because the software estimating tools appeared about 10 years after general-purpose project management tools, they support bi- directional data transfer with general purpose tools such as Microsoft Project or Artemis Views. Another advantage of the commercial software cost estimating tools is that they contain built-in knowledge of topics that affect software projects, which are lacking in general-purpose tools. Examples of these factors include the specific programming languages to be used, the CMM level of the development organization, the methodology to be used such as Agile development or Extreme programming, and the structure or complexity of the application. The commercial software estimating tools also have built-in knowledge of key topics such as the defect removal efficiency levels of various kinds of inspection and testing. The first commercial estimating tool that could predict defect quantities and defect removal efficiency levels was SPQR/20, which entered the market in 1985. This was also the first commercial estimating tool based on function point metrics, although it predicted lines of code as well. Large software projects often spend more money on the production of paper documents such as requirements, specifications, plans, user manuals, etc. than they do on the production of the source code itself. Estimating these costs lagged for many years because lines of code metrics could neither measure nor predict paperwork. However modern commercial software estimating tools can predict the sizes and costs for more than 50 kinds of paper document, which collectively can accumulate more than 35% of the total development cost of a large software project. A few software cost estimating tools can even predict the costs of Nationalization, or converting some documents into foreign languages for international sales. Software Project Risk Estimating: The major forms of risk for software projects include outright cancellation due to failure to reach operational stability, cost overruns, schedule overruns, poor quality, security vulnerabilities, difficult human interfaces, and sluggish performance. Since software project failures are common, it can be stated that software risk assessment and risk predictions are either inadequate or not performed with rigor. As of 2008 there are some risk prediction features in commercial estimating tools such as KnowledgePlan, SLIM, and SEER. There are also some stand-alone risk models as well as risk estimation in integrated tools such as the Application Insight tool (AI) by Computer Aid Inc.. Automated risk estimation is becoming very sophisticated circa 2008, and is also very quick. 16
  • Currently risk analysis by trained risk experts was the traditional and most effective form of risk abatement through 2008, but automated risk analysis is now about equal in effectiveness and much faster in terms of speed.. While risks associated with cost overruns and schedule delays can be predicted early and with good accuracy, risks associated with security flaws are more difficult. In part this is due to the fact that security vulnerabilities are not completely understood circa 2008, coupled with the fact that hackers are constantly adding new forms of viruses and new forms of spyware. Another aspect of risk management is that of “governance” or ensuring the integrity of software applications from hacking, security flaws, and possible misuse for fraudulent purposes. The Sarbanes-Oxley act of 2002 has elevated governance to a very important activity, because it threatens harsh penalties including criminal charges for software executives and boards of directors who are lax in governance. Make no mistake about it, large software projects in the 10,000 function point size range are among the most risky business endeavors in human history. The major risks for large software projects include: 1. Outright cancellation due to excessive cost and schedule overruns 2. Inadequate or careless estimating 3. Failure to compare estimates to validated historical data 4. Cost overruns in excess of 50% compared to initial estimates 5. Schedule overruns in excess of 12 months compared to initial estimates 6. Quality control so inept that the software does not work effectively 7. Requirements changes in excess of 2% per calendar month 8. Executive or client interference, so that the project is disrupted 9. Failure of clients to review requirements and plans effectively 10. Security flaws and vulnerabilities 11. Potential misuse or tampering with outputs for fraudulent purposes 12. Performance or speed too slow to be effective 13. Loss of key personnel from the project during development 14. The presence of “error-prone modules” in legacy applications 15. Patent violations or theft of intellectual property 16. External risks (fire, earthquakes, hurricanes, etc.) From analysis of depositions and court documents in breach of contract litigation, most failing projects did not even perform a formal risk analysis. Worse, project tracking was so inept that major problems were concealed rather than being dealt with as they occurred. Research and prototypes of risk analysis tools are a sign that future kinds of risk analysis may improve the situation. In particular, security vulnerabilities need better predictive capabilities in the modern world where viruses, spyware, and denial of service attacks are daily events that 17
  • affect thousands of companies and millions of software applications. As outsourcing becomes more and more common, litigation avoidance is also a risk that needs more attention than it has received to date. Software Quality Estimating: Quality estimation for software was not well done before the invention of function point metrics because lines of code could not be used for either measuring or predicting defects in requirements, design, user documents and other non-code items. Once function point metrics were applied to the task of quality prediction, it was soon discovered that errors in requirements and design outnumbered coding errors. Commercial software estimating tools that could predict the full spectrum of software defects from all major sources (requirements, design, code, user documents, and “bad fixes” or secondary defects) only became available in 1985 with the SPQR/20 commercial estimation tool. As of 2008 most of the commercial software estimation tools can also predict quantities of defects. Another important aspect of quality estimation is that of defect removal efficiency, or predicting the percentages of defects that will be found by various kinds of review, inspection, and test stage. Historical data gathered since the 1970’s indicate that formal design and code inspections have the highest defect removal efficiency and alone can top 85%. Most forms of testing such as unit test, new function test, regression test, etc. are less than 35% efficient or only find about one bug out of three. A sequence of formal inspections followed by six to eight different testing stages can top 95% in cumulative defect removal efficiency. As of 2008 several commercial estimating tools can predict defect potentials and defect removal efficiency levels. Some commercial estimating tools can also predict the specific sequence of inspections and test stages that should be used to achieve optimal levels of defect removal efficiency. Numbers of test cases and test coverage can also be predicted for many forms of testing such as unit test (manual or automated), regression test, new function test, performance test, system test, etc. Since most forms of testing are less than 40% efficient in finding bugs, it is obvious that a multi- stage series of inspections and tests will be needed to achieve high levels of defect removal efficiency. The good news about this fact is that projects that approximate 95% in cumulative defect removal efficiency have shorter schedules and lower costs than “average” projects with only about 85% cumulative defect removal efficiency. Maintenance costs are also reduced. Phil Crosby’s aphorism that “Quality is Free” has not only been proven, but also measured on many successful software projects. An automated testing company, Agitar, has developed a testing dashboard that provides interesting data on test results, test coverage, and other relevant topics. Because the costs and schedules associated with finding and fixing bugs is the most significant cost element of large software projects and the primary factor leading to cancellation and delay, quality prediction is a critical element of successful cost and schedule prediction. Poor quality is a frequent cause of litigation too. 18
  • Many specific software quality methods such as six-sigma for software, quality function deployment (QFD), formal design and code inspections, automated testing, error-prone module removal, and joint application design (JAD) can now be modeled and predicted using automated tools. The most interesting results come from side-by-side comparisons that show a specific approach such as six-sigma compared against an identical project that did not use the same approach. Software Project Outsource Contract Analysis: Since many outsource contracts are cancelled and about 5% end up in court for breach of contract, it can be stated that the available tools for predicting the results of software outsource contracts need improvement. There are some specialized and proprietary tools available from consulting companies that assist clients in dealing with outsource vendors. Also, normal commercial software cost estimating tools can be useful in predicting topics such as schedules, costs, and quality levels for specific projects. However contract analysis as of 2008 does not have much in the way of sophisticated project management tools. What often occurs in litigation involving cancelled projects are side-by-side comparisons of the project as it was actually performed against an alternate versions that would have used better methods of quality control and dealing with requirement changes. This feature of producing side-by-side comparisons of alternate approaches should become a standard part of predictive tools uses for cost, quality, and schedule planning purposes. Software contracts often are incomplete or ambiguous in dealing with requirements changes and also with quality control. All contracts need language and sections that define how changes in requirements will be funded and what kind of schedule relief will be provided. Contracts also should deal specifically with quality, such as including clauses that require at least 95% defect removal efficiency levels on the part of the developing company. Software Project Commercial Off-the-Shelf (COTS) Acquisition: Most corporations and government agencies lease or purchase more than half of the software they use on a daily basis. Operating systems, office suites, enterprise resource planning (ERP) packages, financial and accounting packages, human resource packages, and a host of others comprise a large portion of total corporate portfolios. There are very few tools available that can assist in making good selections of commercial software. Software vendors are reluctant to publish data on defects in their commercial packages. They are also reluctant to publish size data in either function point or lines of code form. As a result, it is hard to ascertain basic information such as after-market defect levels, installation problems, and other issue. There are some specialized tools available for specific tasks such as estimating the deployment of ERP packages, but these are usually proprietary tools owned by consulting 19
  • companies that specialize in ERP deployment. Overall, this is a major gap in project management tools circa 2008. Historical data on the numbers of bugs and installation problems in major commercial software packages such as Microsoft Vista, of the ERP packages from Oracle, SAP, and other vendors is not available to ordinary consumers and clients, event though the vendors actually have such data available internally. As of 2008 there are prototype tools available that can predict defects and deployment costs of large commercial applications such as Vista and ERP packages. Some of the data comes from litigation filed against these vendors, so the predictions also include learning curves by users, installation defects, and even predications of consequential damages or the harm which bugs in commercial packages can cause to client companies. For example a serious bug in a commercial financial package caused a client company to restate past year earnings and therefore lose credit with its bank. Software Personnel Management: Strictly speaking personnel management is not the same as project management. However in day to day work, software project managers are also software personnel managers so they need to be involved in both kinds of activities. As of 2008 there are dozens of commercial tools for dealing with personnel issues such as hiring, skills inventories, appraisals, compensation, and other human resource issues. This is one of the most extensively automated areas of the entire spectrum of management work. Software Project Measurement, Tracking, and Control: There are scores of commercial and in-house tools for collecting resource and cost data for software projects. Often these cost and resource tracking tools are part of corporate-wide tool suites, so they operate for other kinds of projects besides software projects. However there are some common gaps and omissions in resource and cost tracking. The most common omission is measuring unpaid overtime. Software personnel tend to have rather energetic work habits, and often work on weekends or at home. However accurate measurement of unpaid overtime is rare. Failure to record unpaid overtime tends to artificially inflate productivity rates. Another gap in resource tracking concerns the work of part-time specialists such as quality assurance, data base administration, and technical writers. Since these workers may charge time to several projects or even be classed as overhead, sometimes their effort on specific projects is missing or inaccurate. Yet another kind of gap is the work performed by users when they participate in requirements analysis or are assigned to project teams as may occur when Agile development is used. From on-site studies where software project personnel were interviewed and asked to compare their actual effort to the results of project tracking systems, it was noted that the “leakage” from 20
  • many corporate resource tracking systems was in excess of 25%. That is more than one fourth of the actual effort was not recorded. Unpaid overtime and the work of users and specialists were the most common omissions. However, some companies also failed to track managerial effort and only reported technical effort by the programming team or software engineers. Leakage of more than 50% was noted in some cases. An interesting form of project tracking has been developed by the Shoulders Corporation for keeping track of object-oriented projects. This method uses a 3-Dimensional model of software objects and classes using Styrofoam balls of various sizes that are connected by dowels to create a kind of mobile. The overall structure is kept in a visible location viewable by as many team members as possible. The mobile makes the status instantly visible to all viewers. Color coded ribbons indicate status of each component, with different colors indicated design complete, code complete, documentation complete, and testing complete (gold). There are also ribbons for possible problems or delays. This method provides almost instantaneous visibility of overall project status. The same method has been automated using a 3-D modeling package, but the physical structures are easier to see and have proven more useful on actual projects. The Shoulders Corporation method condenses a great deal of important information into a single visual representation that non-technical staff can readily understand. Software Technology Acquisition: A weak link in the chain of project management tasks is that of technology acquisition. With more than 700 programming languages available, more than 40 named design methods, and more than 30 forms of development method is not easy to be sure that the languages, tools, and methods selected for any given project are in fact the best available. There are many important questions that are not easy to answer: when should Agile methods be used, and when are Agile methods inappropriate? Is level 3 on the capability maturity model (CMM) sufficient for a specific project, or would level 5 give better results? What kind of benefits will occur from using the Team Software Process (TSP) and Personal Software Projects (PSP)? What value does lean Six Sigma provide to specific projects? Will object-oriented programming languages be suitable for a specific project, or should some other language be used? Will automated testing be beneficial or not? If formal inspections are used will they lower testing costs? Are Use Cases more effective than other forms of specification? These are samples of questions that need accurate empirical data. There are few if any tools available for technology selection that match specific projects with the best choices for methodologies, languages, or design techniques. The growing volume of benchmark data collected by the International Software Benchmark Standards Group (ISBSG) is starting to provide better data based on historical projects. However more data and more measured projects are still needed. 21
  • Some of the commercial software estimation tools provide a side-by-side capability that allows the same project to be estimated with varying assumptions, such as changing programming languages, changing design methods, or changing CMM levels. These side-by-side estimates can be helpful, but not every language or method is supported. However the use of side-by-side modeling to predict the results of alternate scenarios is definitely a feature that should be added to all software project predictive tools. Software Mergers and Acquisitions: Some software managers can spend an entire career without being involved with mergers and acquisitions. Other managers may be involved as often as several times a year. Some of the issues with mergers and acquisitions are those of joining the portfolios of the two companies; with merging the software organizations; and with selecting the software tools and methods that will be used after the merger is complete, assuming the two companies use different approaches for development and maintenance. There are no standard tools available for supporting mergers and acquisitions, although some commercial estimating tools and personnel management tools can be helpful. Because the technologies deployed by merging partners are often different, here too side-by-side comparisons of one set of methods against another would be useful. Software Litigation Support: Software litigation comes in many varieties such as breach of contract, patent violations, tax litigation, theft of intellectual property, and violations of non- competition agreements. For breach of contract litigation expert witnesses are often asked to predict what a failing project would have looked like if state of the art methods had been used; i.e. what would the schedule and costs have been had the project been done well instead of done poorly. Questions about the cost and quality impacts of requirements changes often occurs in litigation too. For tax litigation, reconstructing the original development cost of software at a given point in time is a topic that occurs often, as well as questions about what it would cost to build a new version today. A combination of software cost estimating tools, software quality estimation tools, and general- purpose critical path tools are often used by expert witnesses in software litigation. For cases involving patent violations or theft of intellectual property, specialized tools that analyze the structure of software applications and produce graphs of that structure may be used. If two different software applications have common structures, it can be assumed that one version may have been copied from another version. For many kinds of litigation it is important to compare what actually took place against an alternate scenario of what might have taken place had more suitable methods been deployed. Therefore side-by-side comparisons are an important aspect of litigation support. 22
  • Because many lawsuits are filed due to poor quality or the presence of major bugs that prevented the applications from being used successfully, quality prediction and quality measurement are important aspects of litigation support. Litigation is a topic that is seldom taught by universities to computer science students, software engineering students, or even MBA candidates. However litigation is common enough in the software world so that information on litigation costs, probable duration, and disruption of staff time needs to be included in software project management tools. As of 2008 there is a prototype tool that can predict litigation costs and schedules for breach of contract suits for both the plaintiff and defendant. But the prototype does not deal with patent violations, theft of intellectual property, criminal hacking, violations of employment agreements, or other forms of litigation. The Costs and Value of Software Project Management Tools The cost of software project management tools ranges from free (for open-source tools or in- house tools) to more than $5,000 per seat for some commercial software cost estimating tools. Assuming a normal mix of in-house tools, open-source tools, and commercial tools the acquisition cost of a suite of software project management tools would cost about $0.50 per function point using the size of the tools in function points as the base. Thus to equip an individual software project manager with a suite of tools totaling 30,000 function points would cost about $15,000 for one manager. Because of discounts and multi-seat licenses, the effective cost per manager in a large company or a large project would probably be about $5000 for a suite of project management tools that encompass estimating, planning, risk analysis, quality control, and tracking of costs and progress. The average cost to build a successful software application in the 10,000 function point size range is about $1000 per function point or $10,000,000. However, unsuccessful projects that are cancelled usually accrue costs of more than $1,500 per function point and are about 12 months late when they are cancelled. In other words, cancelled software projects in the 10,000 function point range cost about $15,000,000 and obviously are a write-off without any positive value. If the failing project becomes subject to breach of contract litigation, the damage costs can be much higher than the cost of the software itself. But litigation damages are difficult to predict ahead of time. A 10,000 function point project would normally have about eight project managers and perhaps 64 development personnel. The costs of fully equipping the software project managers on 10,000 function point projects with state of the art software cost and quality estimating tools is not very expensive: at an average cost of $5,000 per manager and eight managers, the total cost would be perhaps $40,000. 23
  • Thus an investment of $40,000 in state of the art project management tools and quality control tools has the potential to keep the project within bounds and hence eliminate a $15,000,000 failure. If you subtract the $10,000,000 cost of a successful project from the potential cost of a cancelled project, the savings due to effective project management with a powerful tool suite would be about $5,000,000. Thus an investment of $40,000 in software project management tools could yield savings of $5,000,000 which is a return on investment of $125 for every dollar expended. This prediction is only an approximation, but the basic principles are probably valid. The presence of a suite of project management tools is not, by itself, the main differentiating factor between successful and unsuccessful software projects. The primary reason for the differences noted between failing and successful projects is that the project managers who utilize a full suite of management tools are usually better trained and have a firmer grasp of the intricacies of software development than the managers who lack adequate management tools. Bringing a large software project to a successful conclusion is a very difficult task which is filled with complexity. The managers who can deal with this complexity recognize that some of the cost and resource scheduling calculations exceed the ability of manual methods. They also recognize that state of the art quality control is a necessary prerequisite for success. Managers on failing projects, on the other hand, tend to have a naïve belief that project planning and estimating are simple enough to be done using rough rules of thumb and manual methods. Further, when projects began to show stress and experience problems, the managers on failing projects often do not know how to take corrective actions, and may just ignore the problems until far too late to solve them. Last, managers on failing projects (as noted in litigation) do not understand the economic value of quality control and hence encounter massive delays and cost overruns when testing begins. The Future of Software Project Management Tools Software project management circa 2008 is facing a number of future challenges even before successfully solving historical challenges. For the past 50 years the average size of software projects has been increasing. Since risk of failure and risk of poor quality correlate directly with size, larger projects require more care in predicting quality and controlling quality than small projects. This has long been a weakness of both software engineering and software management. New forms of software such as the service-oriented architecture (SOA) running on new kinds of platforms such as “cloud computing” are rapidly approaching. New development methods such as Agile development, Extreme Programming (XP), and Team Software Programming (TSP) need specialized estimating and planning methods. Security vulnerabilities have long been troublesome, and in the future will become even more troublesome than in the past. Thus project planning and estimating are becoming more complex as the industry evolves. 24
  • Although hundreds of commercial project management tools are available in 2008, many of these deal only with a small subset of overall management tasks. For example a software project management tool that can deal with schedule assignments, but which cannot deal with quality, is not an effective solution because bugs or defects are the primary source of schedule slippage. Further, current project management tools often do not interface well or share data well with other tools, so that it is difficult to create a consolidated picture of every important topic. What would benefit software project management tools for the next generation would be better ease of sharing data, and an improved ability to link together sets of specialized functions. In other words, apply the principles of service-oriented architecture to the domain of project management tools. From examining scores of cancelled projects and hundreds of costs and schedule overruns, it is obvious that a few basic topics need to be estimated and controlled in the future much better than they have been in the past: Quality control is the prime weakness of software projects. Excessive numbers of bugs and inadequate methods of defect removal are associated with every project failure and all litigation involving breach of contract. Requirements changes are endemic in the software world. It is not possible to stop requirements from changing, so the solution is to deal with changing requirements in the most effective and efficient manner. Cancelled projects or those with major overruns tended to be swamped by unexpected changes for which no schedule or cost relief had been envisioned. Tracking of progress on software projects has been a weak link. Project tracking omissions or outright concealment of true status tends to occur in breach of contract lawsuits. When problems are noted, they should be highlighted and addressed, not concealed and ignored. Furthermore, gaps in historical cost data are common. Unpaid overtime, managerial costs, and the work of part-time specialists such as quality assurance and data base administration are often missing or incomplete in software project cost tracking systems. This is especially true for in- house projects, but some omissions such as unpaid overtime even occur for outsourced applications. Accurate plans and estimates are needed before projects start. But it often happens that even accurate plans and estimates are rejected by clients or top executives and replaced by impossible schedule demands. Accurate estimation by itself is not a total solution, but accurate estimation supported by accurate historical data is likely to be believed. Therefore accurate benchmarks of historical projects are needed in larger volumes than have been available in the past. 25
  • The shared goals of the software project management community should be to improve the professionalism of software project management. We need to be able to develop large software projects with lower defect potentials, higher levels of defect removal efficiency, lower costs, and shorter schedules than has been accomplished in the past. The technologies for achieving these goals are starting to become available, but improvements in academic preparation for software project managers and improvements in selecting the best methods are still needed. 26
  • Suggested Readings on Software Project Management Baird, Linda M. & Brennan, M. Carol; Software Measurement and Estimation: A Practical Approach; IEEE Computer Society Press, Los Alamitos, CA; John Wiley & Sons, Hoboken NJ; ISBN 0-471-67622-5; 2006; 257 pages. Boehm, Barry Dr.; Software Engineering Economics; Prentice Hall, Englewood Cliffs, NJ; 1981; 900 pages. Brooks, Fred: The Mythical Man-Month, Addison-Wesley, Reading, Mass., 1974, rev. 1995. Crosby, Phil; Quality is Free; New American Library, Mentor Books; New York, NY; 1979, 270 pages. DeMarco, Tom; Why Does Software Cost So Much?; Dorset House, New York, NY; ISBN 0- 9932633-34-X; 1995; 237 pages. Fleming, Quentin W. & Koppelman, Joel M.; Earned Value Project Management; 2nd edition; Project Management Institute, NY; ISBN 10 1880410273; 2000; 212 pages. Galorath, Daniel D. & Evans, Michael W.; Software Sizing, Estimation, and Risk Management: When Performance is Measured Performance Improves; Auerbach, Philadelphia, AP; ISBN 10-0849335930; 2006; 576 pages. Garmus, David & Herron, David; Function Point Analysis; Addison Wesley, Boston, MA; ISBN 0-201069944-3; 363 pages; 2001. Garmus, David & Herron, David; Measuring the Software Process: A Practical Guide to Functional Measurement; Prentice Hall, Englewood Cliffs, NJ; 1995. Jones, Capers; Program Quality and Programmer Productivity; IBM Technical Report TR 02.764, IBM San Jose, CA; January 1977. Jones, Capers; Sizing Up Software; Scientific American Magazine; New York NY; Dec. 1998, Vol. 279 No. 6; December 1998; pp 104-109. Jones, Capers; Applied Software Measurement; McGraw Hill, 3rd edition 2008; ISBN 978-0-07- 150244-3; 575 pages; 3rd edition (March 2008). Jones, Capers; Software Assessments, Benchmarks, and Best Practices; Addison Wesley Longman, Boston, MA, 2000; 659 pages. Jones, Capers; Conflict and Litigation Between Software Clients and Developers; Version 6; Software Productivity Research, Burlington, MA; June 2006; 54 pages. Jones, Capers; Estimating Software Costs; McGraw Hill, New York; 2nd edition, 2007; 644 pages; ISBN13: 978- 0-07-148300-1. 27
  • Kan, Stephen H.; Metrics and Models in Software Quality Engineering, 2nd edition; Addison Wesley Longman, Boston, MA; ISBN 0-201-72915-6; 2003; 528 pages. Kaplan, Robert S & Norton, David B.; The Balanced Scorecard; Harvard University Press, Boston, MA; ISBN 1591391342; 2004. Love, Tom; Object Lessons – Lessons Learned in Object-Oriented Development Projects; SIG Books Inc., New York NY; ISBN 0-9627477-3-4; 1993; 266 pages. McConnell, Steve; Software Estimation – Demystifying the Black Art; Microsoft Press, Redmond, Wa; ISBN 10: 0-7356-0535-1; 2006. Parthasarathy, M.A.; Practical Software Estimation – Function Point Methods for Insourced and Outsourced Projects; Addison Wesley, Boston, MA; ISBN 0-321-43910-4; 2007; 388 pages. Putnam, Lawrence H.; Measures for Excellence – Reliable Software On-Time Within Budget; Yourdon Press, Prentice Hall, Englewood Cliffs, NJ; ISBN 0-13-567694-0; 1992; 336 pages. Putnam, Lawrence & Myers, Ware; Industrial Strength Software – Effective Management Using Measurement; IEEE Press, Los Alamitos CA; ISBN 0-8186-7532-2; 1997; 320 pages. Strassmann, Paul; The Squandered Computer; Information Economics Press, Stamford, CT; 1997. Stutzke, Richard D.; Estimating Software-Intensive Systems – Projects, Products, and Processes; Addison Wesley, Boston, MA; ISBN 0-301-70312-2; 2005; 917 pages. Yourdon, Ed; Outsource – Competing in the Global Productivity Race; Prentice Hall PTR, Upper Saddle River, NJ; ISBN 0-13-147571-1; 2004; 251 pages. Yourdon, Ed: Death March—The Complete Software Developer’s Guide to Surviving “Mission Impossible” Projects, Prentice Hall PTR, Upper Saddle River, N.J., ISBN 0-13-748310-4, 1997. 28
  • 29