Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

From arms race to green space


Published on


From 1950 to 1993, the Ricky Flats Nuclear Weapons plant operated as a production facility. Production ended in 1992, and today RFETS is a deactivated facility whose closure resulted in 6,000 acres of wildlife refuge. Here's how that happened

 ! " 






Published in: Technology
  • Be the first to comment

  • Be the first to like this

From arms race to green space

  1. 1. See discussions, stats, and author profiles for this publication at: From Arms Race to Green Space: PPM at Rocky Flats Environmental Technology Site Article · April 2003 CITATIONS 0 READS 6 1 author: Some of the authors of this publication are also working on these related projects: DOE Waster Management (Nuclear) conference proceedings View project Estimating Processes for Agile Development in Federal Government View project Glen Alleman University of California, Irvine 18 PUBLICATIONS 83 CITATIONS SEE PROFILE All content following this page was uploaded by Glen Alleman on 27 November 2017. The user has requested enhancement of the downloaded file.
  2. 2. Web site: 21 From 1950 to 1993, the Rocky Flats Nuclear Weapons Plant, located 15 miles northwest of Denver, Colorado, operated as a production facility for nuclear and non-nuclear components of nuclear weapons. Production terminated in 1992, and today the Rocky Flats Environmental Technology Site (RFETS) is a deactivated facility whose closure will result in 6,000 acres of land made available as a wildlife refuge. CH2M HILL’s Communication Group provides information and network services to RFETS in support of the site’s closure activities. Closure of this former nuclear weapons production plant involves the decommissioning of the wireline-based telecommunications, networking, and applications infrastructure from approximately 500 buildings and replacing it with wireless networking and voice over IP (VOIP) systems for use by the workforce removing the physical building structures and their nuclear materials. More than 100 database, Web, and commercial off-the-shelf applications must be maintained, enhanced, and decommissioned during the closure process. Post-closure, many of these applications will be used for long-term stewardship of the site. The delivery of this diverse set of projects takes place through the Program Management Office. Project managers and their staff provide support, guidance, and project coordination to functional managers and their staff, totaling approximately 120 professionals and technicians. Project planning, execution, control, and reporting are performed through portfolio management, balanced scorecard (BSC), earned value (EV) analysis, and project selection and measurement processes. This chapter focuses on the management of software development projects and the issues associated with making “level of effort” (LOE)1 development tasks visible to the management team. CHAPTER 2 From Arms Race to Green Space: PPM at Rocky Flats Environmental Technology Site by Glen B. Alleman 1LOE is a measure of resources consumed over a time period, not the physical progress of a project. The level of effort usually does not result in a final product or process.
  3. 3. What Is Project Portfolio Management? Project portfolio management (PPM) provides a view of projects designed to reveal task redundancies, allocate resources, and report progress to plan across all the projects and not just a few [16]. More importantly, it provides an “investment management” view of the projects — not just how much a project will cost, but also the anticipated risk and return on investment compared to other projects in the portfolio. Before this information can be used in decisionmaking, the portfolio of projects must meet several criteria. Performance data must represent the actual value delivered from the investment. The performance metrics must be connected to the strategies of the organization. The project selection process must be guided by the organization’s strategic goals, not just the tactical needs of individual groups. Without strategic goals and corresponding metrics, PPM provides little more than fancy graphs, charts, and numbers. Legacy Project Management Environment Prior to CH2M HILL’s assumption of the delivery of the IT functions at RFETS, a variety of forces driving the project management activities were in place. These forces, shown in Table 1, were the motivation for the introduction of PPM and its supporting processes [15]. These forces create a gap between the planning and the execution of these IT projects. The planning of a portfolio of projects, although not straightforward, is only half of the solution. Execution of these plans requires the definition of priorities, access to resource requirements across the collection of projects, and an understanding of the consequences of any resource or priority decision for the portfolio, not just for individual projects. In the absence of this process, the most vocal user usually receives the highest priority, at least until the organization experiences a crisis that forces a change in current priorities and resource availability. In his book The 7 Habits of Highly Effective People, Stephen Covey calls this situation “living in Quadrant I,” where everything is urgent and important [7]. This is the quadrant of crises, pressing problems, deadline-driven projects, intervention meetings, and few preparations. To gain control of a portfolio of projects, the project management team has to “live in Quadrant II,” the quadrant of preparation, prevention, value and priority clarification, planning, ©2003 Cutter Information LLC STRATEGIES FOR EFFECTIVE PROJECT PORTFOLIO MANAGEMENT22
  4. 4. and group empowerment. To live in Quadrant II, we have introduced the four systems described in Figure 1. These systems are necessary, but they alone are not sufficient to gain control of the IT project portfolio and deliver value [6]. What is needed is a vision of an effective IT department, one in which: n Resources can be allocated with ease n Work is planned on a daily, weekly, monthly, and quarterly basis, with the concurrence of the stakeholders and the suppliers of services and products n A reliable projection of the labor and material costs is provided by the system with the concurrence of all participants n Risk assessment and intervention are everyday activities Web site: CHAPTER 2: FROM ARMS RACE TO GREEN SPACE: PPM AT ROCKY FLATS ENVIRONMENTAL TECHNOLOGY SITE 23 Driving Force Consequence High demand for immediate delivery of applications, telecommunications, and infrastructure services. Documentation of this demand was located in several systems, none of which were interoperable. Management could not coordinate overall demand from a centralized location. Localized planning activities were developed, further isolating this information. Staffing levels were being reduced as part of the closure plan. Coordinating work demand with resource availa- bility was done on an ad hoc basis, within a short planning window. No “repurposing” of staff could be done, since an integrated resource utilization management plan could not be developed for the workload. A broad mix of clients, funding sources, priorities, and decision- makers created a poor delineation of the relationship between suppliers and consumers of IT services. Correlating needs across tech- nologies, user communities, and delivery systems could not be done. Outcomes that might benefit a broader community could not be identified. The project selection process was ad hoc. There was no alignment between project priorities and the IT strategy for providing these services and delivering value. Project selection was based on local departmental needs rather than broader business needs. No feedback on the business benefits was available to refine the selection process. There was no centralized project inventory, repository, or tracking system to enable the enterprise valuation of projects. Individuals held projects in “private” locations, making open and visible discussion of priorities and resource integration difficult. Table 1 — Forces Driving Legacy Management of Project Portfolios
  5. 5. The Foundation of PPM in the IT Services Domain Successful management of IT project portfolios assumes that the best candidates for success are selected for deployment. To identify candidate projects, the organization must identify its strategic goals. From these goals, it creates initiatives. The organization then assembles collections of projects that support the initiatives. It installs metrics for the projects, followed by data capture and assessment. Feedback from the assessment is then used to verify the impact of the strategy on the organization. A portfolio management approach used to address the gaps shown in Table 1 moves the management process away from Quadrant I’s external drivers and toward Quadrant II’s process management and alignment. This control-based approach makes use of the four management techniques listed in Figure 1. Balanced Scorecard In order to “manage” our portfolio of projects, these projects must be connected to the strategic goals of the site and its closure activities. Balanced scorecard is a management system that clarifies vision and strategy and translates them into actions. These actions are defined through four perspectives: business results, customer expectations, internal processes, and strategic enablers [2]. A “strategy map”2 is constructed that states specific objectives for the deliverables from each perspective. Such a map is shown in Figure 2. Although BSC may appear to be an esoteric management tool, it is a powerful starting point for project portfolio management [10, 12]. BSC answers the ©2003 Cutter Information LLC STRATEGIES FOR EFFECTIVE PROJECT PORTFOLIO MANAGEMENT24 A balanced scorecard strategy to define priorities and establish a connection between every project and a specific strategy and objective A centralized public repository of projects, resources, and deliverables housed in Microsoft Project 2002 Server Earned value analysis to make visible performance metrics for cost and schedule Technical performance measurement processes to select, classify, measure, and guide the implementation of collections of IT projects Figure 1 — Creating a foundation for PPM. 2The strategy map is a third-generation balanced scorecard concept. It assembles in one place the objectives and their relationships in pursuit of the organization’s delivery of value. Each member of the organization should be able to “tell the story” of the role of each objective and how the projects support the fulfillment of these objectives.
  6. 6. questions: “Why are these projects being considered?”; “How does this project support a strategic direction?”; and “What is the value of this project to the organization beyond monetary returns?” In some organizations, there is a distinction between “public” projects and “private” projects. At CH2M HILL, all of our RFETS-related projects are public. If a project cannot be aligned with a BSC objective, it cannot continue to be funded [5]. The four balanced scorecard perspectives require us to consider the following questions [10]: n Business results. What are the objectives of our efforts? How will we know when we are successful? n Customer expectations. What words do our customers use when they are asked to describe our products and services? n Internal processes. What work processes must be in place to deliver value to the customer? n Strategic enablers. What behaviors, attributes, facilities, and resources must be in place before internal processes can function, customer needs can be heard, and value can be delivered to the customer? Web site: CHAPTER 2: FROM ARMS RACE TO GREEN SPACE: PPM AT ROCKY FLATS ENVIRONMENTAL TECHNOLOGY SITE 25 “Do the right things... Do them well... Do them with less...” reduce overall cost enable profitable operations enable accelerated business processes Sustain required services Manage IT compliance Understand customer operations Implement timely and cost-effective IT solutions Reduce cost of providing IT services Manage projects effectively Centralize customer IT resources Strategically deploy IT services Develop innovative IT solutions Improve IT processes Build effective customer relationships Build a high- performance, closure- oriented IT culture Develop and retain critical skills Provide employees with tools and knowledge Recognize team and individual performance Competency Contribution Operating Excellence Leadership Customer/IT Alignment Leverage Customer Knowledge and Best Practices Impact Business Results Customer Expectations Internal Processes Strategic Enablers Figure 2 — A sample strategy map.
  7. 7. One aspect of BSC is difficult to accept at times — the idea that every project in the portfolio must support an objective in the strategy map. If a project does not support an objective, then the question “Why are we doing this?” must be answered. If there is not a strategy-based answer to that question, then the project is a candidate for being dropped. The organization needs the strength to assess projects in this manner. If it can do this, all the work that an IT function performs will support identifiable strategic objectives. Centralized Project Repository The assembly of all projects in a central location is a critical success factor for PPM. By “all” it means all projects, no matter how small or obscure.3 We adopted Microsoft Project Server to house our portfolio of projects. Although it may not be the ideal choice, it provides a central repository for projects and their tasks, resources, and costs. It provides earned value analysis of projects once they have been baselined. Labor hours are captured through the Project Server as well as a time and labor reporting system audited by the Defense Contract Audit Agency. These hours and nonlabor costs are applied to the project baseline to compute cost and schedule variances and project performance indices. The effort needed to capture all activities into a central system should not be underestimated. Organizational and human roadblocks appear that say, “But what you’re trying to measure is not a project.” The solution is to consider all activities to be a “project,” even the LOE tasks. This way, all work is captured in the Project Server. For LOE activities, a project is defined in which LOE tasks are recorded. No matter what the activity, it can be found in the Project Server, with the associated costs, resources, durations, and deliverables. Earned Value Analysis Traditional methods of measuring progress to plan produce poor results. We apply earned value to projects and collections of projects. EV is used to measure and communicate the physical progress of a project based on “work completed,” the effort used to complete the work, and the nonlabor costs incurred to complete the work. EV is used to evaluate and control project risk by measuring project progress in monetary terms for the actual value delivered to the customer (see Figure 3). ©2003 Cutter Information LLC STRATEGIES FOR EFFECTIVE PROJECT PORTFOLIO MANAGEMENT26 3Small projects are captured as “to-do” lists, rather than Gantt-based projects. These to-do lists have resources assigned, deliverables, and due dates. They act just like real projects, but with much lower overhead.
  8. 8. EV provides “leading” performance indicators that allow the project managers to identify and control project problems before they become insurmountable. To do this, EV adds a third dimension to cost and schedule — the actual work accomplished. Measuring the actual work accomplished provides greater insight into potential project risks. It also provides a more accurate estimate of the completion schedule and cost estimates. With these leading indicators, project managers proactively manage projects in ways not possible using only cost and schedule measures. The key to deploying EV for software development portfolio management is to define a measure of value that indicates real progress, not just the passage of time and the achievement of major milestones. Technical performance measurement (TPM) is a program management tool that builds on the strength of earned value by adding a third value — the status of technical achievement [13]. Technical Performance Measurement The typical methods of measuring value are based on subjective assessment of the progress or a single outcome that occurs after a period of time. Both approaches fail to deliver accurate measures of progress for software projects. The TPM approach asks the question, “How do we know the software will behave as specified?” If the software does behave as specified, then the development step is complete. If not, then rework is needed [3]. We use TPM to answer the question, “How do we know what done means?” [8, 13]. Technical performance measurement is the description of expected technical achievement. Actual project progress is compared using fine-grained Web site: CHAPTER 2: FROM ARMS RACE TO GREEN SPACE: PPM AT ROCKY FLATS ENVIRONMENTAL TECHNOLOGY SITE 27 Cost variance (CV) dollars (labor days) of spending behind or ahead of schedule SV (days behind or ahead of schedule) Schedule variance (SV) dollars Planned value (BCWS) Budgeted cost of work performed earned value (BCWP) Actual cost of work performed (ACWP) Cost Time Figure 3 — Simple EV analysis metrics.
  9. 9. deliverables and their supporting tests. The difference between the planned progress and the actual progress represents a technical variance. The basis of TPM is testable requirements [18, 19]. A testable requirement defines the capability of a software system to meet its objectives and the conditions under which it can do so. Testable requirements follow the SMART criteria: Specific, Measurable, Achievable, Relevant, and Time-based. The criteria are met only if it is possible to write a test case that validates that the requirement has or has not been implemented correctly. The Software Management “Level of Effort” Problem Traditional methods of comparing actual spending to planned spending are inadequate for establishing, assessing, monitoring, and predicting the future performance of a software project. The traditional approach of comparing budget to costs fails to consider the technical achievements — the physical progress — that is accrued for the software development tasks. In the traditional software development project, the passage of time is assumed to be equivalent to progress toward completion. But software projects all too often have no deliverables that are physically visible. The common outcome is that large costs are incurred with little useful product being delivered until the end. At RFETS, our first step was to identify the “value” of software deliverables as a set of “verifiable” outcomes. These outcomes have two attributes critical to the deployment of EV: 1. They are defined in fine-grained, self-contained units of work.4 2. Each unit of work is testable to ensure the requirements can be verified on fine-grained boundaries [14]. Earned value principles (as defined in EIA 748)5 are applied to the following activities: n Plan all work scope for the project through completion at a level sufficient to measure progress through testable requirements.6 ©2003 Cutter Information LLC STRATEGIES FOR EFFECTIVE PROJECT PORTFOLIO MANAGEMENT28 4Defining deliverables on fine-grained boundaries (one to three days) increases the resolution of the performance metrics as well as the accuracy of the estimate at completion. 5EIA 748 is the specification that defines the earned value criteria. For more information, see 6In our method, this level of detail is performed for each iteration. The completion of the project is planned, but this level of detail is focused on the current and next iteration.
  10. 10. n Integrate project work scope, schedule, and cost objectives into a performance measurement baseline plan against which accomplishments may be measured. n Use actual costs incurred in accomplishing the work performed. n Objectively assess accomplishments at the work performance level using testable requirements. n Analyze significant variances from the plan, forecast impacts, and prepare an estimate at completion based on performance to date and work to be performed. Project Selection Process With these three processes in place (BSC, EV, and testable requirements), deciding which projects will receive funding, which ones have the highest payback, and which ones to concentrate on turns out to be an ill-structured problem [4, 11]. Addressing this problem requires three activities: 1. Representing the problem by uncovering information and the supporting justifications for the project 2. Stating the solution to the problem by gathering options and selecting from among them 3. Supporting the justification through measures and tests Since the selection problem has multiple solutions, gaining agreement on which projects to include requires considerable effort. In order to succeed, three actions guide the project selection process: 1. Defining strategies that identify the value of the proposed projects 2. Making use of formal evaluation principles 3. Defining the processes by which projects are evaluated for their contribution to the strategic goals There are many selection and evaluation algorithms, some simple and some complex. We have chosen two simple approaches: 1. Making a list. The making of the prioritized list is an algorithm, a simple one. The process of making the list and prioritizing it is sufficient for our purposes, since it exposes the needs of the customer and defines the technical and resource requirements. Web site: CHAPTER 2: FROM ARMS RACE TO GREEN SPACE: PPM AT ROCKY FLATS ENVIRONMENTAL TECHNOLOGY SITE 29
  11. 11. 2. Paired comparison analysis. For projects whose priorities are difficult to sort out or those that have other conflicts that a simple linear list cannot deal with, paired comparison analysis is a useful tool. It is a list-making algorithm, but one that sorts out the priorities. It asks, “Between these two projects, which one must come first?” Paired comparison analysis helps work out the importance of a number of options relative to each other, particularly when there is no objective data [4, 11]. Critical Success Factors for Portfolio Management Using the four techniques defined in Figure 1, PPM is driven by the following critical success factors: n Process. Define a clear and concise process by which the organization can categorize, evaluate, and select projects. n Strategy. Every project must have a “strategic” purpose and support an objective in the balanced scorecard. If not, it should be dropped. n Relationships. Collections of projects are difficult to define in the absence of a higher-level mission. The balanced scorecard provides the means of organizing projects into initiatives. These initiatives can then be directly traced to strategy map objectives. n Decision framework. Deciding which projects are to be executed first can be done by asking which projects need to be executed to fulfill the strategic objectives in the proper sequence. n Decision framing. Clearly define the problem to be solved, the decision criteria, and the realistic tradeoffs and options. The successful deployment of project portfolio management depends very little on the selection of a portfolio management application. Successful deployment depends on defining the desired outcomes of the projects from both a technical and business perspective. We measure the success of these projects using earned value, which includes both financial and technical progress indices. Housing the projects in an enterprise server is useful since resources, budgets, and costs can be consolidated in a single view across all projects. Selecting which projects to pursue is also important, but complex selection algorithms may not be necessary. Simple questions of business value, resource utilization, and returned value need to be answered first before more complex selection processes are used. ©2003 Cutter Information LLC STRATEGIES FOR EFFECTIVE PROJECT PORTFOLIO MANAGEMENT30
  12. 12. In the end, connecting projects and portfolios of projects to balanced scorecard initiatives provides a powerful tool for answering the question, “Why are we doing this project?” If there is no strategic initiative that will be enabled by the project, then the project has no purpose. Without a strategic purpose, no effort should be expended. References 1. Ben-Menachem, Mordechai, and Roy Gelbard. “Integrated IT Management Toolkit.” Communications of the ACM, Vol. 45, No. 4, April 2002, pp. 96-102. 2. Berkman, Eric. “How to Use Balanced Scorecard.” CIO, 15 May 2002. 3. Boehm, Barry, and Kevin Sullivan. “Software Economics: A Roadmap.” In Proceedings of the Conference on the Future of Software Engineering. ACM, 2000. 4. Clemen, Robert T. Making Hard Decisions: An Introduction to Decision Analysis, 2nd edition. Duxbury Press, 1995. 5. Cobbold, Ian M., and Gavin J.G. Lawrie. The Development of Balanced Scorecard as a Strategic Management Tool. 2GC Active Management, 2002. 6. Constantine, Larry. “Work Organization: Paradigm for Project Management and Organization.” Communications of the ACM, Vol. 36, No. 10, October 1993, pp. 35-43. 7. Covey, Stephen. The 7 Habits of Highly Effective People. Simon & Schuster, 1990. 8. Ferraro, Mike. “Technical Performance Measurement: A Program Manager’s Barometer.” Program Manager, November/December 2000, pp. 14-20. 9. Fleming, Quentin, and Joel Koppelman. Earned Value Project Management. PMI, 2002. 10. Kaplan, Robert S., and David P. Norton. The Strategy-Focused Organization. Harvard Business School Press, 2000. 11. Mollaghasemi, Mansooreh, and Julia Pet-Edwards. Making Multiple- Objective Decisions (IEEE Computer Society Technical Briefing). IEEE Computer Society, 1997. 12. Niven, Paul R. Balanced Scorecard Step-By-Step: Maximizing Performance and Maintaining Results. John Wiley & Sons, 2002. Web site: CHAPTER 2: FROM ARMS RACE TO GREEN SPACE: PPM AT ROCKY FLATS ENVIRONMENTAL TECHNOLOGY SITE 31
  13. 13. 13. Pisano, N.D. Technical Performance Measurement, Earned Value, and Risk Management: An Integrated Diagnostic Tool for Program Management. US Naval Air Systems Command Program Executive Office for Air ASW, Assault, and Special Mission Programs (www.acq. 14. Sanderson, Sandra A. “Earned Value (EV) Management Model for Small Software Development Projects.” Paper presented at the 2001 Software Technology Conference, Salt Lake City, Utah. 15. Scott, Judy E., and Iris Vessey. “Managing Risks in Enterprise Systems Implementations.” Communications of the ACM, Vol. 45, No. 4, April 2002, pp. 74-81. 16. Solomon, Melissa. “Project Portfolio Management.” ComputerWorld, March 2002. 17. Summer, Mary. “Critical Success Factors in Enterprise-Wide Information Management Projects.” SIGCPR, 1999. 18. Wilson, P.B. “Testable Requirements: An Alternate Sizing Measure.” The Journal of the Quality Assurance Institute, October 1995. 19. Wilson, P.B. “Sizing Software with Testable Requirements.” Journal of Systems Development Management, August 2000. ©2003 Cutter Information LLC STRATEGIES FOR EFFECTIVE PROJECT PORTFOLIO MANAGEMENT32 View publication statsView publication stats