Establishing the performance measurement baseline (v4)


Published on

Published in: Business, Technology
  • Be the first to comment

Establishing the performance measurement baseline (v4)

  1. 1. AACE Symposium, April 16th, 2011 Thank you all for coming to the work shop today. The title of the Work Shop title speaks about the Performance Measurement Baseline. This may be a new term for some of you. After our four hours today, I’m hoping it will be a term youll be using back on your own projects. Let’s define what we mean at this point in the work shop. At the first slide. The Performance Measurement Baseline is a time-phased budget plan for accomplishing work, against which the project performance is measured. It includes the budgets assigned to scheduled work , their budget spreads, and the applicable indirect budgets. This budget plan is derived from a resource loaded Master Schedule. These resource loads, along with other information, are contained in Work Packages. You’ll hear many times the phrase “cost and schedule.” I want you to start thinking about what it sounds like when you say “schedule and cost.” It’s the schedule – the sequence of the Work Packages – that is the basis of the cost.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 1/34
  2. 2. AACE Symposium, April 16th, 2011 There are four words that we’ll connect together today. These words – when they are connected, will describe how to increase the probability of success for your projects. Any project, doesn’t matter the domain, the context in that domain, the methods used to manage the project, the specific technology of the products in the projects. If you don’t have these four words in your vocabulary, you’re going to reduce the probability of success. This doesn’t mean the project will fail. This doesn’t mean the project will be challenged. All it means is the Probability of Project Success will be reduced. These four words are here in front of us.  Credible  Performance  Measurement  Baseline The title of this talk puts the words together. But they also stand alone. Let’s look at one as a standalone word first.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 2/34
  3. 3. AACE Symposium, April 16th, 2011 Credible. What does it mean to be credible. Well Jiminy Cricket here can tell the difference between credible and not so credible. It’s harder for us to do that on a project. But we have some methods that you’ll see today that can be used to test the credibility of a project’s ability to successfully be completed. These methods are probabilistic in mature. They ask and answer probability questions, using simple statistics.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 3/34
  4. 4. AACE Symposium, April 16th, 2011 When we speak of performance, we should only be speaking in terms of deliverables, percent complete, tangible evidence of completion of our planned work. Not in terms of cost and schedule. The consumption of resources, the expenditure of money or the passage of time. So our Performance Measurement Baseline needs to use units of measures that are tangible. These units of measure must also be meaningful to the decision makers as well.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 4/34
  5. 5. AACE Symposium, April 16th, 2011 Measuring means measuring against the plan, on the day the plan says there is an expected deliverable. The units of measure are predefined in the work package that is going to be on baseline. The measure is always some tangible evidence. A milestone met, a deliverable produced, a count of some kind. The measure is never an opinion of the owner of the measurement.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 5/34
  6. 6. AACE Symposium, April 16th, 2011 The word baseline in Performance Measurement Baseline, means the cost of the scheduled program activities that have been approved for execution. Management of the baseline is a change control process once the Performance Measurement Baseline goes on baseline. We’re going to see today there are actually three baselines. 1. The technical baseline 2. The cost baseline 3. The schedule baselineCopyright ©, 2011, Lewis & Fowler, All Rights Reserved 6/34
  7. 7. AACE Symposium, April 16th, 2011 Here’s how we are going to build our Performance Measurement Baseline. It’s a simple process in principle. But of course in practice, it’s always harder in practice. We’ll use the exercises to pull out these practices from the principles, but let’s have a quick look first at the principles. 1. Build a Work Breakdown Structure. 2. Define the Work Packages that produce the deliverables at the terminal nodes of the WBS. 3. Arrange these Work Packages in some logical sequence. 4. Assign the resources needed to complete the Work Packages in a timely manner. 5. Turn this whole collection of information into a credible Performance Measurement Baseline (PMB). One final step is to perform the continuous risk management processes inside these five (5) steps, so we actually increase our probability of success. The key here is “increasing the probability of success.” We need to remember this phrase. It’s the inverse of many of the efforts made to manage projects. Connecting actions with outcomes is the Critical Success Factor.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 7/34
  8. 8. AACE Symposium, April 16th, 2011 Let’s look at a broader context of the five (5) essential process areas for managing any project. Other project management paradigms have even other process areas – Prince2 for example. But the process areas here have emerged over the course of several decades of managing defense, large construction, and enterprise IT projects. These five (5) processes are: Identify the needed capabilities – this is commonly missing from many projects. A capability is not the same as a requirements. One way to think of this in the Enterprise IT is the ask “if I had the system, it was free, it worked on Monday, what would I do with it? Once you’ve defined the needed capabilities, you need to identify the technical and operation requirements needed to enable these capabilities. Then comes establishing the Performance Measurement Baseline. And then the execution of the PMB. At each process, we must apply continuous risk management in place and operational. The other 4 processes areas are work shops unto themselves, so for today, we’ll concentrate on building the PMB and assume the other 4 areas are in place.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 8/34
  9. 9. AACE Symposium, April 16th, 2011 The term “baseline” has an important role here. It is a “controlled” document that contains the agreed on information about the future performance of the project. When we say “baseline,” we really mean three baselines.  The Technical Baseline is the agreed on set of technical requirements. These can be changing or they can be frozen, or any place in between.  From the technical requirements baseline, we have the work needed to implement them in the Schedule Baseline.  From this work we can establish the Cost Baseline. All three baselines are connected in an inseparable relationship, change one and the other two are impacted. This is sometimes called the “iron triangle.” Trying to make tradeoffs between the three variables can be done early in the project. Once underway, trade offs between these three baselines and the belief that there will be no negative impacts is a Ponzi scheme.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 9/34
  10. 10. AACE Symposium, April 16th, 2011 What we’re here for today is to build the Performance Measurement Baseline. The core elements of the PMB are Work Packages. Work Packages are “lumps of work,” that produce a single outcome. The idea of the single outcome has important attributes. If we have multiple outcomes, it’s hard to measure progress with 0% or 100% completion criteria. If we have multiple outcomes who’s accountable for each outcome? Try as hard as possible to have a single outcome for each Work Package. Once we’ve connected the Work Package with an outcome, we can ask other questions:  How long will it take?  How much will it cost?  Who’s accountable for delivering the outcome?  What are the risks involved in delivering this outcome?  What dependencies are there for this Work Package or other Work Packages or external items? We need to answer these questions, if we ever have a chance at having a credible PMB. By credible I mean “believable.” It doesn’t have to be “right,” there are few “right” answers. It has to be feasible and it has to be credible.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 10/34
  11. 11. AACE Symposium, April 16th, 2011 So let’s look at an example of building a bio lab to BSL Level 4 specifications. BSL Level 4 is REALLY bad stuff.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 11/34
  12. 12. AACE Symposium, April 16th, 2011 Here’s a page from our Deliverables Based Planning ® method handbook. This method is applied in a variety of domains and contexts in those domains. The critical success factor for Deliverables Based Planning® is to focus on the deliverables. Not on the effort, the technology, the resources. These items are important. But the customer paid for the deliverables. By customer I mean the general notion of a customer. A business customer, a government customer, an internal customer. The units of measure for these deliverables must be meaningful to the customer. This is the reason to start with the Capabilities Based processes shown in slide 4. We’ll assume these have been defined, and the technical and operational requirements defined. Now we’re taking those requirements and building the Performance Measurement Baseline that will make them appear. These 6 steps are “immutable” in that they are all needed and they need to be performed in the proper order. This doesn’t mean they not iterative and incremental, but each step takes information for the previous step.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 12/34
  13. 13. Let’s start with the Work BreakdownStructure (WBS). We all know aboutthe WBS right?We should all also know there is anew specific for the WBS MIL-STD-881C that will be coming in June ofthis year. For us in the defense businessthe STD part has replaced the HDBKpart of the title. It is no longer asuggested approach, it is a mandatedapproach.AACE doesn’t have a specific WBS 13
  14. 14. AACE Symposium, April 16th, 2011 Here’s a “notional” starting point. Using the WBS paradigm, we need to capture the Products and the Services that produce those products. The WBS does NOT describe the functional roles of the project. Design, construct, inspect, develop, test, install are not WBS elements. The focus is on “deliverables,” the end item deliverables of the work efforts. So for this example we have.  The design basis. While the word design is here, it is the document that is the deliverable.  The processing document is the other deliverable. This document described how the BSL Lvl-4 materials handling processes will be performance for this lab. That is critical to FDA certification before construction starts, so it’s a deliverable.  Each of these deliverables has two sub deliverables – the elements that make up the actual deliverable. Notice here with well formed structure. Each parent has proper children. It is a well formed tree. All the children only go to one parent. No step-parents.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 14/34
  15. 15. AACE Symposium, April 16th, 2011 Let’s connect the dots between the WBS and the Work Packages that guide the activities that produce the artifacts of the WBS. These Work Packages are derived from the terminal nodes of the WBS. This picture shows where the Work Package lives in this process. We take our notional WBS and for each terminal node make a Work Package. The Work Package contains many things, but the first thing it contains is the list of work needed to produce the deliverable. This might be called a schedule, but let’s not go there yet. Let’s just get the list of work activities, maybe their sequencing. And most of all the list of “named” deliverables produced by the Work Package.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 15/34
  16. 16. With the WBS and the WorkPackages defined, let’s define who isgoing to do the work. 16
  17. 17. AACE Symposium, April 16th, 2011 Just a reminder that the “owner” of the Work Package this person is accountable for. It’s always about the deliverables. There are other activities of course, but the deliverables are what the customer bought. This is why the planning and scheduling process stops at the Work Package. With the Work Package manager being accountable for the deliverables, “how” the work package is executed must be the role of the Work Package Manager and her team. Within the project governance guides, you want to push down the accountability to those doing the work. They can’t do this work in any arbitrary way, but they must be accountable for the outcomes. On our large projects, Work Packages have a period of performance (start- end), assigned resources, and a budget profile. The Work Package Manager may have a detailed schedule, or just a bunch of notes stuck to the wall. This is called a “supplemental schedule.” The details are not on baseline. The Work Package is. Below this level detailing out the work is the role of the Work Package manager.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 17/34
  18. 18. Here’s a real world example of theResponsibility Assignment Matrix(RAM).The first approach is to just put in the“accountable” connections.Once those who are accountable forthe deliverables are defined, it’s alldown hill from there. 18
  19. 19. The next step is to lay out these workpackages in their order of execution.This can be done in a variety of ways,sticky notes on the wall is one.Fancy electronic tools is another.The sticky notes provides much moreflexibility early in the project. 19
  20. 20. Here’s a picture of “real” WorkPackages being arranged by “real”Work Package managers, on a “real”project.This is a simple process. Brown paper,sticky notes, hand written WorkPackage descriptions.This process is called ProductDevelopment Kaizen.The critical idea is to have collectiveownership of the arrangement of theWork Packages, while having singleaccountability of the contents of eachWork Package by the Work PackageManager.Arranging the Work Packages is a fullcontact sport.When complete, leave the sticky noteson the wall for all to see.You will move these into a projectscheduling tool of course, but even thenyou should have a “plot” of theseWork Packages. 20
  21. 21. AACE Symposium, April 16th, 2011 With the Work Packages in place, documented to some level of detail, the estimated durations, and work efforts, it’s time to lay them out in some order of execution. We need to identify the predecessors and successors of the “lumps of work.” Note the tasks – that’s an issue for the Work Package Manager. The order needs to be logical in the sense that the products produced by the Work Packages can be consumed by the next Work Package. With this sequence, we can ask questions about the flow of value, the impacts on resources, the general logic of how the work progresses toward DONE. At this level of granularity, the emerging Master Schedule comes out. This is then the basis of the Work Authorization process. The authorization of work is not used to prevent work from being performed, but to assure the work is performed in the agreed on order. Performing work “out of order,” creates lots of problems, not the least of which is outcomes sit idle while others are waiting for materials. This is a “work flow” issue and needs to be addressed to ensure the best effectiveness from the resources.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 21/34
  22. 22. AACE Symposium, April 16th, 2011 Let’s take a short diversion here for a critically important concept. Leads and Lags in the schedule are restricted in most defense projects. NAVAIR, the Navy’s aviation arm, allows 5 days lead or lag in the Integrated Master Schedule between any two Work Packages. On a 5 year project 5 days might as well be zero (0) days. The reason to restrict leads and lags is the follow on work activity then uses partially complete products. This is the source of “rework.” When the predecessor activities finally complete, there is likely more work done and likely changes in that work. The successor work then has to readjust for these changes. If there is intermediate output needed by successor work, split the Work Package and produce 100% of the maturity for the intermediate output. By “thinking” in terms of “increasing maturity,” leads and lags can be removed. The project is a work flow of increasing maturity. We want to maximize this value within the constraints of schedule and cost.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 22/34
  23. 23. Now that we’ve got the WorkPackages in some kind order – not thefinal order – but something that’sstarting to look like a plan, let’s seewhat resources and costs are going tobe needed. 23
  24. 24. AACE Symposium, April 16th, 2011 Putting budget to Work Packages at this point is real simple. We’ve got definitions of the deliverables, the needed resources assigned – at least in lumps – to the Work Packages. Now it’s simple to budget. From the head count, assign a labor rate, possibly forward adjusted for all the changes – and see what number comes out for each Work Package. In Earned Value terms – which we’ve avoided so far – this is the Budgeted Cost for Work Schedule – BCWS. No matter what cost performance management system we are using, we’ve now got the “budget spreads” defined for each Work Package. Put this number back into the document that describes the Work Package. This is one advantage of building the Work Packages in Excel, you can now do math on the budget numbers.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 24/34
  25. 25. We don’t have a lot of time today totake deep dived into building thePMB, but this topic is criticallyimportant to any credible PMB.We must have a probabilisticunderstanding of how cost, schedule,and technical performance all interact.Here’s a notional example of using aMonte Carlo simulator to answer thequestion – what’s the probability ofcomplete on or before a date andspending an amount of money or less? 25
  26. 26. Now we reach the end and the hardpart.Let’s start with an understanding thatmeasures of progress on in units of“tangible evidence.”No opinion, no guesses, no “well I thinkwell be OK, we’re probably 60%done with the work to date.”Nope, tangible evidence of progressto plan is the only acceptable measureof progress. 26
  27. 27. AACE Symposium, April 16th, 2011 The first thing to do when defining the units of measure of DONE is to agree that they must be “objective.” Tangible Evidentiary Materials is the formal word. They have to be tangible. Something you can touch, see, smell, look at. A simple example can be around drawings. Our Work Package produces 10 drawings for our workshop example toaster. We say the period of performance for these drawings is 2 weeks. Assuming a linear production of work, we should see 5 drawings at the close of business Friday for the first week. We defined this Measure of Performance (MoP) in the Work Package. On the Friday afternoon, you can walk over to the drafting area and ask to see the 5 drawings that are planned to be completed. If you see the 5 hanging in the stick- file, then you are 100% complete for the planned 50% completion point in the Work Package. The Estimate to Complete is now 50% - the other 5 drawings – and with the past performance of “on time,” you can naively assume you’ll get the next 5 at COB next Friday. It may serve you better if you went to visit the drafting department over lunch on Wednesday to see if 2 ½ drawings are done – this answers the question how long are you willing to wait before you find out you’re late?Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 27/34
  28. 28. AACE Symposium, April 16th, 2011 In the file that contains all the project information, we need to write down what these success criteria or exit criteria are, so no one forgets what we agreed to when we planned out the Work Packages. The term “earned value method” is on this slide and we haven’t mentioned its use – and we won’t go there for now. But EV is a critically important measurement tool for any type of project. This is a topic for another workshop, but I want to plant the seed around EV. EV measures physical percent complete against planned percent complete in ways no other project management performance measurement process can. Agile’s story points are Uncalibrated, Lean and Critical Chain’s “value flow” is Uncalibrated. EV measures progress to plan in units of “money.” And money is what project management is all about.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 28/34
  29. 29. With all this data collected in oneplace – the Integrated MasterSchedule – we need to actually put iton baseline. 29
  30. 30. AACE Symposium, April 16th, 2011 So why all these words about “setting the baseline?” The baseline is the “contract” between the people doing the work, the people managing the work, and the people receiving the results of that work. It is a “contract” about “how long,” “how much,” “when,” sometimes “why,” and most importantly “what.” It’s an agreement of how to get to DONE. Like all good agreements, making changes to the agreement is only done with the full consent of all the parties involved. So where do we go to look at what we agreed to? The Baseline.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 30/34
  31. 31. Annual Rocky Mountain Project Management SymposiumRisk Management in Five Easy PiecesDenver Colorado April 11, 2008 This list comes from guidance in the defense program management business. While some of the statement may not be appropriate for the commercial world, every statement has some applicability to every project – not matter what the domain. Whether formal or informal the programmatic risk assessment of a project needs to ask or answer these statements. The actions needed to close any gaps from these statements are outside the scope of this presentation. But the next step is to have the project management team start to answer these questions.Glen B. AllemanLewis & Fowler, 8310 South Valley Highway, 31
  32. 32. AACE Symposium, April 16th, 2011 Thank you for investing your time today. With the time we have left, let’s open this up to questions. If we run out of time, please drop a card with my colleague and we’ll make direct contact.Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 32/34
  33. 33. AACE Symposium, April 16th, 2011Copyright ©, 2011, Lewis & Fowler, All Rights Reserved 33/34
  34. 34. AACE Symposium, April 16th, 2011 Here’s my contact information if you have any questions later, want a soft copy of this presentation with the speaker notes, or any other itemsCopyright ©, 2011, Lewis & Fowler, All Rights Reserved 34/34