Extreme Project Management

Extreme Project Management Ed Yourdon email: ed@yourdon.com Website: www.yourdon.com Blog: www.yourdonreport.comTwitter, LinkedIn, Facebook, Plaxo, Flickr: “yourdon” October 2011 green stuff = “internal” links — i.e., URLs to other pages in this presentation blue stuff = “external” links — i.e., URLs to pages on the Internet red stuff (sometimes) = new material, not in your printed copy of materials

Publication Details, and General Disclaimer This “Extreme Project Management” presentation is an open-content collaborative document. Anyone with an Internet connection and World Wide Web browser may view and/or alter its content — for better or worse. Please be advised that nothing in this document has necessarily been reviewed by Ed Yourdon ("Ed"); the theories and business practices expressed by the “Extreme Project Management” document are not necessarily his. This isnt to say you wont find valuable and accurate information herein; however, Ed cannot summarily guarantee the validity of this “Extreme Project Management” document. The content of any given page may recently have been changed, dumbed-down, or other wise edited by someone whose opinion does not correspond to Ed’s original “Extreme Project Management” material (or any subsequent drafts). Neither Ed, nor any of the contributors, collaborators, nor anyone else connected with this “Extreme Project Management” document, can in any way whatsoever be held responsible for the appearance of any inaccurate information, or for your use of the information contained in or linked from this document. You are being granted a limited license to copy anything from this document; it does not create or imply any contractual or extra-contractual liability on the part of Ed, nor any of the contributors, collaborators, or viewers of this material. There is no agreement or understanding bet ween you and Ed regarding your use or modification of this information beyond the GNU Free Documentation License (GFDL); neither is Ed responsible should someone change, edit, modify, or remove any information that you may post on this “Extreme Project Management” document. Any of the trademarks, service marks, collective marks, design rights, personality rights, or similar rights that are mentioned, used, or cited in this “Extreme Project Management” document are the property of their respective owners. Their use here does not imply that you may use them for any purpose other than for the same or similar informational use — as recognized under the GFDL licensing scheme. Unless other wise stated, Ed and this “Agile Project Management” document are neither endorsed by nor affiliated with any of the holders of any such rights; as such, Ed cannot grant any rights to use any other wise protected materials. Your use of any such or similar incorporated property is at your own risk.Published under the GNU Free Documentation License (GFDL) 2

Preliminary details Introduction Schedule • 09:30 - Begin • 11:00 - Coffee break • 13:00 - Lunch • 14:00 - Resume • 15:30 - Coffee break • 17:00 - Finish Seminar objectives Class introduction My background... Exercises/workshops Updated PDF version of seminar materials will be available at end of seminarPublished under the GNU Free Documentation License (GFDL) 3

Preliminary Introduction Please tell us what organization you are from... ... and whether you are currently involved in an “extreme” IT project (if so, what kind of project is it?) ... and the main thing that you would like to learn from this seminarPublished under the GNU Free Documentation License (GFDL) 4

AgendaIntroductionQuick summary: what is an extreme project?Project PoliticsProject NegotiationsPeopleware IssuesSoft ware ProcessesMonitoring and Controlling ProgressLanguages, Tools, and TechnologyAn over view of system dynamics modeling 5

What is an “extreme” project?Almost always: Significant schedulepressure – project must be finished in farless than “nominal” time.Often: Staffing shortages – project mustbe done with significantly fewer peoplethan in a “normal” projectSometimes: budget limitations –inadequate funding to pay for people,development tools, and other resources.Inevitably: greater risks (>50% chance offailure), more pressure, unpleasant politicsAlmost always: heavy overtime (more thanjust 10-20% extra effort), personalsacrifices, increased burnout, higherturnover, lower productivity, lower qualityIncreasingly often: significant corporateconsequences (e.g., bankruptcy), lawsuits,personal legal liabilities 6

Your next assignment “Give me an estimate for the XYZ system. I think it will take… 6 months 5 people €5,000,000 Photo courtesy of Patrick Q ,published via Creative Commons I need the estimate in one hour, for my meeting with the budget committee.” 7

Your assessment “I think it will take… 12 months 10 people €10,000,000 …but I really need more time for a careful estimate!”Photo courtesy of Andres Thor,published via Creative Commons Question for discussion: what if this scenario occurs on every project? 8

Dilbert’s perspective 9

Managing extreme projects (V1)Not: tyrannical behavior (which rarely works anyway, and wont be toleratedmore than once if people can vote with their feet)Not: Charismatic, visionary leadership — though it may be sufficient toaccomplish the task.Compatible with, but not the same as: extreme programming (XP), agilemethods, rapid application development (RAD), etc.An appreciation that time is the most precious resource Avoiding the squandering of time (usually raises political issues) Requires appreciation of the dynamics of soft ware processes Must help project team members manage their time effectivelyUsually a combination of: Sav vy "political" behavior — including the absolute necessity of breaking some rules!! Skillful negotiation of deadlines, budget, resources Appropriate “peopleware” strategies Appropriate system development processes (SEI-CMM, XP, agile, etc) Monitoring and controlling real progress on a day-by-day basis Appropriate use of development tools & technologies 10

1. INTRODUCTION : What’s so different about extreme projects?Users and managers are becoming ever more demanding. Many of todaysprojects are defined as “mission-critical”Many extreme projects require BPR to succeed – just like the early days ofclient-server development projects, during which we learned that 80% ofBPR projects were failures.Peopleware issues are often exacerbated – shortage of skills, decreasingcompany loyalty, inclusion of people with no formal IT background (“testing?what’s that?”), employee burnoutPace of business demands ever-faster implementation. Extreme projectschedules tempt project teams to abandon all “process”“External” extreme projects are often exposed to much greater risks thanbefore – e.g., performance problems, reliability, security, privacy, threats oflawsuits, etc.New technologies are emerging ever faster – e.g., mobile/social technologies,embedded systems, new development environments, etc. 11

1.1 Soft ware Project Outcomes Early On-time Delayed Canceled Sum 1 FP 14.68% 83.16% 1.92% 0.25% 100.00% 10 FP 11.08% 81.25% 5.67% 2.00% 100.00% 100 FP 6.06% 74.77% 11.83% 7.33% 100.00% 1,000 FP 1.24% 60.76% 17.67% 20.33% 100.00% 10,000 FP 0.14% 28.03% 23.83% 48.00% 100.00%100,000 FP 0.00% 13.67% 21.33% 65.00% 100.00% Average 5.53% 56.94% 13.71% 23.82% 100.00% From Capers Jones, Patterns of Soft ware Systems Failure and Success (International Thomson Computer Press, 1996) 12

Soft ware Planned and Actual Schedules in Calendar Months Minimum Maximum Actual Avg. Estimate Variance Percent 1 FP 0.06 0.40 0.16 0.15 0.01 6.25% 10 FP 0.35 2.35 1.07 1.00 0.07 6.54% 100 FP 3.60 19.00 10.00 9.00 1.00 10.00% 1,000 FP 12.24 43.52 27.20 22.00 5.20 19.12% 10,000 FP 24.90 84.66 49.80 36.00 13.80 27.71%100,000 FP 44.28 132.84 73.80 48.00 25.80 34.96% Average 14.24 4713 . 27.01 19.36 7.65 1743% .From Capers Jones, Patterns of Soft wareSystems Failure and Success (InternationalThomson Computer Press, 1996) 13

1.2 the quality issueAverage error rate: approximately 1 defect/function point,afterdelivery to user.Variation in error rate: 100X across industry.One reaction to these gloomy statistics: developing new ITtechnologies, formal methods, and management approaches toenable “zero-defect” IT systemsAnother approach being hotly debated in U.S. today: “goodenough” soft ware, and “good enough” hardware. Examples:Windows 3.1, maybe Windows 2000 and even Vista – and theWeb, which is always “slightly broken” and Web-based productsthat are in “perpetual beta” (though Google no longer says so)This leads to related debates bet ween various “processparadigms” – e.g., the SEI approach versus the “thin method/fatskill” approach exemplified by Kent Beck’s XP approach (see hiseXtreme Programming eXplained) 14

1.2, continued: Typical IT Quality statistics Defect origin Defect potential/FP Removal efficiency Delivered defects/FPRequirements 1.00 77% 0.23 Design 1.25 85% 0.19 Coding 1.75 95% 0.09Documentation 0.60 80% 0.12 Bad fixes 0.40 70% 0.12 TOTAL 5.00 85% 0.75from Estimating Soft ware Costs, 2nd edition, byCapers Jones, 2007 15

2. PROJECT POLITICSKey questions for extreme projectsIdentifying owners, customers, shareholders, andstakeholders in a extreme soft ware projectStakeholder disagreementDetermining the basic nature of the projectLevels of commitmentA new idea: prediction markets & transparency 16

2.1 Key questionsWhat are the realistic chances for success at various stages? At the very beginning of the project At the end of various life-cycle activities: analysis, design, "code complete" Half way through the project schedule A month (or a week, or a day) before the deadline Question: who has an opinion about chances for success at all of these stages? (Google’s “I told you so” project suggests that this is an important question!)When should it become evident to a reasonably intelligent observer/participant that the project is in trouble?Who is the person who should know whether theres a realistic chance ofsuccess, or whether there are serious problems? The project manager The user/customer Senior management Others — who might these be?What should a reasonably intelligent observer/participant do when itbecomes obvious that the project is in trouble? Work 24 hours a day to “save” the project? Escalate the problem to higher levels of management? Gather together key stakeholders to find a consensus solution? 17

2.2 Identifying the playersKey point: your chances of success in a extreme project are often zero if youdon’t know who the key players areAlso crucial that everyone on the project understands this – not just theproject managerTypical players: Owner: who pays for, or “ accepts” the system? Customer: who will actually use the system? Shareholder: one of many “co-owners” Stakeholder: someone whose interests are affected by the success/failure of the project, and who can affect the outcome of the project – and who may thus become an ally or obstacle to project success. See “Project Clarity Through Stakeholder Analysis,” by Larry Smith, Crosstalk: The Journal of Defense Soft ware Engineering, Dec 2000 The "loser user" (more about this in a moment)Note: these roles may not be obvious, and they may shift during the course ofthe project 18

2.2 continued: The loser userWho is the person(s) whose interests would be harmed/decreased if theproject succeeds?Typical negative impact: power, prestige, compensation, convenience, etcTypical examples of loser-users: The vendor whose legacy system will be replaced by new system The end-users who will lose their jobs when the new system is installed The end-users who are perfectly happy with their existing system, and who believe that the new system will be less functional or convenient The end-users who worry they may not have the skills required to use the new system The manager whose budget will be reduced in order to provide funding for the development of the new system The manager who believes that the project is so ambitious that it will fail – and who is determined to make his prediction come true! The manager whose influence, prestige, power, and overall "empire" will be diminished if the new project succeeds.Like other roles, this may change during the course of the projectWe need to remember this when doing risk management! 19

2.3: Stakeholder disagreementAn observation from Tom DeMarco: incomplete,ambiguous specifications are often a sign ofirreconcilable disagreements among stakeholders,rather than incompetence on the part of systemsanalystsIn addition, these disagreements invariably causesignificant project delays, which imperils the deadline.One solution that project manager should try very hardto implement: JAD sessions. (see Joint ApplicationDevelopment for good discussion of techniques.)If thats not possible, try to get a commitment fromstakeholders that all issues requiring their approval,consent, or review will be resolved in no more than 24hours. For a good example of this, see discussion of“extreme soft ware engineering” in the 100-day projectscarried out by Tom Loves ShouldersCorp. 20

2.3, continued: more on 24-hour turnaroundRemember: time is likely to be your most preciousresourceKey political strategy: document and publish day-for-day slippage when key decisions are delayed.Crucial to recognize the political nature of thisbehavior, and the fact that its a response to apolitical phenomenon.If this behavior is declared politically unacceptable,then its nevertheless crucial to identify it as a keyrisk factor for the overall project"The surest poison is time." Ralph Waldo Emerson 21

2.4 Determining the basic nature of the extreme project Typical examples: mission impossible: if we succeed, we live happily ever after kamikaze: the project may succeed, but it will kill all of us ugly: the project manager is prepared to sacrifice any and all of the team members in order to succeed. suicide: the project has no chance of success, and we’re the scapegoats Remember that public assurances from executive sponsor may not reflect the “reality” of the situation Also remember that the situation can change dynamically, based on politics, external business conditions, government regulations, etc. 22

2.4 Determining the Basic Nature of the Project (V1)hap mission kamikazep impossibleine “Ugly”s suicides chances of success Key point: get the project team members to indicate where they think they fit into this grid. 23

2.5 Levels of Commitment The parable of the chicken and the pig Remember that different members of the “key players” have different levels of commitment... ... some of which may be publicly stated, and some of which may not ... and all of this may change on a daily basis A reminder: one of the primary reasons we succeeded with Y2K is that (for the first time!) we actually had substantial commitment from the CEO and the Board of Directors 24

2.6 Prediction Markets and TransparencyA metaphor: abused children, abusedspouses, abused project managersWisdom of the crowdsPrediction marketsTransparency 25

2.6 (continued) A metaphor... photo courtesy of Flickr member E. Schulz, via Creative Commons license 26

2.6 (continued) A metaphor... Note: bruises shown here were Photoshopped by photographer. Flickr page has comments from the model who was photographed. Photo courtesy of a 17 year old high-school student and Flickr member, Lin Pernille, via Creative Commons license 27

2.6 (continued) A metaphor... photo courtesy of Flickr member gxdoyle, via Creative Commons license 28

2.6. Wisdom of crowdsRefers to the process of taking into account the collective opinion of agroup of individuals, rather than a single expert, to answer a question orprovide informationOne popular example, involving collecting/gathering of information:WikipediaAnother popular example, involving non-experts to determine whatnews is important, so that people outside the group can view the newsbased on those ratings: DiggBut also used to for estimating/predicting outcomes and results (e.g.,elections winners, likelihood of achieving a deadline/schedule, value ofcoins in a jar, etc). Remember: such events may change dynamically!Works best when crowd has: diversity, independence, decentralization,and ability to aggregate judgments/opinions.Group does not have to be cohesive; members do not even have to knoweach other.Failures more likely if crowd exhibits: homogeneity, centralization,division (lack of access to other’s info), imitation, emotionality 29

2.6. Prediction marketsBasic concept: let “wisdom of the crowds” provide assessmentof likelihood of achieving deadlines and/or otherorganizational goalsCurrently being studied/researched by Google, Microsoft, andother soft ware/IT organizationsSee “Using Prediction Markets to Support IT ProjectManagement,” by Herbert Remidez and Curt JoslinBasic idea: assign a value of, say, $1.00, to the achievement ofa specific goal (e.g., meeting a deadline)And then let participants buy and sell “shares” of stock in afictional company trying to achieve that goalIf average price is $0.55, then there the “crowd” is sayingthere is a 55% chance of achieving that goal.A tool to consider: Inkling Markets (see case study fromlarge telecommunications company) 30

Details on Inkling Markets Spoke recently to Adam Siegel, adam@inklingmarkets.com Hosted version of ser vice uses U.S.-based hosting vendor — not acceptable to many European IT organizations Hosted version of ser vice costs $6 per user per month, with minimum of 250 users. Volume discounts for larger groups “Installed” ser vice (on your ser vers, behind your own firewall) is $45,000 plus $4 per user per month, min 250 ppl 45-day free demo lets you try it at no risk “Public” Web-based access is free, and you can create a “private group” to try out the concept within your organization 31

2.6. TransparencyOnce its basic usefulness has been validatedand confirmed, a prediction market obviouslyworks best if it is public, visible, and“transparent”But that might not be popular in a highlypolitical environmentConsider setting up an “ anonymous” predictionmarket, using Facebook, Google+, or otherpublicly-available social-media tools 32

3. PROJECT NEGOTIATIONS Managing project definition at the beginning of the project Using project definition to manage requirements creep Estimating techniques Tools for assisting estimation process Tradeoffs bet ween schedule, budget, staff, quality Tools for rational negotiation Documenting political issues and decisions What to do when rational communications are impossible 33

3.1 Managing Project Definition: What does “success” mean? (v1)Many projects succeed/fail at the very beginning, before any tech. work is done.Fundamental requirement: identifying who has the right to declare “success” – owner,shareholder, etc, etc.Typical definitions of “success” finishing on time, or at least "reasonably close" to on-time staying within budget, or at least not too far above the budget delivering the required functionality, or at least a tolerable amount of functionality providing “good enough” level of quality, in terms of "show-stopper" bugs Providing adequate efficiency/capacity/performance to get the work done getting the next round of VC funding, or launching the IPOKey indicators of a doomed project: Nobody can articulate what “success” means for this project… …or it’s such a vague definition that it will be hard to demonstrate success Key stakeholders cannot reach agreement on definition of successThe combination of these constraints may prove impossible to achieve – so the pragmaticaspect of success often depends on agreement as to which areas can be compromised orsatisfied.Biggest risk: lack of realistic triage at beginning of projectFor more details, see my articles, “Spelling Success,” Computer world; and “Long-Term Thinking,”Computer world; and “The Value of Triage,” Computer world 34

3.2 "Good Enough"General concepts Familiar concepts of Pareto Principle (the “80-20 rule”) are often ignored Remember: "Triage" is not the same as "prioritization" Early triage is crucial Documentation of good-enough standards is crucialFunctionality Critical: without these functions, system cant be used at all Important: without these functions, there will be substantial cost/problems Desirable: without these functions, users will whine and complainQuality Defects by severity Mean Time To Repair (MTTR) by severity level Credible evidence of “convergence” to an acceptable level of qualityPerformance (response time, throughput, volume, capacity, etc.) Critical: failing this level means that daily/weekly workload cant be done Important: failing this level means significant loss of productivity/capacity Desirable: failing this level creates a noticeable, annoying impact 35

3.2 "Good Enough"General concepts Familiar concepts of Pareto Principle (the “80-20 rule”) are often ignored Remember: "Triage" is not the same as "prioritization" Early triage is crucial Documentation of good-enough standards is crucialFunctionality Critical: without these functions, system cant be used at all Important: without these functions, there will be substantial cost/problems Desirable: without these functions, users will whine and complainQuality Defects by severity Mean Time To Repair (MTTR) by severity level Credible evidence of “convergence” to an acceptable level of qualityPerformance (response time, throughput, volume, capacity, etc.) Critical: failing this level means that daily/weekly workload cant be done Important: failing this level means significant loss of productivity/capacity Desirable: failing this level creates a noticeable, annoying impact 36

3.3 Using Project Definition to Manage Requirements Creep Typical behavior in projects: new requirements are added at the rate of 1% per month Requirements “creep” (or “scope creep”) and requirements “churn” are a major element of project management risk – particularly in extreme projects that often involve fundamental changes to business processes. But if you don’t have a formal document describing the requirements, it’s hard to identify creep or churn. Assuming that you do have such a document, you need to use it to negotiate schedule/budget/staff modifications if the requirements change or increase. Biggest risk of all: an ambiguous spec is usually a sign of unresolved conflict bet ween diverse political camps in the user community. Related risk: techies assume that it’s their fault they can’t understand ambiguous spec. For more about this, see my Nov 2000 Computer world article, “How to Help Users Write Good Requirements Statements” 37

3.4 Estimating:Introduction What is estimation? The calculated approximation of a result (e.g., a completed deliverable), even if the input data is incomplete or uncertain Estimation is not the same as negotiation Estimation is not the same as a political demand/ decree that a project be finished on a particular date Traditionally, there have been many, many problems with the way estimating has been carried out in IT development projects...Published under the GNU Free Documentation License (GFDL) 38

Problems with traditional estimation The estimate is actually a thinly-disguised acceptance of a political demand for a specific completion date The estimate is the result of a negotiating process analogous to haggling over the price of a rug in a Middle East bazaar The estimate is carried out by people with no experience in the details of the work to be done — so it’s just a guess The estimate is carried out by people who have a vested interest in achieving a desired result, and/or who are susceptible to persuasion and “group-think” Even if it’s acknowledged that the initial estimate is only approximate, the “cone of uncertainty” is ignored — re-estimating is forbidden. Note: all of this has been well understood in the IT field for 30 years or more — but we have rarely confronted the problem directly and chosen a different approach And there’s no guarantee that it will “magically” change just because someone says “we’re agile now!”Published under the GNU Free Documentation License (GFDL) 39

“Estimate” is actually a “guess” It’s easy to estimate many soft ware development tasks within a factor of ten e.g., from a brief description, you can probably estimate whether it’s more likely to require a day, a week, a month, or a year to finish But a more precise estimate almost always requires detailed information about the work to be done, AND... Accurate “metrics” about previous projects carried out by this team, and/or its individual members. In the absence of such information, the so-called “estimate” is more likely just a rough “guess”Published under the GNU Free Documentation License (GFDL) 40

Estimates involve a vested interest, optimism, group-think People making an estimate are likely to want to “please” their boss or the customer... And they obviously know that the boss/customer would prefer to have the project performed faster and cheaper Many IT professionals are instinctively optimistic, and their estimates reflect their optimism (especially about testing!) It’s also common to overlook unpleasant tasks like documentation, training When multiple people discuss the reasons and rationale for their estimates, it’s common for other people on the team to get persuaded by their optimismPublished under the GNU Free Documentation License (GFDL) 41

The “cone of uncertainty” Note the 4x range of uncertainty at beginning of project First identified by Barry Boehm in 1981; also validated by NASA Soft ware Engineering Lab in 1990. “Cone of uncertainty” term introduced by Steve McConnell in Soft ware Project Survival Guide (Microsoft Press, 1997)Published under the GNU Free Documentation License (GFDL) 42

Agile Estimating Initial estimate of entire project often necessary as a result of “envisioning” activity Key point: envisioning activity takes hours or days, not weeks or months Fundamental concept: accept the reality that the initial estimate is likely to be quite inaccurate Take advantage of “wisdom of the crowd” Use techniques to reduce the likelihood of “group-think” Revise estimates after every iteration/version/sprint Use previous “actuals” to guide future estimates Resist temptation to promise future miracles/overtime to compensate for past shortfallsPublished under the GNU Free Documentation License (GFDL) 43

Estimating at the appropriate level of detail: user stories Begin by working with user to describe system functionality in terms of user stories Short description in a sentence or t wo (fits on a note card) Brevity encourages subsequent changes, add/deletes Expressed in everyday business language of the business user, describing something the user wants to achieve Typical form: as a <role>, I want <goal/desire> so that <benefit> Becomes formalized when (a) user describes acceptance criteria, and (b) developer agrees to story+acceptancePublished under the GNU Free Documentation License (GFDL) 44

Estimating user stories Start by getting team consensus on whether each user story is “super hard,” “hard,” “medium,” “easy,” or “super easy” Alternatively, assign a number of “story points” to each user story, e.g., a number from one to five An entire user story typically requires 1-3 weeks of “ideal development time” — i.e., no distractions, no other work to do, and you understand what it is you have to do Less than one week typically means you can combine some stories Longer than 3 weeks typically means you should break a user story into smaller piecesPublished under the GNU Free Documentation License (GFDL) 45

Sprints and releases Sprints should be frequent enough to keep the user involved and able to see actual, tangible progress by the team ... but also far enough apart so that “meaningful” progress can be shown, rather than just one tiny new feature Sprints are typically 1-2 weeks long in early stage of project But sometimes as frequent as 1-2 days near project end A sprint does not necessarily result in a “release” of a system to user community for actual productive use. A “release” typically has approx 80 ± 20 user stories A “release” often requires additional training, documentation, QA, distribution, marketing, and other administrative activities Hence a “release” might occur as frequently as weekly, but might also occur once every 2-3 months.Published under the GNU Free Documentation License (GFDL) 46

Avoiding “group-think”: Planning Poker A relatively popular approach for consensus-based estimating of development effort for user stories Initially described by James Grenning in 2002, popularized by Mike Cohn in 2005 in Agile Estimating and Planning; see also Wikipedia article Uses a deck of cards (one for each participant), in which the cards have Fibonacci sequence numbers: 0, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, etc Fibonacci sequence reflects increasing uncertainty of large estimates One commercially-available product uses 0, 1/2, 1, 2, 3, 5, 8, 13, 20, 40, 100 Another uses standard playing cards with Ace, 2, 3, 5, 8, and King (“too big and complicated to estimate”)Published under the GNU Free Documentation License (GFDL) 47

Planning Poker Procedure A moderator chairs the meeting, supported by the project manager The most knowledgeable developer for a specific features provides an overview; the team can ask questions and discuss to clarify assumptions and risks. (Project manager documents a summary of this discussion.) Each individual places a card, face down, representing his estimate (in days, “ideal days,” story points, etc.) Everyone turns over their cards simultaneously Persons with high and low estimates are given a short opportunity to explain the basis for their estimates; short discussion then allowed Repeat the process until consensus is reached The developer who is likely to “own” the actual implementation may get a larger vote in the consensus activity; moderator can also negotiate the detailsPublished under the GNU Free Documentation License (GFDL) 48

Planning Poker Example From “Planning Poker Estimation” on YouTubePublished under the GNU Free Documentation License (GFDL) 49

Estimating TechniquesFor complex projects, get a commercial estimating toolFundamental truth: to estimate a project you need metrics from previous projects.Most IT organizations have almost no metrics about previous extreme projects.Thus, most of what’s described as “estimating” is either “guessing” or“negotiating” (see “Metrics and the Seven Elements of Negotiation,” by MichaelMah, IT Metrics Strategies, April 2001)Political reality: estimates are produced by people with little prior estimatingexperience, and who have a vested interest in believing their optimisticpredictionsA radical suggestion: create a separate estimating group whose work is judged andrewarded by accuracy of its estimates, not the political acceptability of estimatesFor a new approach to estimating, see “critical chain” approach, which “pools”safety estimates across several tasks within a project.Also, try the concept of “prediction markets” 50

3.4 continued: more estimating guidelinesAvoid single-person estimation – estimates should be undertaken by more than one person, andpreferably in a team-based process.Always complete a risk assessment prior to estimation – any estimate undertaken without a formalrisk assessment is almost certain to be incorrect.Be sure you have taken into account the “Top 10 Project Factors” (from Assessment and Control ofSoft ware Risks, by Capers Jones): Team skills and experience Level and quality of executive sponsorship Level and quality of stakeholder "buy-in" System or product size System or product complexity Required product quality Development tools and environment Level of innovation required Product requirement stability Schedule constraint (sometimes known as schedule compression)Where possible, use relevant (unbiased, uninvolved) experts – who can give you a confirmation or awarning about the "reasonableness" of your estimatesAlways carefully document estimating assumptions – especially because some of these assumptions willbe based on political "assurances" that are never written down, and subject to change.Review the work breakdown structures – to make sure you have not overlooked such project tasks aseducation, documentation, etc. 51

3.4, continued: Additionalstrategies for estimating System dynamics models, e.g., Tarek Abdel-Hamid’s model in iThink (more on this later in the seminar) Copies of The Mythical Man-Month for all concerned …or copies of Tom DeMarco’s The Deadline for those who prefer a less serious book. Estimating tools, such as COCOMO-2; also, COSTAR and SLIM (Michael Mah) and SPR-20 and SEER- SEM (see YouTube video) Time-boxing to see how feasible/infeasible the project constraints really are 52

3.5 Tradeoffs bet ween schedule, budget, functionality, staff, quality Key point: it’s not a linear tradeoff – see Fred Brooks, The Mythical Man-Month (Addison-Wesley, 1995) Relationship is a non-linear, third-order polynomial relationship – see Larry Putnam and Ware Myers, Measures for Excellence: Reliable Soft ware on Time, Within Budget (Prentice-Hall, 1992) Biggest risk: tradeoffs are usually negotiated, under pressure, late in the project schedule – without accepting the non-linear tradeoffs ... ... and without accepting the reality that much of the partially- finished work will be lost forever To negotiate tradeoffs rationally, you need to have one of the estimating packages mentioned earlier For more details, see “Internet-Speed Deadline Management: Negotiating the Three-Headed Dragon,” by Michael Mah, IT Metrics Strategies, May 2000 53

3.6 Project Negotiations Even if the boss’ “estimate” seems reasonable, it’s still a political demand The boss has almost certainly not spoken with the users about their detailed requirements So he has no basis at all for the “estimate” he has given you More likely, his “estimate” is based on a similar dialogue with someone else at the higher level of executive politics where he spends his time In any case, he is creating an environment in which you know that the most you can hope for is an “incremental” change in the proposed schedule or budgetPublished under the GNU Free Documentation License (GFDL) 54

3.6, continued Negotiating games Doubling and add some... Reverse doubling Guess the Number I’m Thinking of... Double Dummy Spit The X-Plus Game Spanish Inquisition Low Bid Gotcha – throwing good money after bad Chinese Water Torture Smoke and Mirrors/Blinding with Science thanks to Rob Thomsett, for his “Estimating Games” Web article 55

3.6 Negotiating strategiesDon’t get tricked into making an “instant estimate” – ask fortime to think about (a week, a day, even an hour)State the estimate in terms of confidence levels, or ± ranges, etc.Jim McCarthy (formerly of Microsoft, author of Dynamics ofSoft ware Development): make the customer, or other members ofthe organization, share some of the uncertainty.Project manager: “I don’t know precisely when we’ll finish – butI’m more likely to be able to figure it out than anyone else in theorganization. I promise that as soon as I have a more preciseestimate, I’ll tell you right away.”Do some reading and research to become better at this area, e.g.: Bargaining for Advantage: Negotiating Strategies for Reasonable People, by G. Richard Shell (reissue edition, Penguin Books, June 2000) Getting Past No: Negotiating Your Way from Confrontation to Cooperation, by William Ury (Bantam Doubleday Dell, 1993) The Prince, by Niccolo Machiavelli (Viking Penguin, 1999) 56

3.7 Documenting political items Document all of these key issues, as part of the permanent record – other wise, youre likely to find that people “forget” unpleasant realities that you told them about when the project began Or even worse, the people who made commitments to you about key political issues may have quit, died, retired, been fired, or gotten run over by a crazy Italian motorcycle driver. Key things to document (probably in a “project plan” document) Background of the project (why is this project being carried out?) Scope (what to leave in, what to leave out; use a context diagram for simplicity) Technical assumptions Technical dependencies Constraints Project approach (what kind of development process will be used) Quality approach Development plan (resource schedule, project schedule, release schedule) 57

3.8 What to do when rational negotiation breaks downRemember Nancy Reagans advice: “Just say no!” Would you commit to running a marathon in one hour?Quit (the project or the company)Appeal to a higher authorityGo see the movie Gladiator, and learn to say, like Russell Crowe, “We who areabout to die salute you!”Decide which “rules” you’re going to break in order to achieve an “irrational”set of schedule/resource demands that have been imposed upon you.Redefine the project as a kamikaze, suicide, etc., and make sure entire projectteam knows it.Key point: project leader has to believe in the possibility of achieving projectgoals... and must be able to convince team members without lying to them 58

A comment from Napoleon“It follows that any commander in chief who undertakesto carry out a plan which he considers defective is atfault; he must put forth his reasons, insist on the planbeing changed, and finally tender his resignation ratherthan be the instrument of his army’s downfall.” Napoleon, Military Maxims and Thoughts; see also The Military Maxims of Napoleon, by William E. Cairnes (De Capo Press, 1995) 59

4. PEOPLEWARE ISSUESHiring and staffing issuesRecruitingIdentifying loyalty and commitment issuesImportance of communicating projectprioritiesTeam-building issues 60

4.1 Hiring and Staffing Issues Strategy #1: hire superstars and turn them loose (Doesn’t always work: see Dreaming in Code: t wo dozen programmers, three years, 4,732 bugs, and one quest for transcendent soft ware) Strategy #2: insist on a well-honed “dream team” that has worked together before. (Ideal solution, but unlikely, and probably unrealistic to expect.) Strategy #3: choose mere mortals, but make sure they know the risks and rewards associated with the project Strategy #4: take whoever you’re given and convert them into a mission-impossible team (e.g., the Dirty Dozen approach) 61

4.1 Hiring and Staffing, continuedRisk increases substantially if project manager cannotchoose his/her team members …… or at least veto the assignment of totallyunacceptable morons to his/her team.Crucial to avoid losing people during the project; highlydesirable not to add new people during projectThis is another area where the project manager may beable to “break the rules” to minimize problems. 62

4.2 The Importance of RecruitingDon’t let HR department dominate.Offer bonuses to existing staff members who can recommend new peopleDevelop a good relationship with local universities who may be a “spawningground” for new peopleSend top IT managers to university campus to recruit new graduatesReengineer your process for recruiting Advertise job openings on the Web Allow prospective people to send e-mail inquiries via the Web Collect resumes on the Web Interview (with video hookup) via the WebRecruit continuously 63

The Audition Approach Complements inter views, reference checks, etc. Ask candidate to give oral presentation to a group of technicians, managers, and non-technical people — especially other members of the agile team!!! Good presentation topic: the candidate’s last project. Alternative: ask them to write some code, develop a test case, design a database schema, etc. Ask them to drink a “shot” of alcohol after every 10 lines of code, as in The Social Net work (just kidding) When finished, get a consensus from the group: yes, no, or “maybe.” Don’t overlook the social impact: passing the audition is a “certification” process; failing the audition improves the morale of the existing staff members.Published under the GNU Free Documentation License (GFDL) 64

Audition ChecklistEnergy levelAbility to communicate in wholesentencesAbility to answer questionsA sense of excitement/passionabout his/her work 65

4.3 Identifying Loyalty and CommitmentIdeal case: loyalty to project comes before all else.Often associated with young, unmarried techno-nerds,with no life outside the officeMay also depend on charisma of project leaderAlso depends on length of project: total devotion isfeasible for 3-6 month extreme project, but not ≥ 24monthsCrucial that everyone knows what the loyalty issues are– e.g., “If my spouse threatens to divorce me, then I’ll haveto quit this project.” 66

Motivating Soft ware People “There is nothing more discouraging to any worker than the sense that his own motivation is inadequate and has to be ‘s upplemented’ by that of the boss... You seldom need to take Draconian measures to keep your people working; most of them love their work.” Tom DeMarco and Tim Lister, Peopleware 67

The Impact of Money“Money, benefits, comfort, and so on are “hygiene” factors – theycreate dissatisfaction if theyre absent, but they dont make peoplefeel good about their jobs and give them the needed internalgenerator. What does produce the generator are recognition ofachievement, pride in doing a good job, more responsibility,advancement, and personal growth. The secret is job enrichment.” Frederick Herzberg, “One More Time: How Do You Motivate Employees?” Har vard Business Review, Sep-Oct 1987 68

More about moneyOvertime and bonuses should be based on work carried out; bewaredangling the lure of “successful completion” in front of the projectteam, because they may not be able to control success of theprojectVarious other quasi-monetary and non-monetary rewards maysometimes be just as valuable as overtime pay (e.g., ability to keepone of the development workstations at the end of the project)Project manager needs to have a budget item for late-night meal,Friday-night pasta & wine, etc. Don’t “nickel-dime” the project teamin this area! Microsoft refers to this as a “morale” budgetAnother example: concierge services such as Circles, LesConcierge,Peapod, Webvan Group for chores that programmers don’t havetime to do themselves. (Alas, several companies in this area havegone bankrupt!) 69

Rewards for Exemplary Performance Improvement: Attendance at conferences Long-term improvement: pay for graduate degree @ university Respect: Lunch with the senior manager Respect: let them give “best practice” presentations to other employees in the organization ... or write a blog, etc. Awards: “distinguished employee,” “employee of the month”, etc. Allow them to become employees of the vendor organization More? 70

Breakthrough Performances With the right conditions, “breakthroughs” in productivity are possible. Solution is unknown when objective is established – e.g., JFK’s 1961 statement that the U.S. would put a man on the moon within a decade (and bring him back alive). Breakthroughs occur randomly (and rarely) in large shops, but are more common in small soft ware organizations. Possible computer-industry examples: Microsoft’s Windows-NT project, DEC’s Alpha project, etc. Key question: What is required to deliberately instigate breakthrough performances in a large shop? See Reinventing Strategy: Using Strategic Learning to Create and Sustain Breakthrough Performance, by Willie Pietersen (Wiley, 2002) 71

Identifying an extraordinary effort Must clearly define breakthrough project: one that meets a deadline or achieves a result previously thought unlikely, impractical, or even impossible. Exceeds what’s predictable or expected based on existing criteria for measurement. Identifying characteristic: commitment to achieving a breakthrough without knowing how it will be done. 72

Identifying an extraordinary effort Key factor in defining a breakthrough is time and effort invested by both senior and project-level management to assemble the team before start of project to emphasize its importance . 73

Identifying an extraordinary effort Management must communicate to the team, in a credible fashion, why the breakthrough is needed. Team members must understand why a breakthrough is so important – e.g., to avoid bankruptcy ...or even how achieving a successful result can enhance the reputation of the IT department within the organization. 74

Identifying an extraordinary effort This kind of communication rarely occurs – especially in big companies. Often there is no formal kickoff meeting. Managers may not have mastered necessary communication techniques to empower the team with the required sense of mission and to overcome skepticism that this is merely “the company line.” See Steve Jobs’ Apple Computer fall 1983 keynote as an example ... 75

More suggestionsGive team members a choice of whether toparticipate. Just like the Mission Impossible instructions: “Your mission, if you choose to accept it …” Irrelevant if decision not to participate leads to demotion, firing, insults, etc.Look for ways for the team to make acommitment to a visionWatch for breakdowns – frequently lead tobreakthroughs.Recognize and accept the greater degree ofrisk in this approach. 76

“Voluntary” overtimeAn inevitable part of many soft ware projects, especiallyhigh-intensity extreme projectsWorst mistake is not recording it, so that it appears to be“free” and “infinite”Typically easier to accomplish if project team is young,unmarried, and fanaticalAlso easier to accomplish if overall duration of project is 3-6months.Beware the insidious effects of long-term heavy overtime onprojects of 18-24 months, etc. 77

Overtime and net productivity Net production (new work minus lost time for rework) Hours per week 40 60 80 90 100 120 Note that the shape of this curve depends on age, motivation, and length of the overall project 78

4.4 The importance of communications with the team Ideal strategy: the project manager has no secrets Crucial that everyone knows current information on priorities, politics, constraints, risks, etc. Communication bet ween team members is also crucial, especially if members have not worked together before Intra-team communication has to be kept confidential (from outsiders) to encourage honesty and frankness Strongly implies the need for e-mail and probably more sophisticated features – e.g., wikis, blogs, Twitter, discussion forums to keep ongoing “threads” of discussions, etc. Weekly lunch/beer/dinner sessions likely to be crucial, too 79

4.5 Team-Building IssuesQuestion: how would you recognize an effective extremeproject team?Be aware of team roles – e.g., importance of visionary,skeptic, team worker, etc.Consider using Myers-Briggs profiles so people will know howthey’ll react to each other. Especially relevant in extremeprojects, because there is often a mix of IT- and non-ITpersonalities.Watch out for “teamicide” issues often caused by projectstress 80

The team role model: 8 critical rolesChairman (“leader/decision-maker”)Shaper (“visionary”)Plant (“provocateur,” suggests new ideas)Monitor-Evaluator (“skeptic/critic”)Company Worker (“proletariat”)Team Worker (“diplomat”)Resource Investigator (“scavenger”)Completer (“finisher”) 81

Signs of an Effective TeamStrong sense of identity: Team names, common “in-jokes,”common lunch area, etc.Sense of eliteness: Team members feel they are part ofsomething unique.Joint ownership: Participants are pleased to have theirnames grouped together on a product.Obvious enjoyment: Do good work and have fun. (Seemsobvious, but “having fun” is contrary to corporate culturein many organizations!) (See “Pink Glove Dance” for anexample.) 82

TEAMICIDEDefensive management — not trusting the teamBureaucracy — too much paper workPhysical separation of team membersFragmentation of people’s timeQuality reduction of the productPhony deadlinesClique control — splitting up teams(see “Teamicide revisited,” by Tom DeMarco andTim Lister) 83

The impact of workplace on quality Workers who say their workplace is acceptably quiet are 1/3 more likely to deliver zero-defect work. 66% of zero-defect workers report noise-level is OK in their workplace 8% of one-or-more-defect workers report noise level is OK in their workplace Of course, most project managers have no direct control over physical workplace — so this is often an area where you have to break, or redefine, the “official” rules. (More about this in a few minutes.) 84

Best and Worst Performers in the Coding War Games 4th Environmental factor 1st quartile quartileHow much dedicated workspace do you have? 78 sq. ft. 46 sq ft.Is it acceptably quiet? 57% yes 29% yesIs it acceptably private? 62% yes 19% yesCan you silence your phone? 52% yes 10% yesCan you divert your calls? 76% yes 19% yesDo people often interrupt you needlessly? 38% yes 76% yes (from a 600-person coding “war game,” discussed in Tom DeMarco and Tim Lister, Peopleware) 85

Office Space and InterruptionsSome soft ware development work requires intense, uninterruptedperiods of concentration.Typical environment for all of today’s projects: 300“interruptions” per day!Phone calls, etc require a period of “re-immersion” of 5-15 minutes.For a discussion of this, see Tom DeMarco’s book, Slack: Gettingpast burnout, busywork, and the myth of total efficiencyNew info: see “Meet the Life Hackers,” from New York Times, Oct16, 2005. Based on 1,000 hours of observations of t wo WestCoast high-tech firms: “Each employee spent only 11 minutes on any given project before being interrupted and whisked off to do something else. What’s more, each 11-minute task was itself fragmented into even shorter three-minute tasks, like answering e-mail messages … And each time a worker is distracted from a task, it takes 25 minutes, on average, to return to that task.” 86

Breaking Office Space RulesRemember: the bureaucracy often moves much more slowly thanIT development projects do ...By the time the bureaucracy recognizes your actions, andactually does something about it, you might be finished withyour project.Possible strategies: Work at home Work during 2nd shift or weekends Commandeer unused office space Other ideas? 87

5. SOFTWARE PROCESSES“Heavy” vs. “Light/agile/XP” processes vs. Anarchy: don’t let extremeprojects provide the excuse for utter anarchy. (See Building the EmpireState for an example of agile processes for building the Empire StateBuilding.)“Best practice” and “worst practice” concepts: in a extreme project, itmay be more important to avoid disastrous processes than to strivefor perfect processes. (Consider a project “breathalyzer test”)In an extreme project: The dynamics of processes (delays bet ween activities, feedback loops, etc.) are crucial… For minimizing the “cycle time” of the overall process (i.e., the duration from beginning of project to the deadline) So get a copy of iThink, and read Tarek Abdel-Hamid’s classic text, Soft ware Project Dynamics. See also Raymond Madachy’s Soft ware Process Dynamics. A trivial, but deeply important, observation from Barry Boehm: the most important reason projects finish late is that they start late. 88

5.1 A key soft ware process:configuration management (CM) (also known as Change Control) Often not formalized at all Need to indicate which “ artifacts” or “work products” will be subjected to CM – hopefully more than just code! Beware an almost universal phenomenon in extreme projects: At the beginning, everyone agrees that CM is important – and everyone “follows the rules” But then it becomes frustrating to see how much time and energy is required to perform the CM processes in a formal fashion – e.g., documenting changes to requirements, getting formal approval, updating schedules and budgets. And this often becomes unacceptable as time-pressure grows more intense in the later stages of the project … and as a result, CM is gradually abandoned. … if the project ultimately fails, there is often no formal record of who made certain changes, why they were made, who was informed, who approved it, etc. 89

5.2 Systems Dynamics of Processes Requirements Designs Analyzing Designing Coding The traditional view of a familiar soft ware process Linear, “straight-line” progress “Instantaneous” performance of processes 90

5.2 Systems Dynamics of Processes Bad requirements Requirements Designs Analyzing Designing Coding Hence, key priority is to reduce “conveyer-belt” delays associated with processes Thats why JAD technique is so important – get all the stakeholders together at the same time, in order to resolve differences and reach a consensus. And another priority is to reduce the “leakage” associated with errors in the process Thats why prototyping is so important And also why walkthrough/inspections and other QA techniques are so important 91

The Agile Manifesto (Feb 2001) Customer satisfaction by rapid, continuous delivery of useful soft ware Working soft ware is delivered frequently (weeks rather than months) Working soft ware is the principal measure of progress Even late changes in requirements are welcomed Close, daily cooperation bet ween business people and developers Face-to-face conversation is the best form of communication (co- location) Projects are built around motivated individuals, who should be trusted Continuous attention to technical excellence and good design Simplicity Self-organizing teams Regular adaptation to changing circumstancesPublished under the GNU Free Documentation License (GFDL) 92

5.3 SEI vs. “lite” processes Optimizing • Process change mgmt • Technology change mgmt • Defect prevention Managed • Quality management • Quantitative process mgmt Defined • Peer reviews • SW product engineering • Integrated SW management • SW process definition • SW process focus Repeatable • SW configuration management • SW quality assurance • SW subcontract management • SW project tracking & oversight • SW project planning • Requirements management Initial 93

Current SEI-CMM maturity levels See http://www.sei.cmu.edu/cmmi/casestudies/profiles/pdfs/upload/2009SepCMMI.pdf 94

5.3 “Lite” vs. “Heavy” ProcessesFormal (heavy) processes are great if you know what you’re doing...... and if you’ve done the same thing several times beforeSEI-CMM guru Watts Humphrey: “if a process can’t be used in a crisis, it shouldn’tbe used at all.”But many extreme projects involve doing things that have never been done before– with teams that have never worked together before.Conversely, if a team has worked together before, and really “jells”, then it doesn’tneed a formal, heavy processNevertheless, team needs to agree on what processes will be formalized (e.g.,change management, source code control, testing (a la XP)), and what processeswill be done on a completely ad hoc basis.For more details, see “Extreme Programming,” by Jim Highsmith, extreme Application Delivery, Feb 2000. November 2000 issue of Cutter IT Journal on “Light Methodologies” “Put Your Process on a Diet,” by Martin Fowler, Soft ware Development, Dec 2000 “Retiring Lifecycle Dinosaurs,” by Jim Highsmith, Soft ware Testing & Quality Engineering, Jul/Aug 2000 “The Light Touch,” by Ed Yourdon, Computer world, Sep 18, 2000 95

5.3, continued: SEI vs. “lite” processes“Heavy,” SEI-style soft ware processes are desirable(!) fornuclear reactors, air traffic control systems, and other safety-critical systems...But unnecessary, impractical, and even undesirable in manyhigh-pressure extreme projects...What users really want is a system that’s cheap enough, fastenough, feature-rich enough, and available soon enough – i.e.,“good enough”A project team oriented towards “good enough” soft ware hassome very different behaviors and strategies thanconventional project teamsSoft ware managers trained in traditional soft wareengineering approaches have a hard time with this, but SiliconValley lives with it every day… as illustrated by Google andother Web 2.0 companies that practice “perpetual beta” 96

5.3 More on “lite” versus “heavy”Areas where there are differences Degree of documentation Frequency of reviews and approvals Degree of decision-making authority – borrowed from “lean manufacturing” approachExamples of documentation differences: the requirements analysis phase Lite approach: one sentence per requirement Medium approach: one paragraph per requirement Heavy approach: detailed UML models, data dictionary,etc. What happens to requirements when development is done?Criteria for choosing lite vs heavy: Project cost Project duration, and degree of schedule “compression” Staff size Risk assessment – consequences of failure (safety-critical?) 97

Summary of XP practicesShort, 2-week release cycles; “ship early, ship often” Functionality for each new release is based on “conversations” with users This can be documented with UML use cases, etc. if you want more rigor In each new release, you get to choose as much functionality as the team was able to produce in its previous release.Schedule based on a sustainable 40-hour weekDon’t design for the future, make the best possible design for today Old paradigm: requirements errors are extremely costly, hence it’s important to spend as much time as necessary to obtain “correct” requirements from users New paradigm: it’s impossible to obtain correct requirements, so don’t waste your time. Instead, invest those resources in continuously “re-factoring”“Re-factor” the design every day, in tiny little stepsPair programming: t wo people work together on each assignmentCollective ownership of code (anyone can change any code written by any othermember of the team)Testing: programmers write tests before writing code, save tests for subsequentregression testing.Team works in an “open” space (“bullpen”)There is a very active Internet discussion group where these ideas are beingdiscussed/debated by the XP community. Also, see Extreme ProgrammingExplained: Embrace Change by Kent Beck, and Planning Extreme Programming, byKent Beck, Martin Fowler, and Jennifer Kohnke. 98

XP: test-driven development ... it’s a soft ware development process, developed or popularized by Kent Beck (and typically associated with XP variant of agile) involves repetition of a short development cycle: write a failing automated test case that defines a new function (or improvement to existing function) write code that will pass the test re-factor the code to acceptable standards Popular as part of the XP form of agile, but not necessarily used by everyonePublished under the GNU Free Documentation License (GFDL) 99

Key aspects of Scrum Term introduced by Peter DeGrace and Leslie Hulet Stahl in Wicked Problems, Righteous Solutions (YOURDON Press/Prentice Hall, 1990) Popularized by Jeff Sutherland and Ken Schwaber, starting at OOPSLA ’95 conference in 1995. Puts more emphasis on roles than XP Scrum master, who maintains the process (in lieu of project leader) Product owner, who represents stakeholders and business Team members (typically no more than 7) Somewhat more likely to be commercialized, with certification steps Popularized terms such as “sprint,” “sprint backlog,” “daily standup meeting”, “burn-down chart,” etc Emphasizes co-location of team Emphasizes concept of “done” (satisfying exit criteria, usually meaning that all regression tests have run; “done done” when customers have reviewed it, and agree that it’s complete),Published under the GNU Free Documentation License (GFDL) 100

Agile Unified Process Simplified version of the Rational Unified Process (RUP) developed in the late 1990s Developed and popularized by Scott Ambler Emphasizes test-driven development, agile modeling (to replace formalized UML notation), and refactoring Consists of seven disciplines Model (to understand the business and problem domain) Implementation Test Deployment Configuration management Project management (including risk management, directing people, coordinating with people and systems outside the project Environment Identifies t wo types of iterations: Development release iteration (deployment to QA or demo area) Production release iteration (deployment to a production area)Published under the GNU Free Documentation License (GFDL) 101

5.4 The Airlie Council “Principal Best Practices”Formal Risk managementAgreement on InterfacesPeer ReviewsMetric-Based Scheduling and managementBinary Quality Gates at the “Inch-Pebble” LevelProgram-Wide Visibility of Project Plan and Progress Vs. PlanDefect Tracking Against Quality TargetsConfiguration managementPeople-aware management Accountability 102

5.4, cont’d: Worst PracticesDon’t expect schedule compression of ≥10% compared to statistical norm forsimilar projectsDon’t justify new technology by the need for schedule compressionDon’t force customer-specific implementation solutions on the projectDon’t advocate the use of silver bullet approachesDon’t miss an opportunity to move items that are under external control offthe critical pathDon’t bury all project complexity in soft ware as opposed to hardwareDon’t conduct critical system engineering tasks without sufficient soft wareengineering expertiseDon’t expect to achieve an accurate view of project health from a formalreview attended by a large number of unprepared, active reviewersDon’t expect to recover from a schedule slip of more than 10% withoutacknowledging a disproportionately greater reduction in soft warefunctionality to be delivered.For more discussion along the same lines, involving the concept of “ anti-processes,” see Anti-Patterns and Patterns in Soft ware ConfigurationManagement, by William J. Brown, Hays W., Iii McCormick, Scott W. Thomas(Wiley, 1999). 103

6. MONITORING ANDCONTROLLING PROGRESSHelping project team members monitorand control their own progressMeasuring, managing, and controllingprogressMaximizing project life cycle review 104

6.1 Helping Project Team monitor its own use of time Peter Drucker: "Until we can manage time, we manage nothing else" Key objective for management: help your project team distinguish bet ween urgent and important. Best reference: First Things First, by Stephen R. Covey, et al (Fireside, 1996) Project manager should act as a buffer to keep Q3 items from distracting the project team. For more details, see my Apr 23, 2001 Computer world article, “Finding Time to Think.” u r Q3 Q1 g e n c Q4 Q2 y importance 105

6.2. Controlling progressManagement approaches based on classical waterfall approach make itdifficult to gauge real progress on a day-to-day basisNeed some kind of “time-box” approach based on versions, features,deliverables, etc.Jim McCarthy: “Never let aprogrammer disappear into adark room.” (The antithesisof this problem: “visibleprocesses.” )If team understands whatfeatures and dependenciesare required for a milestonecompletion, they will exertpressure upon themselves,rather than having themanager beat themIf you miss one milestone deadline, it’s crucial to succeed on the next one.Milestone post-mortems can be incredibly valuable (but end-of-projectpostmortems are usually a waste of time). 106

6.3 The “daily (nightly) build”Basic concept: every day, you should compile, link, load,execute, test the growing/evolving system/product you’redevelopingPopularized by Dave Cutler at MicrosoftJim McCarthy (former head of Microsoft’s Visual C++project): “The daily build is the heartbeat of the project – it’show you know you’re alive”Should be automated, and performed overnight – or evenmore often.Various “tricks” can be used to increase its effectiveness Punishing people who “break” the daily build Using red-flag/green-flag at office entrance 107

Burn-down charts A burn down chart is a graphical representation of work left to do in a sprint Outstanding work (story points, etc.) on the vertical axis, (calendar) time on horizontal axis. Updated daily, or at end of each “sprint” Useful for predicting when all of the work will be completed. Tool support needed, but many people just use Excel Alternative burn down chart useful if there are frequent scope changesPublished under the GNU Free Documentation License (GFDL) 108

6.4 Maximizing project reviewsFormal reviews at lifecycle milestones are often used to gauge the progressand status of the projectBut project manager must remember that these are often highly politicalevents, in which the real problems and the real status is hidden in order toavoid political repercussions.While you can’t eliminate these formal reviews, consider augmenting themwith informal, peer-level reviews See Soft ware Inspection, by Tom Gilb, Dorothy Graham, and Suzannah Finzi (Addison-Wesley, 1993) for discussion of formal inspections. …or Project Retrospectives: A Handbook for Team Reviews, by Norman Kerth (Dorset House, 2001) for an up-to-date perspective on reviews.Most important: mini post-mortems with the team at the end of each inch-pebble deliverable.Another idea, growing more popular with agile projects: daily “stand-up”meetings.See my Computer world article, “Mini-Postmortems” for more detail.Consider also the use of videotaping project walkthroughs and meetings inorder to prevent future maintenance problems. For more details, see myarticle, “Internet Legacy Nightmares,” Computer world, Jul 31, 2000) 109

Daily standup meetings Also referred to (by some) as the “heartbeat” of agile projects, also known as “morning roll-call” or “daily scrum” Basic idea: daily “status” meeting, held same time and same place (sometimes as a way of starting the day) in which every member of the project team quickly identifies accomplishments since last meeting, the t wo or three main things he/she intends to accomplish today (hence building shared commitment) ... ... and the obstacles/problems/questions that might prevent them from being accomplished. So that the project leader and/or other team members can suggest alternatives and solutions and answers, and/or take their own action to eliminate the obstacles. Not intended as a full presentation/discussion to management (and should not be attended by people not directly involved in the meeting Everyone stands, to maintain focus/attention, and to prevent meetings from running too long (hold the meeting in a room with no chairs)Published under the GNU Free Documentation License (GFDL) 110

Additional daily standup notes and suggestions/guidelinessense isn’t Basic concept is “common sense” — but as Voltaire observed, “common common” Experienced agilistas will recognize an unproductive daily standup meeting, and will generally know how to fix it ... But novices may not understand, may get frustrated, and abandon the concept If more than 10 people in meeting, invoke “Pigs and Chickens” concept If discussion degenerates into “problem-solving,” then invoke “Take if Offline” Use “last arrival speaks first” and “rotate the facilitator” to encourage more “self- organizing” rather than a traditional “the boss runs things” kind of meeting Additional common-sense suggestions to make meetings productive: Everyone needs to be there, on time; but meeting not delayed if someone absent Turn off cell phones, iPods, Blackberrys, laptops, etc, etc. Entire meeting should take only 5-15 minutes Can combine with Skyping or other forms of video/teleconferencing Signal the end of the meeting, to prevent ongoing conversations etc...Published under the GNU Free Documentation License (GFDL) 111

What not to do in standup meetingsPublished under the GNU Free Documentation License (GFDL) 112

Other indicators of problems Reporting to the “leader,” rather than addressing the team Traditional politics: team leader imposes 7 AM Monday meetings as a loyalty test People frequently show up late (may indicate meeting is being held at a bad time/ place) Stand-up meeting starts the day ... late (often, nobody does any work b4 meeting) Observers interrupt Socializing on non-project-related issues (“did you watch the football game?”) “I can’t remember” Story telling Problem solving Low energy Obstacles are not raised (for various reasons) Obstacles are not removed Obstacles are raised only during stand-up meetingsPublished under the GNU Free Documentation License (GFDL) 113

Daily standup references Wikipedia: stand-up meeting “It’s Not Just Standing Up: Patterns of Daily Stand-Up Meetings,” Jason Yip, ThoughtWorks Inc. YouTube: “Agile Chalk-Talk: the daily meeting” YouTube: “Stand-up Meetings” YouTube: “Agile in Practice: Stand-ups” “The Daily Scrum Meeting” “Stand-Up Meeting”Published under the GNU Free Documentation License (GFDL) 114

6.5 Soft ware Risk Management (SRM) Introduction and definition The context of soft ware risk management Critical cultural aspects Likely causes of risk in soft ware projects 115

Changes in Risk Management with today’s IT ProjectsRisks often occur more quicklyThere is less time to resolve/respond to risksRisks are more often associated with multiple failures, but westill plan for single “points of failure”Risks are often “malevolent,” rather than “neutral”We have tools like Twitter to communicate and collaborate aboutrisks as they occurWe realize that it’s important to empower “front-line” people torespond to risks in real time. 116

The Age of Complexity, Uncertainty, Disruption Complexity: today’s systems often involve multiple architectures, hardware, OS, DBMS, net works, vendors Today’s systems are complex if only because they are so enormous: no single person can understand 50 million lines of code Uncertainty is a fact of life in many different aspects of today’s systems: Requirements (the user maybe uncertain what he needs) The user himself (may be replaced by someone else) Business value of the delivered system Technologies used to build the system Vendors, suppliers, business partners Disruption — technological, environmental, deliberatePublished under the GNU Free Documentation License (GFDL) 117

SRM is part of a larger environment Competitor Software application Organizational environment Business Software project environment Software Process Product application risks risks Software Competitor applicationThe major causes of project failures often exist in the organizational environment,or in the business environment – both of which are outside the scope of thesoft ware manager’s jurisdiction.Conversely, sometimes project manager is unaware of the consequences of hissoft ware risks that are outside the soft ware project per se.So SRM should be viewed as a dynamic, competitive process rather than a staticdefensive once-only procedure. 118

Key components of risk-management plan Risk identification at beginning of project, and then reviewed/updated frequently Documented policy on who is allowed to identify, report, assess, track risks Risk assessment Assessment of severity of the risk, if it occurs — financial, legal, schedule, reputation, and any other relevant dimensions Assessment of the likelihood that the risk will occur (consider using “prediction market” concept for this) This also needs to be reviewed frequently; consider using a simple red- yellow-green visual code to show how things have changed since last time Risk mitigation Strategies for preventing and/or mitigating the risk Identification and assignment of responsibility for dealing with risk Also needs to be reviewed frequently Risk tracking Implied by “frequent review” comments above Distinguish bet ween risks and issues But needs to be widely available in spreadsheet/database form 119

How is this done in “ agile”?Project envisioning, and/or any other initial/preliminary planningmeetings — to identify major, “global” risks that are likely to existthroughout the projectSprint planning sessions — to discuss the “global” risks that are likelyto become significant during the next sprint, and also any less-globalrisks associated with the sprint itselfSprint review/retrospective meetings — to discuss the risks that werefaced during the just-completed sprint, and which may or may notpersist into the next springDaily standup meetings — to identify “immediate,” local risksNote: this is often an area where the project manager plays a crucialrole because he/she can communicate the risk issues to otherstakeholders outside the team — and because some of the risks may belocated outside the team (e.g., external business risks, environmentalrisks, etc. 120

Ways of managing risk Reactive “Fire-fighting” “Fix-on-failure” Transitional Proactive Total quality management strategies Expanding scope of risk assessment Management of changeMost extreme projects do reactive risk management –but you should be aware of your situation, and makingplans to move to a more pro-active approach 121

Breathalyzer TestDo you have a current, credible activity net work supported by a work breakdownstructure (WBS)?Do you have a current, credible schedule and budget?Do you know what soft ware you are responsible for delivering?Can you list the top ten project risks?Do you know your schedule compression percentage?What is the estimated size of your soft ware deliverable? How was it derived?Do you know the percentage of external interfaces that are not under yourcontrol?Does your staff have sufficient expertise in the project domain?Have you identified adequate staff to allocate to the scheduled tasks at thescheduled time?All of this comes from the Airlie Council; for another discussion about the sameconcept, see “Blowing the Whistle on Troubled Soft ware Projects,” by Mark Keil(not this Mark Keil!) and Daniel Roby, Communications of the ACM, April 2001 122

Cultural IssuesAs Tom DeMarco points out (see Waltzing with Bears: Managing Risk on Soft wareProjects), an organizations “true” approach to risks, and risk management, areoften illustrated by some of the "folklore" that is not written down, but is animportant part of the culture:See Dilbert’s explanation of risk management politics...“Failure is not an option!” (thanks to Ken Orr for remembering this one!)“We’re on a mission from God!” (thanks to Tim Lister — see his talk)Outright denial that certain risks exist“He who identifies a risk is responsible for solving it.”“We dont have risks in our organization, we just have opportunities.”“Dont tell me about a problem unless youve got a solution to the problem.”“We don’t lie, but we tell the truth slowly.” A phrase attributed to President Clintons press secretary, Mike McCurry See this article for an explanation by subsequent presidential Press Secretary Joe Lockhart about why McCurry felt obliged to tell the truth slowly. Hint: McCurry made his comment in the midst of the Monica Lewinsky scandal, when he had to choose his words carefully — perhaps because he felt that President Clinton wasnt telling him the truth. 123

Changes in Risk Management with today’s Extreme ProjectsRisks often occur more quicklyThere is less time to resolve/respond to risksRisks are more often associated with multiple failures, but we still plan for single “points of failure”Risks are often “malevolent,” rather than “neutral”We have tools like Twitter to communicate and collaborate about risks as they occurWe realize that it’s important to empower “front-line” people to respond to risks in real time. 124

Likely Causes of Soft ware Risks1. Poorly-defined or changing requirements2. Poor project planning/management3. Uncontrolled quality problems4. Unrealistic expectations/inaccurate estimates5. Naive adoption of new technologyAccording to Steve McConnell, author ofSoft ware Project Sur vival Guide 125

7 LANGUAGES, TOOLS, AND . TECHNOLOGYAn extreme project must be allowed to chooseits own tools, regardless of whether itconforms to organizational standards... but team members need to agree on commontools within the project – other wise, chaoswill occurUnless team has worked together before onseveral projects, this implies a “minimal” set oftools that everyone will useFor more on this topic, see my December 2000Computer world article, “The Value of Tools” 126

71 Tool checklist .Collaboration toolsPrototyping/RAD development toolsConfiguration management/version controlCASE tools for analysis/designTesting, debugging toolsProject management (estimating, scheduling,PERT/GANTT, etc)Libraries of reusable components 127

7 Collaboration tools .2Note: email is part of the standard “infrastructure”in virtually every organization today. Insist on“threaded” email support.More important in today’s environment: use wikis,blogs, Twitter, (see “Twittering in Plain English”) andother tools to support fast, informal collaborationbet ween team members.Make sure that team members have access to email/IM at home, while traveling, etc. 128

Introduction to Twitter ü Twi t te r simply ask s, “ What are you doing now?”and allows 140-char re sponse s ü Starte d in 2007 by Evan Williams, who no t ice d 200 pe ople eme rging f rom a le ct ure at Stanford, all picking up the ir ce ll phone s to ask the ir f r ie nds, “ What are you doing no w?” ü Basic ide a: re al-t ime interact ions wi th a se lf-se le cte d ne t work of “like-minde d” indi v iduals wi th whom you share ide as, que s t ions, s tat us update s, obser vat ions abou t things in your world, e tc. ü Also allo ws you to subscr ibe to any me ssage containing spe cif ie d key words, or “hash- tags” ü A Twi t te r use r s ays: “IM is re al-t ime, person-to-pe rson communicat ion, while Twi t te r is ‘baby email’ wi th e ve r yone whe re you ge t to pick whose me ssage s to re ad.” ü Operate s on a var ie t y of plat forms: Web, s tand-alone clie nt programs (Twhirl, e tc.), Blackbe r r y, iPhone, e tc. ü Vocabular y: me ss age s = “t wee ts”. Communicat ing = “t wi t ter ing”. Re dis t r ibu t ing an intere s t ing me ss age to your o wn ne t work = “re t wee t ing”, or “RT” ü See “Twi t te r in Plain English” for YouTube e xplanat ion of how i t work s. ü See “From He re to Twee terni t y: A Pract ical G uide to Ge t t ing Starte d on Twi t te r ” for good t ipsPublished under the GNU Free Documentation License (GFDL) 129

An example of Twitter client-based program: TweetdeckPublished under the GNU Free Documentation License (GFDL) 130

One of the mobile platforms for Twitter: TweetiePublished under the GNU Free Documentation License (GFDL) 131

Use of Twitter-like tools for SW development ü Use wikis for collaboration, problem-solving ü Use Twitter & microblogging to tell others – both inside and outside the organization – how exciting the project is, and how great the new system will be. ü Consider letting “outsiders” contribute code, perform testing, help trouble- shoot, provide tech support, invent new product features Former employees Retired employees Business partners Random citizensPublished under the GNU Free Documentation License (GFDL) 132

Use of Twitter-like tools for project management ü Use Web 2.0 project management tools that facilitate quick, fast “grass-roots” data-entry for status, progress ü Examples: eProject, Basecamp — look for API’s with Twitter & other microblogging tools... ü Caveat: these are usually light-weight tools, may not be adequate for big “mission-critical” projects ü Basic “cultural” issue: quick, informal communication, sharing and democratization of project-management data ü Also, use wikis & micro-blogging to collaborate and share things like risk-management, issue-managementPublished under the GNU Free Documentation License (GFDL) 133

Use of Twitter-like tools for process improvement ü Use wikis blogs, Twitter/micro-blogs to collaborate and share: “lessons learned” Tips and techniques Best practices Worst practices ü Cultural issue: use micro-blogs and wikis to encourage quick, fast, bottom-up “grass roots” process improvement, rather than (or in addition to) top-down initiatives ü Use external blogs and wikis to get more rapid, more “unfiltered” feedback from users about the good, the bad, and the ugly aspects of the system/productPublished under the GNU Free Documentation License (GFDL) 134

7 Reusable components .3This is very definitely language/platform dependentAlso involves significant architectural decisions – e.g., CORBA vs. COM/DCOM. (See “Components for the extreme Age,” by Paul Harmon,Component Development Strategies, March 2000)Often vendor-dependent, too – e.g., various C++ and Java libraries fromother vendors (Rogue Wave, etc.)An interesting area: industry-specific reusable “frameworks” (see IBM’s“San Francisco” project)Key point: you won’t get much of a productivity benefit from theselibraries unless your project team is already familiar with them!Be aware of the cost of reuse: if more than 20% of “reusable” code has tobe modified, you might not achieve any productivity benefit at all. See“Soft ware Estimating Technology: A Survey,” by Richard D. Stutzke,Crosstalk: the journal of defense engineering, May 1996 for details. 135

74 Risks of choosing new tools .Some extreme projects grab new tools as a “silver bullet” toaccomplish much higher levels of productivity than wouldother wise be possible ...... but they ignore the learning cur ve, confusion, and politicaldebates associated with the introduction of new toolAnd the tools are often so new that they don’t even workproperly yetAn irony: new tool sometimes is the “straw that breaks thecamel’s back” (see equivalent idiom in 10 other languages) –and project failure is then blamed on the tool 136

8. SYSTEM DYNAMICS MODELINGThe problems with extreme projects are seldomtechnical – they are managerial, social, cultural,people-orientedSome organizations are emphasizing basic“peopleware” issues: recruiting, motivating,training, self-managed teams, etc.But the organizational system of processes(including the “soft” components of the process) is socomplex we don’t understand itHence the current interest in “dynamic” modeling andsimulation of soft ware development processes 137

Static versus dynamic models Managers plan simple “cause-and-effect” relationships Example: if a soft ware project is late, and more work needs doing – then hire more people. Brooks Law says: adding more people to a late soft ware project just makes it later. The problem: “feedback” relationships bet ween different variables in many “dynamic” systems Tom DeMarco: “everything is deeply intert wingled” 138

Examples of “dynamics”What is effect of a “safety factor” in project estimate?When do productivity tools lower the productivity rate?Does productivity decline affect morale?Does morale decline affect turnover?If turnover goes up, who leaves first?If high-producers leave, will productivity decline more? 139

A soft ware project model Developed by Tarek Abdel-Hamid and Stuart Madnick in their book, Soft ware Project Dynamics Models activities in one medium-sized development project These are all illustrated using the iThink modeling package on Macintosh and Windows platforms, available from ISEE Systems Concentrates on four major activities: Human resource management Soft ware production Project control Project planning 140

Human resource Management BEGINNERS VETERANSHiring Rate Assimilation Rate Quit Rate 141

Human resource Management, continued Trainers per Hiree Avg Empl Time Daily MP for Training Assim Delay BEGINNERS VETERANSHiring Delay Hiring Rate Assimilation Rate Quit Rate 142

Human resource Management, continued Trainers per Hiree Daily MP for Training Assim Delay Avg Empl Time BEGINNERS VETERANS Hiring Delay Hiring Rate Assimilation Rate Quit Rate Rookie Transfer Rate Pro Transfer RateRookie Transfer Delay Pro Transfer Delay Rookie Transfer Rate 143

Human resource Management, complete subsystem Trainers per Hiree Most Hirees per FTE Avg Daily MP per Staff Ceiling on Hires Daily MP for Training Assim Delay FTE Veterans BEGINNERS VETERANS Avg Empl Time Hiring Delay Hiring Rate Assimilation Rate Quit Rate Work Force Ceiling Rookie Transfer Rate Pro Transfer Rate Rookie Transfer Delay Pro Transfer Delay Total Work Force Rookie Transfer Rate Avg Daily MP per Staff Work Force Gap Work Force Level Sought Total Work Force Full Time Equiv Work Force Workforce Level Needed 144

WAR GAMES “War games help you to evaluate your relative strengths and weaknesses and help the organization to obser ve its global strengths and weaknesses.” “For the purpose of stimulative creative disorder, the most effective form of war game calls for participants to take part in teams."Tom DeMarco and Tim Lister, Peopleware 145

A Soft ware War Game Example Imagine a 200-person IT organization with 150 veterans and 50 beginners Backlog of 150 systems + 50 new systems per year demanded by users. Management has decided to make a “strategic investment” in CASE technology to reduce the backlog, improve productivity, etc. Question: how much to spend on various types of CASE tools? How much to invest in training? 146

War Games Results... CASE War Games Cumulative Results at end of Round 9, with team-specified model changes Raises Office Partici PM tool P M PM Mthdlg Mthdlg Mthdlg Repoz Repoz Metrics Metrics Metrics SQA SQA SQA Reuse Reuse Reuse Size ofRe-Eng Re-Eng (%) Cost pation K $$ Train $ Change Tool $ Train $ Cust $$ Tool $$ Train $$Tools $$Train $$Group Tool $$ Train $$ Group Tool $$ Train $$Group LibraryTool $$Train $$T01 5.0% $810 1.00 400 1500 0.75 3000 1500 200 800 300 700.0 200 3.0 1600 500 5.0 500 300 3 400 600 400T02 2.0% $1,050 0.85 20 100 0.80 1200 800 500 350 100 50.0 50 2.0 50 500 2.5 50 100 2.5 500 100 100T03 4.0% $700 0.75 200 300 0.95 1000 1500 200 0 0 100.0 30 3.0 10 140 3.0 50 250 2 500 100 150T04 1.0% $1,055 1.00 40 300 0.90 1000 1000 100 0 0 70.0 200 3.0 75 350 4.0 50 100 3 500 0 0T05 2.0% $950 0.95 200 200 0.30 1000 800 50 400 200 10.0 5 2.0 100 600 3.0 200 300 10 1000 80 40T06 3.0% $900 0.90 200 150 0.80 1500 1000 0 300 350 100.0 55 3.0 50 500 3.0 150 250 10 750 300 50T07 5.0% $1,001 0.80 15 100 0.70 550 600 10 150 75 1.0 0.5 2.5 3 1.5 3.0 10 15 5 800 100 50T08 1.0% $1,000 0.95 50 750 0.50 4000 3000 1500 0 100 75.0 25 4.0 100 250 5.0 10 10 2 300 0 10T09 3.0% $900 0.65 40 60 0.95 3000 1760 500 60 100 30.0 30 3.0 40 30 3.0 30 20 1 200 300 100T10 2.0% $1,000 1.00 200 2000 1.00 1500 2000 500 800 500 300.0 300 3.0 500 1000 5.0 200 1500 3 500 500 500T12 2.0% $950 0.70 300 500 0.80 1000 2000 1000 100 50 75.0 125 3.0 250 500 3.0 150 200 4 750 25 25T13 2.0% $900 0.90 75 100 0.70 1500 1500 100 50 100 75.0 100 3.0 75 100 3.0 100 200 5 500 50 100T14 2.0% $1,000 1.00 100 200 0.80 1500 2000 100 250 500 150.0 100 3.0 5 200 2.0 10 100 2 400 200 20T15 3.0% $1,000 1.00 100 300 0.70 700 2300 200 400 1600 600.0 1800 5.0 10 100 1.0 0 100 2 500 10 20T16 2.0% $1,200 1.00 200 500 1.00 2000 3000 250 200 300 100.0 200 3.0 200 400 3.0 200 400 4 100 100 200T17 2.5% $1,100 0.90 100 300 0.73 1000 2000 100 100 100 75.0 100 2.0 250 500 3.0 200 400 4 500 100 200T18 2.0% $950 1.00 40 25 0.85 1600 960 40 50 30 10.0 6 2.0 10 6 3.0 30 20 4 1000 30 20T19 1.0% $1,000 0.75 100 50 0.30 1000 100 0 100 10 100.0 10 2.0 500 1000 3.0 500 100 5 500 0 0T20 2.0% $950 0.55 60 1000 0.80 3000 1500 50 750 200 0.5 100 2.0 300 1000 10.0 100 500 30 750 1000 1000Avg 2.4% $969 0.88 128 444 0.75 1634 1543 284 256 243 138.0 181 2.8 217 404 3.6 134 256 5 550 189 157 Q turn- percentage Rookies Veterans CASE Develop. Backlog in What Quarter does over turnover remaining remaining Capital $$ Budget Q20 backlog disappear? (people) (ANNUAL) in Q 20 in Q 20 in Q 20 in Q 20T01 2 5.50% 4 148 9050 7376 18 12 productivity improvmts: 15% for PM, 17% for Meth, 1% for Repoz, 10% Metrics, 15% SQA, 2% Reuse & Re-EngT02 2 7.30% 3 149 16430 13194 18 12 allowed hiring of veterans = 20% of rookies hiredT03 2 8.10% 21 147 16170 10043 41 30 productivity improvmts: 15% for PM, 17% for Meth, 1% for Repoz, 10% Metrics, 15% SQA, 2% Reuse & Re-EngT04 2 5.90% 2 148 16815 15399 18 11 no changes requestedT05 2 6.30% 8 148 15865 13440 18 15 no changes requestedT06 2 6.70% 12 148 15045 11562 18 16 reusable components can be created in one month, instead of taking 3 monthsT07 2 7.50% 0 146 18329 4369 18 11 rookies turn into veterans in one quarter; as a result, rookies vanish in Q 12T08 2 5.90% 4 148 11620 15355 18 12 no changes requestedT09 3 9.20% 27 137 14400 12618 18 19 no changes requestedT10 2 5.80% 0 148 8200 13766 18 10 reusable components created in 1 mo, instead of 3 mo -- NO EFFECT NOTED, possibly inadequate tool $$T12 3 8.80% 21 137 14700 14332 18 17 meth customizing $$ double productivity impact of methodology - NO EFFECT NOTEDT13 2 6.80% 10 148 15875 13711 18 16 no changes requestedT14 2 5.70% 2 148 14665 13375 18 11 reusable components created in 1 month instead of 3 months -- NO EFFECT NOTEDT15 2 5.70% 0 148 9160 11632 18 10 max productivity improvement with no training reduced to 30% - NO EFFECT NOTEDT16 1 5.30% 0 147 12000 10404 18 8 Rookies eliminated, company hires only veteransT17 2 6.65% 2 148 14575 10670 18 11 T17 asked for organizational "rate of learning" factor -- too hard to implement at this timeT18 2 5.70% 0 152 17163 12328 18 12 rookies turn into veterans in 6 quarters instead of 10T19 2 7.90% 12 148 16430 15224 18 16 Training now has twice the impact on productivity increase of each type of toolT20 2 8.30% 24 144 9490 13618 88 when hell freezes over NO CHANGES REQUESTED TO MODELAVG 2 6.79% 8 147 13999 12232 23 14 147

Words to live by in the soft ware field“I wake up each morning determined to changethe World ...... and also to have one hell of a good time.Sometimes that makes planning the day a littledifficult.” E.B. White found in the opening of the preface of Succeeding with Objects, by Adele Goldberg and Kenneth S. Rubin 148

Exercise 0Part 1: Without looking in your pocket or purse, write downyour estimate of the total value of the coins that all of theseminar participants have (not including me, not including theinterpreters)Part 2: now count the value of the coins that you have in yourpocket or purse, and write down that number.Part 3: now make a second estimate of the total value of thecoins that all of the seminar participants have (not includingme, not including the interpreters) 149

Exercise 1: Reasons for Extreme projects: Oct 2011Reason Number1. Competitive pressure 212. Politics 2113. Arrogance of management: deadline by fiat 14. Naive optimism of project leaders, IT professionals 385. External demand — example: new government 112regulations6. Poor estimating and planning 23327 Poor project tracking/monitoring and control . 728. Unexpected risks 3459. Other: 151

Reasons for Extreme projects: what to do?Reason Number1. Competitive pressure ?2. Politics Top priority!3. Arrogance of management: deadline by fiat this is politics4. Naive optimism of project leaders, IT professionals naivete is inevitable5. External demand — example: new government Essentiallyregulations politics6. Poor estimating and planning but estimates are ignored7 Poor project tracking/monitoring and control . This is IT’s job!!8. Unexpected risks Danger9. Other: ? 152

Exercise 2: Why Participate in Extreme projects? Oct 2011Reason Number1. Risks are high, but so are rewards 133222. The “Mt. Everest” syndrome 6133. Excitement of working intensely with committed people 3389124. Naive optimism of youth 415. The alternative is unemployment 2736. Required for promotion, career growth 273117 Alternative is bankruptcy or calamity . 146 (10)8. Opportunity to escape “normal” bureaucracy 2214 (10)9. Revenge 5510. Participate in urgent “social crisis” - e.g., earthquakes 29811. Other: 153

Exercise 3: What has changed since previous extreme projects in 1997? Oct 2011Reason Number1. Order-of-magnitude improvements in technology 1311422. Better tools for communication, coordination, 2162212collaboration3. Outsourcing & global/distributed projects 34223114. Global recession 3343335. Fraud, scams, lack of trust 2566. Other: less time to market 155 154

Exercise 3: What has changed since previous extreme projects in 1997? so what?Reason Number1. Order-of-magnitude improvements in technology big -but enough?2. Better tools for communication, coordination, big - but enough?collaboration non-negotiable for most of us3. Outsourcing & global/distributed projects makes agile harder eliminates biggest leverage creates more pressure4. Global recession eliminates alternative jobs more people avoiding IT careers5. Fraud, scams, lack of trust not so important6. Other: less time for shorter projects more pressure for agile 155

Exercise 4: Biggest political problems in extreme projects Oct 2011Reason Number1. Deadlines are presented as a “fait accompli” 123222. Nobody believes estimates, so no commitment for success 33. Estimates are treated as “promises,” with no margin for error 334. Refusal to admit that project is experiencing problems (risk mgmt) 3215. No real justification for project; nobody really wants it 56. Lack of significant commitment from executives/sponsors 11112317 Nobody learns lessons from previous extreme projects . 32418. Other9. I don’t know10. None of the above 156

Exercise 4:Biggest political problems in extreme projects, so what?Reason Number1. Deadlines are presented as a “fait accompli” “I told you so”2. Nobody believes estimates, so no commitment for success Anonymous polls better visual presentation of3. Estimates are treated as “promises,” with no margin for error estimates?4. Refusal to admit that project is experiencing problems (risk mgmt) Anonymous tools5. No real justification for project; nobody really wants it ??6. Lack of significant commitment from executives/sponsors ??7 Nobody learns lessons from previous extreme projects . blog?8. Other9. I don’t know10. None of the above 157

Exercise 5:Biggest estimating problems in extreme projectsReason Number1. No estimates — deadline/budget imposed by fiat 113112. Naivete: estimates created by inexperienced project manager 123. Hysterical optimism: project team thinks they are super-heros 24. Inadequate communication of assumptions re tradeoffs of time/$ 3225. Overly optimistic assumptions (nothing will go wrong) 3216. Estimates assume “infinite” unpaid overtime by project team 237 “Estimating games” by shrewd stakeholders against naive P.M. . 338. Other: difficult to estimate hardware effort; incomplete plan. 31Lack of historical dataWhat is the most important thing that could be done to improve the effectiveness ofestimating IT projects? More time to formulate a good estimate. Need to have anoverview about what politically happens. Enough time to have detailed informationabout the complete list of requirements. Checkpoints to review estimates (1 month) 158

Exercise 5: Biggest estimating problems in extreme projects Nov 2010Reason Number1. No estimates — deadline/budget imposed by fiat 2112. Naivete: estimates created by inexperienced project manager 33. Hysterical optimism: project team thinks they are super-heros 34. Inadequate communication of assumptions re tradeoffs of time/$ 1225. Overly optimistic assumptions (nothing will go wrong) 3136. Estimates assume “infinite” unpaid overtime by project team 27 “Estimating games” by shrewd stakeholders against naive P.M. .8. Other:9. I don’t knowWhat is the most important thing that could be done to improve theeffectiveness of estimating IT projects? 159

Biggest estimating problems in extreme projects, Dec 2009Reason Number1. No estimates — deadline/budget imposed by fiat 113212. Naivete: estimates created by inexperienced project manager 1323. Hysterical optimism: project team thinks they are super-heros 134. Inadequate communication of assumptions re tradeoffs of time/$ 3231125. Overly optimistic assumptions (nothing will go wrong) 2312216. Estimates assume “infinite” unpaid overtime by project team 422437 “Estimating games” by shrewd stakeholders against naive P.M. . 43328. Other9. I don’t know10. None of the above 160

Exercise 6: (Oct 2011)Would you break rules to succeed on an extreme project? For example, would you considerimplementing a “prediction market” and publish it anonymously, even if not approved?Answer Number1. YES! Always, under any circumstance! 32. yes ... but only minor, administrative rules 221113. maybe ... but only in an emergency 1114. NO! Never! 11 (if needed, notify senior mgmt) avoid some steps of the usual/formal chain of command, in order to have some task accomplished faster than usual. Avoid hierarchy5. If yes to #1, #2, or #3 then : WHICH rules flow. Don’t follow company process or standardwould you break? procedure. Remote work. Negotiate extra effort, working spaces. (?) Break rules only for escalation purposes. More flexible working time. Milestone approval. 161

Exercise 7: Biggest peopleware problems in extreme projects, Oct 2011Reason Number1. Project manager does not have “veto” on staff assignments 222. Inadequate teamwork/cooperation bet ween team members 1121113. No mechanism for providing rewards for exemplary effort/results 2334. Project manager has no “morale budget” 45. Overtime/weekend work is mandatory, not optional 53236. Project team has bad office/working conditions 2147 Inadequate attention/empathy for personal issues (e.g., family) . 38. Other : transparency, communication, sharing perspectives 49. I don’t know10. None of the above 162

Exercise 7: Biggest peopleware problems in extreme projects Nov 2010Reason Number1. Project manager does not have “veto” on staff assignments 33232. Inadequate teamwork/cooperation bet ween team members 11223. No mechanism for providing rewards for exemplary effort/results 1114. Project manager has no “morale budget”5. Overtime/weekend work is mandatory, not optional 26. Project team has bad office/working conditions 27 Inadequate attention/empathy for personal issues (e.g., family) . 38. Other (management does not care about staff at all - 3including career opportunities)9. I don’t know10. None of the above 163

Exercise 7: Biggest peopleware problems in extreme projects, May 2010Reason Number1. Project manager does not have “veto” on staff assignments 13242. Inadequate teamwork/cooperation bet ween team members 52313. No mechanism for providing rewards for exemplary effort/results 21124. Project manager has no “morale budget” 365. Overtime/weekend work is mandatory, not optional 456. Project team has bad office/working conditions 5437 Inadequate attention/empathy for personal issues (e.g., family) . 4748. Other 89. I don’t know10. None of the above 164

Exercise 7: Biggest peopleware problems in extreme projects, Dec 2009Reason Number1. Project manager does not have “veto” on staff assignments 33552. Inadequate teamwork/cooperation bet ween team members 11211133. No mechanism for providing rewards for exemplary effort/results 1131224. Project manager has no “morale budget” 2123415. Overtime/weekend work is mandatory, not optional 2226. Project team has bad office/working conditions 333237 Inadequate attention/empathy for personal issues (e.g., family) . 14248. Other9. I don’t know10. None of the above 165

Exercise 1: Reasons for Extreme projects: Nov 2010Reason Number1. Competitive pressure 13112. Politics 323. Arrogance of management: deadline by fiat 2224. Naive optimism of project leaders, IT professionals5. External demand — example: new government 22311regulations6. Poor estimating and planning 3237 Poor project tracking/monitoring and control . 38. Unexpected risks 39. Other: Project scope not well defined; incomplete...? 11 166

Exercise 1: Reasons for Extreme projects May 2010Reason Number1. Competitive pressure 8712. Politics 1333. Arrogance of management: deadline by fiat 231224. Naive optimism of project leaders, IT professionals 555. External demand — example: new government 1221regulations6. Poor estimating and planning 2447 Poor project tracking/monitoring and control . 748. Unexpected risks 36649. Other: requirements change, unconsolidated standards 94 167

Exercise 3: What has changed since previous extreme projects in 1997? Nov 2010Reason Number1. Order-of-magnitude improvements in technology 1332. Better tools for communication, coordination, 22312122collaboration3. Outsourcing & global/distributed projects 31221314. Global recession 115. Fraud, scams, lack of trust6. Other: less time for shorter projects 33 168

Reasons for Extreme projects — Dec 2009Reason Number1. Competitive pressure 7832. Politics 111112113. Arrogance of management: deadline by fiat 2223224. Naive optimism of project leaders, IT professionals 586245. External demand — example: new government 831422113regulations6. Poor estimating and planning 66234367 Poor project tracking/monitoring and control . 44335358. Unexpected risks 757379. Other: 39398 169

Exercise 2: Why Participate? Nov 2010Reason Number1. Risks are high, but so are rewards 2112. The “Mt. Everest” syndrome3. Excitement of working intensely with committed people 321214. Naive optimism of youth 35. The alternative is unemployment 36. Required for promotion, career growth 32227 Alternative is bankruptcy or calamity .8. Opportunity to escape “normal” bureaucracy 1113339. Revenge10. Participate in urgent “social crisis” - e.g., earthquakes 211. Other: 170

Exercise 2: Why Participate? May 2010Reason Number1. Risks are high, but so are rewards 82. The “Mt. Everest” syndrome 5423. Excitement of working intensely with committed people 41334. Naive optimism of youth 75. The alternative is unemployment 946. Required for promotion, career growth 12117 Alternative is bankruptcy or calamity . 5 ten8. Opportunity to escape “normal” bureaucracy 23249. Revenge 610. Participate in urgent “social crisis” - e.g., earthquakes 3611. Other: guilt, embarrassment, “it must be an exception” eleven 171

Exercise 3: What has changed since previous extreme projects in 1997? May 2010Reason Number1. Order-of-magnitude improvements in technology 33222. Better tools for communication, coordination, 11111collaboration3. Outsourcing & global/distributed projects 21344. Global recession 42435. Fraud, scams, lack of trust 54666. Other: less time for shorter projects 6255 172

Exercise 3: What has changed since previous extreme projects in 1997? Dec 2009Reason Number1. Order-of-magnitude improvements in technology 21111112152. Better tools for communication, coordination, 312333123collaboration3. Outsourcing & global/distributed projects 12232241314. Global recession 4112322425. Fraud, scams, lack of trust 23453666. Other: less time for shorter projects 3132354 173

Exercise 4:Biggest political problems in extreme projects Nov 2010Reason Number1. Deadlines are presented as a “fait accompli” 11312. Nobody believes estimates, so no commitment for success3. Estimates are treated as “promises,” with no margin for error 2124. Refusal to admit that project is experiencing problems (risk mgmt) 3235. No real justification for project; nobody really wants it6. Lack of significant commitment from executives/sponsors 122217 Nobody learns lessons from previous extreme projects . 3338. Other9. I don’t know10. None of the above 174

Exercise 4: May 2010 Biggest political problems in extreme projectsReason Number1. Deadlines are presented as a “fait accompli” 1312. Nobody believes estimates, so no commitment for success 23. Estimates are treated as “promises,” with no margin for error 41324. Refusal to admit that project is experiencing problems (risk mgmt) 34445. No real justification for project; nobody really wants it 216. Lack of significant commitment from executives/sponsors 12137 Nobody learns lessons from previous extreme projects . 252458. Other 89. I don’t know10. None of the above 175

Exercise 4:Biggest political problems in extreme projects, 2009Reason Number1. Deadlines are presented as a “fait accompli” 623142211112. Nobody believes estimates, so no commitment for success 73213173. Estimates are treated as “promises,” with no margin for error 321252224. Refusal to admit that project is experiencing problems (risk mgmt) 441313365. No real justification for project; nobody really wants it 27271336. Lack of significant commitment from executives/sponsors 11212223247 Nobody learns lessons from previous extreme projects . 5236158. Other 939. I don’t know 8310. None of the above 3 176

Exercise 4:Biggest political problems in extreme projects, so what?Reason Number1. Deadlines are presented as a “fait accompli” “I told you so”2. Nobody believes estimates, so no commitment for success Anonymous polls better visual presentation of3. Estimates are treated as “promises,” with no margin for error estimates?4. Refusal to admit that project is experiencing problems (risk mgmt) Anonymous tools5. No real justification for project; nobody really wants it ??6. Lack of significant commitment from executives/sponsors ??7 Nobody learns lessons from previous extreme projects . blog?8. Other9. I don’t know10. None of the above 177

Examples of breaking rules (emergency) - Overcome formal aspects using informal “gentleman’s agreements” (emergency) Ask people to work extra time without paying additional money to them. (occasionally) We need to create a tradeoff bet ween administrative and internal process rules, and the project requirements (e.g., to be on time) — and inform our managers about these decisions. (emergency) break some rules in agreement with hierarchical leaders, but without going against the law. 178

My suggestionsFind a way to make the “prediction market” assessment ofproject deadlines visible to a wide audienceFind specific issues where executive support/assistance ismissing, and make them widely visible.Flexible working hoursGive people unofficial time off (maybe several days, or more)if they have worked many long hours of overtime to achieve adeadlineLet them work at home occasionally (assuming this does notbreach significant security/privacy rules)Let people bring their children and pets to workEncourage vendors to provide free meals (lunch, dinner) totheir own employees if those people work overtime 179

Exercise 8: Biggest “process” problems in extreme projectsReason Number1. Insistence on document-driven waterfall process 33222. Management insists on new process, without any training 31333. “Agile” process used as an excuse for “hacking”, no documentation 224. “Official” process ignored by team, no QA to ensure it’s being used 35. Process requires user involvement, but users won’t cooperate 1111116. Process requires automated support, but no tools provided. 337 Process used as a panacea to “cover up” unrealistic estimate, etc. . 28. Other: unable to share the risk: the team “disappears” at the first problem9. I don’t know10. None of the above 180

Exercise 8: Biggest “process” problems in extreme projects Nov 2010Reason Number1. Insistence on document-driven waterfall process 33312. Management insists on new process, without any training 3123. “Agile” process used as an excuse for “hacking”, no documentation 124. “Official” process ignored by team, no QA to ensure it’s being used 2125. Process requires user involvement, but users won’t cooperate 2136. Process requires automated support, but no tools provided.7 Process used as a panacea to “cover up” unrealistic estimate, etc. .8. Other:9. I don’t know10. None of the above 181

Exercise 8: Biggest “process” problems in extreme projects May 2010Reason Number1. Insistence on document-driven waterfall process 452. Management insists on new process, without any training 1233. “Agile” process used as an excuse for “hacking”, no documentation 54244. “Official” process ignored by team, no QA to ensure it’s being used 31455. Process requires user involvement, but users won’t cooperate 23116. Process requires automated support, but no tools provided. 6327 Process used as a panacea to “cover up” unrealistic estimate, etc. . 6768. Other: process requires automated support, but there are too many, toocomplicated, and requires the same data9. I don’t know10. None of the above 182

Biggest “process” problems in extreme projects, Dec 2009Reason Number1. Insistence on document-driven waterfall process 32232. Management insists on new process, without any training 51233. “Agile” process used as an excuse for “hacking”, no documentation 412434. “Official” process ignored by team, no QA to ensure it’s being used 14315. Process requires user involvement, but users won’t cooperate 11112226. Process requires automated support, but no tools provided. 3777 Process used as a panacea to “cover up” unrealistic estimate, etc. . 3234668. Other: process requires automated support, but there are too many, toocomplicated, and requires the same data 19. I don’t know10. None of the above 183

Exercise 9: Biggest risk management problems in extreme projects — Oct 2011Reason Number1. Complete absence of a risk management plan 121112. Risks considered only at proj. beginning, not throughout the proj. 22213. Risks not shared bet ween various project stakeholders 31132224. Inadequate/no mechanism for reporting new risks 335. “Disloyal/weak” reaction from senior mgmt to warnings of risk 356. Management: those who report risks must fix them 357 Risks reported, but not acted on. . 334338. Other9. I don’t know10. None of the above 184

Exercise 9: Biggest risk management problems in extreme projects Nov 2010Reason Number1. Complete absence of a risk management plan 1212. Risks considered only at proj. beginning, not throughout the proj. 2123. Risks not shared bet ween various project stakeholders 31214. Inadequate/no mechanism for reporting new risks 35. “Disloyal/weak” reaction from senior mgmt to warnings of risk 326. Management: those who report risks must fix them 337 Risks reported, but not acted on. . 3338. Other9. I don’t know10. None of the above 185

Exercise 9: Biggest risk management problems in extreme projects — May 2010Reason Number1. Complete absence of a risk management plan 21112. Risks considered only at proj. beginning, not throughout the proj. 23463. Risks not shared bet ween various project stakeholders 11274. Inadequate/no mechanism for reporting new risks 3435. “Disloyal/weak” reaction from senior mgmt to warnings of risk 426. Management: those who report risks must fix them 347 Risks reported, but not acted on. . 5558. Other 89. I don’t know10. None of the above 186

Exercise 9:Biggest risk management problems in extreme projects, Dec 2009Reason Number1. Complete absence of a risk management plan 111112. Risks considered only at proj. beginning, not throughout the proj. 2233213. Risks not shared bet ween various project stakeholders 3422334. Inadequate/no mechanism for reporting new risks 7325. “Disloyal/weak” reaction from senior mgmt to warnings of risk 63226. Management: those who report risks must fix them 321137 Risks reported, but not acted on. . 5121118. Other9. I don’t know10. None of the above 187

Exercise 10: Changes in risk management caused by extreme projectsReason Number1. Risks often occur more quickly2. There is less time to resolve/respond to risks3. Risks are more often associated with multiple failures, but we still planfor single “points of failure”4. Risks are often “malevolent,” rather than “neutral”5. We have tools like Twitter to communicate and collaborate about risks asthey occur6. We realize that it’s important to empower “front-line” people to respondto risks in real time.7 Other .8. No changes; it’s still the same as before9. I don’t know10. None of the above 188

Exercise 10: Changes in risk management caused by extreme projects Nov 2010Reason Number1. Risks often occur more quickly 222. There is less time to resolve/respond to risks 111113. Risks are more often associated with multiple failures, but we still planfor single “points of failure” 3234. Risks are often “malevolent,” rather than “neutral”5. We have tools like Twitter to communicate and collaborate about risks asthey occur 36. We realize that it’s important to empower “front-line” people to respondto risks in real time. 32327 Other .8. No changes; it’s still the same as before9. I don’t know10. None of the above 189

Exer 11: Most important strategies/techniquesfor succeeding in an extreme project Oct 2011Reason Number1. Better understanding, ability to deal with politics 31312. Better negotiating/estimating techniques for schedules, budget 12233. Peopleware strategies 21364. More effective processes: SEI-CMM, agile, XP, prototyping, etc 135. Emphasis on “good enough” systems, rather than “zero-defect” 24216. Better ability to monitor/control progress in the project 3527 Better development tools & techniques .8. Other9. I don’t know10. None of the above - success with “extreme” projects cannotbe planned or controlled 190

Exer 11: Most important strategies/techniquesfor succeeding in an extreme project Nov 2010Reason Number1. Better understanding, ability to deal with politics 461112. Better negotiating/estimating techniques for schedules, budget 532223. Peopleware strategies 153334. More effective processes: SEI-CMM, agile, XP, prototyping, etc 726565. Emphasis on “good enough” systems, rather than “zero-defect” 344756. Better ability to monitor/control progress in the project 215447 Better development tools & techniques . 677678. Other 79. I don’t know10. None of the above - success with “extreme” projects cannotbe planned or controlled 191

Exer 11:Most important strategies/techniques for succeeding in an extreme project, 2009Reason Number1. Better understanding, ability to deal with politics 211122. Better negotiating/estimating techniques for schedules, budget 122213213. Peopleware strategies 46231114. More effective processes: SEI-CMM, agile, XP, prototyping, etc 644235. Emphasis on “good enough” systems, rather than “zero-defect” 35336. Better ability to monitor/control progress in the project 53122117 Better development tools & techniques . 77334428. Other9. I don’t know10. None of the above - success with “extreme” projects cannotbe planned or controlled 192

Extreme Project Management Ed Yourdon email: ed@yourdon.com Website: www.yourdon.com Blog: www.yourdonreport.comTwitter, LinkedIn, Facebook, Plaxo, Flickr: “yourdon” October 2011 green stuff = “internal” links — i.e., URLs to other pages in this presentation blue stuff = “external” links — i.e., URLs to pages on the Internet red stuff = new material, not in your printed copy of materials

Extreme Project Management

by Ed Yourdon on Oct 15, 2011

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 2

Statistics

Extreme Project Management Presentation Transcript

http://a0.twimg.com	1
http://www.linkedin.com	1