Learning from Projects

Conducting a Post–Mortem Analysis
Presenter -Introduction
A practitioner of Analytics, Information Modelling,
Enterprise ProjectProgram Portfolio Management,
Operational Productivity & Execution Management.

He has also setup PMO groups, PM Forums and led
enterprise    project       management         products
implementation initiatives at global scale for MNCs.
He also writes articles/ blogs on project                            Jaiveer Singh, PMP
management, business intelligence,
social analytics and conducts workshops
on productivity & planning related                                    @ singhjaiveer
software products like MS Excel, MS
Project, Axure RP, Mind Map etc for the
management community.                                     http://www.slideshare.net/Jaiveer



                                                          www.singhjaiveer.blogspot.com

                                     www.project-management-practice.blogspot.com
Session Context
The creation of temporary enterprises for project-based work has
become an increasingly salient feature of the new economy.

•These organisations specialise in execution of certain type of
projects while leveraging their unique competencies, industry best
practices and learnings from their past projects.

•Often learnings from firms' past projects execution acts as most
important key differentiating factor while positioning their
organisation as preferred partner/ vendor.


Leveraging Experience as Key Differentiator
Post Project Analysis
What is a Project?

When Project is considered finished ?

What is involved in Analysis?
Apollo 11- Project




                                                             Neil Armstrong
July 20, 1969




                We can learn from those who have gone before us.
Projects comes in all sizes & complexities
Projects comes in all sizes & complexities
             Project 1                      Project 2                      Project 3

Project      Standard Operating             e-government system , July     Commercial Building, Apr,
Title/       Environment (SOEasy), Feb      2006                           2011
Award        2008
                                            It will provide information    Construction of a
date
                                            about services being           commercial facility at
Scope        Standardize desktop, network   provided by the                Orchard Road having 12
             and messaging components       government to the public in    levels above ground, one
             for 60,000 public officers     an easily accessible manner.   level below ground, and a
             across 74 government                                          gross floor area of 20,072
             agencies.                      Information,                   m2.
                                            Communication &
Industry /   Information, Communication                                    Construction, Commercial
                                            Technology
Type         & Technology                                                  Building
Timelines                                   Completed                      Dec 15, 2013
             Mar 31, 2011- Delayed

Budget       S$ 1.3 billion                 USD $ 3.4 m                    8.3 billion Yen

Client       IDA, Singapore Government      Maldivian government           RE Properties Pte. Ltd, Real
                                                                           Estate Company
Vendor       HP – EDS, Singapore            NCS, Singapore                 SHIMIZU CORPORATION
Project Journey – What’s your experience




Were objectives clearly defined?
Was journey planned well?
Did you had your plans approved?
Did you met your milestones on time?

What was your resources burn rate?     Budgets Over
What unexpected events happened?
How many issues impacted project?
What worked What didn't
Retaining knowledge from Experience


Analyze   Identify   Document   Archive
Accessing past learnings- Project Library



                                                                                                                     Search




Category           What Worked and What Didn't Work                                        Learning Description
Scope Mgmt         WBS based techniques helped to ensure all work scope is accounted for   SoW must be detailed enough covering main & supporting
                   planning                                                                work required to execute project deliverables
Resource Mgmt      Required skills & compentency for team was not estimated properly which Key resources comptencies and availability should be
                   affected work quality and delayed deliverables completion               checked with resource manager for entire project duration
Procurement Mgmt Decision to sub-contract one part of develolpment was taken too late      Review of internal competencies and available capacity
                 which caused delays as vendors asked for min. lead time to setup team     suiting project timelines must be done in early stages of
                                                                                           projects to get job executed from outside in time
Issues Mgmt        Many assumptions were not validated with stakeholders which caused      All positive & negative assumptions must be checked with
                   rework later.                                                           stakeholders and data for similar projects using
                                                                                           organisation project archives
Project Metrics – Run Rate/ Burn Rate
Experiential Learning
Experiential learning is the process of
making meaning from direct experience.
Experiential learning is learning through
reflection on doing.
While it is very effective way of learning, it comes with
its associated high level of costs, therefore it is very
important that we learn from others experiences and
document our own experience for benefit of others.

So there is a need of a structure process to gather
learnings from any experiences
Identify and Capture Project Learnings
Post Mortem Analysis
A postmortem is both a process and a document
(set of documents).
                      Discuss &
            Collect                    Consolidate    Recommen
   Setup                Build
             Data                       Outcome         dations
                      Consensus




                                  Analysis Summary/ Recommendations
Preparing for Meeting
Before Meeting
1. Send out copies of mini, functional post-mortem results for
   review.

2. Ask for people to send list of top issues to discuss—should be
cross-functional issues that require the whole group in order to be
resolved.

3. Send out agenda with a list of potential topics.
    (a) Prioritize topics to discuss.
    (b) Discuss each topic; emphasize what to do in future.
    (c) Summarize and prioritize recommendations.

4. Reserve large room, tape flipchart paper to walls.

5. Compose list of discussion topics.
Meeting- Preparation
The post-mortem meeting provides a chance for the team to get
together and work through the important issues to create an action
plan that will improve the development process for the team.

Meeting Length : 4 hours or less, Schedule 2-3 meeting if required with focus group

Room : Choose a room with a round or oval shaped table so people can easily see each
other and no one appears as the boss at the table’s head.

Who Should Attend: Anyone who was involved in the project should be invited. If
inviting everyone makes the group too large, consider having mini-postmortems with
people who worked on specific parts of the project.

Facilitator: Ideally it should be someone who was not involved in the project and has
no reason to be involved in the discussion.

Recorder: Taping large sheets of flipchart paper on walls or using white boards are very
effective for recording information. This way everyone involved in the discussion
automatically focuses on the recorded information (instead of on each other).
Analysis Guidelines
Be Inclusive                   Scope Management
                                  Issues Management
Cover all key disciplines
                                            Risks Management
Project Participants                        Communications Management
Stakeholders
Analysis Guidelines
Be Self Critical

Participants should check their egos at
the door.

The post-mortem will necessarily find
“flaws” with processes and team
members who executed (or failed to
execute) aspects of the project.
Analysis Guidelines
Be Professional
Discussions should cover a broad
range of team issues and
dynamics, from process to product
issues.

However it should not under any
circumstances become personal.
Most projects have enough
elements that need
to improve that any mention of
names or specific instances can
best be skipped



                                    Focus on issues and not on people
Analysis Guidelines
Be Factual
Documentation and data should be included in both
the discussions and in the final report. Future projects
will find it valuable to learn from project metrics and
other project management data.

The post-mortem provides a good process for
gathering that information and including it in the
report.


Be Brief
Suggestions and commentary in the final report should be brief and agreed to
by broad consensus. Although dozens or more issues will surface during the
post-mortem process, the next project will benefit more from a small number
of very specific suggestions.
Managing Meeting
During Meeting

1. Start with reviewing and ranking topics to be discussed.

2. Begin with the top issue and record what went wrong as well as
how to do it differently in future.

3. Stop “wrong” discussion after 5-7min. and start asking what to do
differently.

4. Check that all functional groups have contributed.

5. Save time at the end to prioritize recommendations.
Discussion Notes
Timeline and Resources:

This includes who was involved and amount of time each person was involved in the
project.

What Went Poorly/Should Be Done Differently?

If the list of what went wrong was collected ahead of time, go over it now and ask for any
additions. Once you have the list generated, the facilitator will need to prioritize the list so
that the most important issues are discussed first (in case you run out of time or energy).

What Went Well

The team simply needs to list what participants thought went well and record why it was
successful.

Recommendations:

Looking back at the information recorded for both the “good” and “bad” discussions
summarize what the group would recommend for future Projects.
Microsoft- Case Study
Microsoft is the one of highly successful global software product company.

Project Name: MS Word Version 6 - Development

Key Findings from Post Project Analysis

Teams participating in post mortem analysis meeting

1. Word 6 development
2. Word 6 program management
3. Word 6 testing
Microsoft- Word 6 Case Study
Word 6 development

* “Scheduling on [Word 6] was not done well and was seen later in the project as totally
unrealistic. Milestones were left before they were really complete. This meant carrying
bugs and work from the pervious milestone over to the next. By the time we realized
what shape we were in it was too late to make adjustments. [Word 6] milestones were
longer milestones than on any other project, this should have been an indicator that
there was a bigger problem.”




* “Many of the problems with proposed features do not become obvious until
development has starting working on it. By this time it is usually later in the project and
program management has very little flexibility redesigning a feature.”
Microsoft- Word 6 Case Study
Word 6 program management

* “The project was too large with too many primary goals. We ended up touching too
many areas of the product without consciously realizing it at the outset of the project.
There were times individuals got so caught up in their feature, that it was hard to
remember the priorities for the project as a whole (“what did I spend four days on that
feature for?”). This highlighted the fact that the decision making process was not well
defined. There was no one key person that would make the final decision when the team
could not reach consensus on a problem.”



* “There were two principal reasons we were six months late in shipping: the inability
to cut features and not know what to cut. We should have been more ruthless about
cutting features in order to meet the schedule. People knew we were slipping (it was
obvious after we missed the first Major Milestone), yet no one wanted to cut features.
Development was concerned that it would hurt morale. When in fact, morale was hurt
more by not being honest about the slipped date.”
Microsoft- Word 6 Case Study
Word 6 testing

* “The size and complexity of *Word 6+ had changed dramatically from WinWord 2. 0. Yet
it was never acknowledged that different processes and tools were needed to manage
such a large project. It was planned and managed as a small project. Once again, care
should be taken on the front end instead of full steam ahead and hoping that it will all
work out.”


 * “The schedule became everything. Individuals across all groups knew that the
 schedule was a myth but all were discouraged from communicating any slips in their
 work. Test knew where we were at with bugs, but this was not broadly communicated.
 Development Leads did not share this information with their team members because it
 was seen as de-motivating. As a result of this schedule deception, poor decisions were
 made (i.e., quick fixes instead of well thought out changes, other tradeoffs were
 managed very poorly) based on inaccurate schedule information
Group Activity
 Identify top 10 list of things which often didn’t went
 well in your projects and top 10 things which worked
 for your projects

Planning                   Scope Mgmt   Time Mgmt     Quality Mgmt
Resources
Project Mgmt/ Scheduling                  Human
                                                       Communicati
Design & Specifications    Cost Mgmt     Resources
                                                         on Mgmt
Communication                              Mgmt
Team/ Organization
                                         Procurement    Integration
Product                    Risks Mgmt
                                            Mgmt           Mgmt
Management
Tools & Practices
General
Seat No   Group   Topic
2-9       1       Scope Mgmt
10-17     2       Time
18-25     3       Quality
26-33     4       Cost
34-41     5       Resources
42-49     6       Communication
50-58     7       Risks
59-66     8       Procurement
Presenting Group Findings
Each group representative present their list of top issues & tips
www.project-management-practice.blogspot.com

Project post-mortem analysis

  • 1.
    Learning from Projects Conductinga Post–Mortem Analysis
  • 2.
    Presenter -Introduction A practitionerof Analytics, Information Modelling, Enterprise ProjectProgram Portfolio Management, Operational Productivity & Execution Management. He has also setup PMO groups, PM Forums and led enterprise project management products implementation initiatives at global scale for MNCs. He also writes articles/ blogs on project Jaiveer Singh, PMP management, business intelligence, social analytics and conducts workshops on productivity & planning related @ singhjaiveer software products like MS Excel, MS Project, Axure RP, Mind Map etc for the management community. http://www.slideshare.net/Jaiveer www.singhjaiveer.blogspot.com www.project-management-practice.blogspot.com
  • 3.
    Session Context The creationof temporary enterprises for project-based work has become an increasingly salient feature of the new economy. •These organisations specialise in execution of certain type of projects while leveraging their unique competencies, industry best practices and learnings from their past projects. •Often learnings from firms' past projects execution acts as most important key differentiating factor while positioning their organisation as preferred partner/ vendor. Leveraging Experience as Key Differentiator
  • 4.
    Post Project Analysis Whatis a Project? When Project is considered finished ? What is involved in Analysis?
  • 5.
    Apollo 11- Project Neil Armstrong July 20, 1969 We can learn from those who have gone before us.
  • 6.
    Projects comes inall sizes & complexities
  • 7.
    Projects comes inall sizes & complexities Project 1 Project 2 Project 3 Project Standard Operating e-government system , July Commercial Building, Apr, Title/ Environment (SOEasy), Feb 2006 2011 Award 2008 It will provide information Construction of a date about services being commercial facility at Scope Standardize desktop, network provided by the Orchard Road having 12 and messaging components government to the public in levels above ground, one for 60,000 public officers an easily accessible manner. level below ground, and a across 74 government gross floor area of 20,072 agencies. Information, m2. Communication & Industry / Information, Communication Construction, Commercial Technology Type & Technology Building Timelines Completed Dec 15, 2013 Mar 31, 2011- Delayed Budget S$ 1.3 billion USD $ 3.4 m 8.3 billion Yen Client IDA, Singapore Government Maldivian government RE Properties Pte. Ltd, Real Estate Company Vendor HP – EDS, Singapore NCS, Singapore SHIMIZU CORPORATION
  • 8.
    Project Journey –What’s your experience Were objectives clearly defined? Was journey planned well? Did you had your plans approved? Did you met your milestones on time? What was your resources burn rate? Budgets Over What unexpected events happened? How many issues impacted project?
  • 9.
  • 10.
    Retaining knowledge fromExperience Analyze Identify Document Archive
  • 11.
    Accessing past learnings-Project Library Search Category What Worked and What Didn't Work Learning Description Scope Mgmt WBS based techniques helped to ensure all work scope is accounted for SoW must be detailed enough covering main & supporting planning work required to execute project deliverables Resource Mgmt Required skills & compentency for team was not estimated properly which Key resources comptencies and availability should be affected work quality and delayed deliverables completion checked with resource manager for entire project duration Procurement Mgmt Decision to sub-contract one part of develolpment was taken too late Review of internal competencies and available capacity which caused delays as vendors asked for min. lead time to setup team suiting project timelines must be done in early stages of projects to get job executed from outside in time Issues Mgmt Many assumptions were not validated with stakeholders which caused All positive & negative assumptions must be checked with rework later. stakeholders and data for similar projects using organisation project archives
  • 12.
    Project Metrics –Run Rate/ Burn Rate
  • 13.
    Experiential Learning Experiential learningis the process of making meaning from direct experience. Experiential learning is learning through reflection on doing. While it is very effective way of learning, it comes with its associated high level of costs, therefore it is very important that we learn from others experiences and document our own experience for benefit of others. So there is a need of a structure process to gather learnings from any experiences
  • 14.
    Identify and CaptureProject Learnings
  • 15.
    Post Mortem Analysis Apostmortem is both a process and a document (set of documents). Discuss & Collect Consolidate Recommen Setup Build Data Outcome dations Consensus Analysis Summary/ Recommendations
  • 16.
    Preparing for Meeting BeforeMeeting 1. Send out copies of mini, functional post-mortem results for review. 2. Ask for people to send list of top issues to discuss—should be cross-functional issues that require the whole group in order to be resolved. 3. Send out agenda with a list of potential topics. (a) Prioritize topics to discuss. (b) Discuss each topic; emphasize what to do in future. (c) Summarize and prioritize recommendations. 4. Reserve large room, tape flipchart paper to walls. 5. Compose list of discussion topics.
  • 17.
    Meeting- Preparation The post-mortemmeeting provides a chance for the team to get together and work through the important issues to create an action plan that will improve the development process for the team. Meeting Length : 4 hours or less, Schedule 2-3 meeting if required with focus group Room : Choose a room with a round or oval shaped table so people can easily see each other and no one appears as the boss at the table’s head. Who Should Attend: Anyone who was involved in the project should be invited. If inviting everyone makes the group too large, consider having mini-postmortems with people who worked on specific parts of the project. Facilitator: Ideally it should be someone who was not involved in the project and has no reason to be involved in the discussion. Recorder: Taping large sheets of flipchart paper on walls or using white boards are very effective for recording information. This way everyone involved in the discussion automatically focuses on the recorded information (instead of on each other).
  • 18.
    Analysis Guidelines Be Inclusive Scope Management Issues Management Cover all key disciplines Risks Management Project Participants Communications Management Stakeholders
  • 19.
    Analysis Guidelines Be SelfCritical Participants should check their egos at the door. The post-mortem will necessarily find “flaws” with processes and team members who executed (or failed to execute) aspects of the project.
  • 20.
    Analysis Guidelines Be Professional Discussionsshould cover a broad range of team issues and dynamics, from process to product issues. However it should not under any circumstances become personal. Most projects have enough elements that need to improve that any mention of names or specific instances can best be skipped Focus on issues and not on people
  • 21.
    Analysis Guidelines Be Factual Documentationand data should be included in both the discussions and in the final report. Future projects will find it valuable to learn from project metrics and other project management data. The post-mortem provides a good process for gathering that information and including it in the report. Be Brief Suggestions and commentary in the final report should be brief and agreed to by broad consensus. Although dozens or more issues will surface during the post-mortem process, the next project will benefit more from a small number of very specific suggestions.
  • 22.
    Managing Meeting During Meeting 1.Start with reviewing and ranking topics to be discussed. 2. Begin with the top issue and record what went wrong as well as how to do it differently in future. 3. Stop “wrong” discussion after 5-7min. and start asking what to do differently. 4. Check that all functional groups have contributed. 5. Save time at the end to prioritize recommendations.
  • 23.
    Discussion Notes Timeline andResources: This includes who was involved and amount of time each person was involved in the project. What Went Poorly/Should Be Done Differently? If the list of what went wrong was collected ahead of time, go over it now and ask for any additions. Once you have the list generated, the facilitator will need to prioritize the list so that the most important issues are discussed first (in case you run out of time or energy). What Went Well The team simply needs to list what participants thought went well and record why it was successful. Recommendations: Looking back at the information recorded for both the “good” and “bad” discussions summarize what the group would recommend for future Projects.
  • 25.
    Microsoft- Case Study Microsoftis the one of highly successful global software product company. Project Name: MS Word Version 6 - Development Key Findings from Post Project Analysis Teams participating in post mortem analysis meeting 1. Word 6 development 2. Word 6 program management 3. Word 6 testing
  • 26.
    Microsoft- Word 6Case Study Word 6 development * “Scheduling on [Word 6] was not done well and was seen later in the project as totally unrealistic. Milestones were left before they were really complete. This meant carrying bugs and work from the pervious milestone over to the next. By the time we realized what shape we were in it was too late to make adjustments. [Word 6] milestones were longer milestones than on any other project, this should have been an indicator that there was a bigger problem.” * “Many of the problems with proposed features do not become obvious until development has starting working on it. By this time it is usually later in the project and program management has very little flexibility redesigning a feature.”
  • 27.
    Microsoft- Word 6Case Study Word 6 program management * “The project was too large with too many primary goals. We ended up touching too many areas of the product without consciously realizing it at the outset of the project. There were times individuals got so caught up in their feature, that it was hard to remember the priorities for the project as a whole (“what did I spend four days on that feature for?”). This highlighted the fact that the decision making process was not well defined. There was no one key person that would make the final decision when the team could not reach consensus on a problem.” * “There were two principal reasons we were six months late in shipping: the inability to cut features and not know what to cut. We should have been more ruthless about cutting features in order to meet the schedule. People knew we were slipping (it was obvious after we missed the first Major Milestone), yet no one wanted to cut features. Development was concerned that it would hurt morale. When in fact, morale was hurt more by not being honest about the slipped date.”
  • 28.
    Microsoft- Word 6Case Study Word 6 testing * “The size and complexity of *Word 6+ had changed dramatically from WinWord 2. 0. Yet it was never acknowledged that different processes and tools were needed to manage such a large project. It was planned and managed as a small project. Once again, care should be taken on the front end instead of full steam ahead and hoping that it will all work out.” * “The schedule became everything. Individuals across all groups knew that the schedule was a myth but all were discouraged from communicating any slips in their work. Test knew where we were at with bugs, but this was not broadly communicated. Development Leads did not share this information with their team members because it was seen as de-motivating. As a result of this schedule deception, poor decisions were made (i.e., quick fixes instead of well thought out changes, other tradeoffs were managed very poorly) based on inaccurate schedule information
  • 29.
    Group Activity Identifytop 10 list of things which often didn’t went well in your projects and top 10 things which worked for your projects Planning Scope Mgmt Time Mgmt Quality Mgmt Resources Project Mgmt/ Scheduling Human Communicati Design & Specifications Cost Mgmt Resources on Mgmt Communication Mgmt Team/ Organization Procurement Integration Product Risks Mgmt Mgmt Mgmt Management Tools & Practices General
  • 30.
    Seat No Group Topic 2-9 1 Scope Mgmt 10-17 2 Time 18-25 3 Quality 26-33 4 Cost 34-41 5 Resources 42-49 6 Communication 50-58 7 Risks 59-66 8 Procurement
  • 32.
    Presenting Group Findings Eachgroup representative present their list of top issues & tips
  • 33.

Editor's Notes

  • #2 Project management became recognized as a distinct discipline arising from the management discipline with engineering model.[Henry Gantt, called the father of planning and control techniques,[9] who is famous for his use of the Gantt chart as a project management tool; The "Critical Path Method" (CPM) was developed as a joint venture between DuPont Corporation and Remington Rand Corporation for managing plant maintenance projects and the "Program Evaluation and Review Technique" or PERT, was developed by Booz Allen Hamilton
  • #5 A project is a temporary endeavor with a defined beginning and end (usually time-constrained, and often constrained by funding or deliverables),[1] undertaken to meet unique goals and objectives,[2] typically to bring about beneficial change or added value.
  • #6 Apollo 11 was the spaceflight which landed the first humans, Neil Armstrong and Edwin "Buzz" Aldrin, Jr, on Earth's Moon on July 20, 1969
  • #8 The S$1.3 billion SOEasy (Public Sector) tender was awarded to oneMeridian, the consortium led by EDS International. The other key members in the consortium include Alcatel-Lucent, Avanade, Cisco Systems, Frontline, Microsoft, Singapore Computer Systems and SingTel.
  • #9 Estimation using equations , Estimation using comparison Estimation using analogy It is important that you do not rely on a single estimation method for a project. Using a combination of both micro and macro estimation techniques has proven to give the most accurate results. In addition, a formal risk assessment is an essential project estimation prerequisite.
  • #10 Project, life cycle, post mortem
  • #14 Ability – Can you still drive bike which you learnt many years back? Can you also still solve problems which you have learnt & solved many years back.As stated by the ancient Chinese philosopher, Confucius, "tell me and I will forget, show me and I may remember, involve me and I will understand."
  • #15 Analysis is the process of breaking a complex topic or substance into smaller parts to gain a better understanding of it.
  • #22 • Brainstorm: Participants say whatever comes to mind as they think of it.• Nominal technique: Participants list individually what they thought wentwrong (may have done this prior to the meeting) and then the facilitator asks each person to say one item from their list until everyone’s lists are recorded.• Storyboarding: Participants write their lists on index cards (one item per card)which are then combined together and sorted by topic to create a comprehensive list.
  • #23 Note: If the recorder/facilitator needs to contribute to the discussion, give someone else the marker to record the information.
  • #24 Documenting the postmortem is very important not just for future reference, but also for thepeople involved. Having a summary of the meeting serves as a project archive and closure, and also allows the team to spread lessons beyond the meeting attendees.
  • #25 Microsoft Office Word is a word processor designed by Microsoft. It was first released in 1983 under the name Multi-Tool Word for Xenix systems.[1][2][3] Subsequent versions were later written for several other platforms including IBM PCs running DOS (1983), the Apple Macintosh (1984), the AT&T Unix PC (1985), Atari ST (1986), SCO UNIX, OS/2, and Microsoft Windows (1989).
  • #26 MS Word 6 was released in MS office 4.0 in Jan 17, 1994, Word 7 was released in Aug 1995 in MS Office 95
  • #32 MS Word product - timelines
  • #33 4-5 mins max.