Slides from the session I ran for APM PMO SIG at MMU business school on 12 Dec. (Most of the session was taken up with the simulation, but these slides cover some of the lessons you might learn as a project reviewer.)

1. Project Reviews, Assurance and the PMO
Project Reviews, Assurance, PMO
Dec 2012 1

2. Project Reviews, Assurance, PMO
Dec 2012 2

3. Agenda
10 min -- Introductions
60 min -- Project review simulation
50 min -- Debrief (& models)
Project Reviews, Assurance, PMO
Dec 2012 3

4. Objectives
 Explore project review process & the pressures it creates
 Explore what support the PMO can give
 Share experience and expertise
Project Reviews, Assurance, PMO
Dec 2012 4

5. Introductions
 Who are you?
 Why are you here?
Project Reviews, Assurance, PMO
Dec 2012 5

6. People lose touch with reality
The single underlying cause for most project failures…
Project Reviews, Assurance, PMO
Dec 2012 8

7. Role of the reviewer…
 … is to help people keep in touch with reality
PMI Research
About 75% of the things that go
wrong on projects, someone knew
about the issue but didn’t know how
to deal with / who to tell about it.
Project Reviews, Assurance, PMO
Dec 2012 9

8. Assessing project Airfix
 24:1 scale model
(1 hour = 3 working days)
 6 teams
 Conduct a health check of
project Airfix
Project Reviews, Assurance, PMO
Dec 2012 10

9. Assessing project Airfix - Rules
1) You have
 Project briefing
 Team / resource list
 Planning framework
2) Read briefing & identify who / what
you want to see over the 3 days
3) I will give document outlines &
interview notes in line with your
plan
4) Analyse notes and present findings
(2 mins)
5) Describe how you got these
findings
Project Reviews, Assurance, PMO
Dec 2012 11

10. Findings
 2 minutes (= 45 mins in real time)
1. Present your key findings
2. Describe how you got to these findings
Project Reviews, Assurance, PMO
Dec 2012 12

11. Debrief
 What was common to your findings? Different? Why?
 What was common to your plans? Different? Why?
 How did you feel during the exercise?
 What was challenging?
 What was confusing?
 What was interesting?
 How was this like real life? How did it differ?
 How can the PMO help make reviews easier?
Project Reviews, Assurance, PMO
Dec 2012 13

12. Some possible learnings
 Planning  Analysis
 Knowing who to talk to is tough  Knowing what to look for is tough
 You can’t cover everyone in  You may need technical skills
limited time  People will say conflicting stuff –
 You will probably identify more you need to probe
people, documents, etc  You can’t uncover every issue –
 You need to time to analyse you have to focus
and frame an actionable report  Balancing breadth versus depth
 You have to pace yourself or is tough
you’ll burn out  It’s easy to make inferences
based on limited information, or
to be overwhelmed with data
(And you didn’t have to deal with all the negotiating and so on that
happens when managing stakeholders)
There is never enough time!
Project Reviews, Assurance, PMO
Dec 2012 14

13. Some possible learnings
 Data Gathering
 Getting stakeholders together  You may need depth for results
may mean they spark off each to be actionable – evidence
other adds to credibility and specific
details help action.
 Two people may make slower
progress together initially (e.g.  There’s always a risk that you’ll
interview coverage), but they miss stuff. This damages the
project and your credibility.
may also fill gaps in each
Don’t let this freeze you. Be
other’s coverage organised and open minded to
 There’s always noise that you’ll minimize the risk.
need to untangle (if it was
obvious, others would have
covered it)
 Organisations & teams have
inconsistent terminology
Project Reviews, Assurance, PMO
Dec 2012 15

14. Some thoughts
 Which sources will give you a quick overview?
 Review & project sponsors, programme manager
 Risk register
 Business case
 Plans
 Are the documents still current? Are they well maintained?
 What does your experience tell you to look for?
 Patterns in this company or industry?
 Bad smells?
 How do you hit the ground running?
 Define charter up front
 Clear process for data gathering, recording and analysis
 Checklists to help with planning (what to look for, what to ask)
 Heuristics and models to help you plan will buy time for data collection
and analysis
Project Reviews, Assurance, PMO
Dec 2012 16

15. Role of PMO
 Establish context for regular reviews
 Maintain review checklists and guidelines
 Monitoring effectiveness
 Tracking project-level actions
 Supporting programme- and portfolio-level actions
 Embedding lessons learned into organisational stds
 Providing administrative support
Project Reviews, Assurance, PMO
Dec 2012 17

16. Thank You
graham@grahamoakes.co.uk
@GrahamDOakes
Projects fail when...
Dec 2012 18

17. Useful Models
Project Reviews, Assurance, PMO
Dec 2012 19

18. Types of Project Review
 Timing  Attributes
 Event-based  Objectives
 Periodic  Status
 Ad hoc  Risk
 Quality
 Degree of Independence  Process
 Independent assurance  Compliance
 Peer
 Self-assessment  Summative vs Formative
 Degree of formality
 Spectrum
Project Reviews, Assurance, PMO
Dec 2012 20

19. Gate or Regular Review?
Gate Review Regular Review
Depth • Tend to go deep into • Tend to be more
project status and issues lightweight
Trends • Infrequent • Regular reviews can
track trends
Reviewer • Takes time to get up to • Get to know the project
Overhead
speed
Project • Can be disruptive • Small but frequent
Overhead
disruption
Objectivity • Reviewers are well • Risk of going native
separated
Specialist • May be needed for deep • Can often spot trends
skills
review without them
Project Reviews, Assurance, PMO
Dec 2012 21

20. Peer review or independent assurance?
Peer Review Independent Assurance
Availability of • Must find time from own • Dedicated team
Reviewers
projects
Skills of • Non-specialist • Rigorous approach. Develop
Reviewers
skills in gathering evidence,
conducting interviews, etc
Understanding • Reviewers are managing • Reviewers risk losing touch.
of Projects
similar projects themselves.
Understanding • See many projects
of Issues
Relationship to • Open, friendly relationship • Risk being adversarial
Proj Team
Organisational • Good way to share • Risk focusing on assessing
Learning
experiences projects
Match to • Organisational learning • Executive info & assurance
Overall
Objectives
• Formative reviews • Summative reviews
Project Reviews, Assurance, PMO
Dec 2012 22

21. Independent Assurance or Peer Review?
 Peer review  Independent Assurance
 Project team’s peers provide  Dedicated team focused on reviews
outside perspective (people may rotate through it)
 Probably take time off own projects  Work across multiple projects in a
to do it (so it can be hard to get programme or portfolio (so need to
time from reviewers) manage availability for each project)
 Provide advice and reality check to  Probably focused on checking status
project manager (& sponsor?) (but may provide advice), reporting
to sponsor or external executives
 Open and friendly, with emphasis  May become adversarial – “project
on 2-way learning from their peers police” – and lose learning focus
 Non-specialists may be  Can develop review skills and
unstructured in approach, and methods, but need to overcome
reluctant to challenge peers team resistance
 Dependent on team goodwill  Have authority (if dangerous to use!)
May mix roles, but helps to be clear which hat you wear
Project Reviews, Assurance, PMO
Dec 2012 23

22. Independent Assurance or Peer Review?
 Peer review  Independent Assurance
 Project team’s peers provide  Dedicated team focused on reviews
outside perspective (people may rotate through it)
 Probably take time off own projects  Help across multiple projects in a
Work executives manage
Help the project team
to do it (so it can be hard to get programme or activities need to
manage the project external portfolio (so
time from reviewers) manage availability for each project)
 Provide advice and reality check to  Probably focused on checking status
project manager (& sponsor?) (but may provide advice), reporting
to sponsor or external executives
 Open and friendly, with emphasis  May become adversarial – “project
on 2-way learning from their peers police” – and lose learning focus
 Non-specialists may be  Can develop review skills and
Help sponsors manage interventions
unstructured in approach, and methods, but need to overcome
reluctant to challenge peers team resistance
 Dependent on team goodwill  Have authority (if dangerous to use!)
May mix roles, but helps to be clear which hat you wear
Project Reviews, Assurance, PMO
Dec 2012 24

23. Types of Project Review
 You can mix up the types
 Formal, independent gateways at key points during initiation
 Regular peer reviews during execution
 Occasional health checks
 Self-assessments for low-risk projects or high-capability teams
 Shift from objectives, risk and process to quality and status
 How much to budget?
 ½ to 2%?
Project Reviews, Assurance, PMO
Dec 2012 25

24. Review process (Chapters 3, 4, 5)
Control parameters Criteria
Baseline Reference Feedback
Models to improve
reference
models
Inputs Review execution Outputs
Artefacts & other Analysis Loop
items to review, plus Go / No -go Improved
supporting details. decision. artefacts.
Recommendations to
improve review artefacts
Project Reviews, Assurance, PMO
Dec 2012 30

25. Analysis Loop (Chapter 6)
Raw Data
(e.g. Documents Structured Data
& Interviews) (e.g. Issues Log)
Analysis drives
additional data gathering
(e.g. to buy information on key risks) Analysis
&
Hypothesis
Generation
Project Reviews, Assurance, PMO
Dec 2012 37

26. Analysis Loop Raw Data
(e.g. Documents
& Interviews)
Structured Data
(e.g. Issues Log)
 Iterations may be based on Analysis drives
additional data gathering
(e.g. to buy information on key risks) Analysis
 Breadth then depth &
Hypothesis
Generation
 Risk
 Dependencies
 …
 Gather information into “issues log” for analysis and clustering
 Hence build models of potential problems, and of information we
might gather to refine our understanding and confirm / disconfirm
 Build traceability from problem to issues cluster to detailed issues
and their supporting notes
 You’ll often start with some sort of hypothesis: be aware of biases
Project Reviews, Assurance, PMO
Dec 2012 38

27. Interviewing (Chapter 6)
1) Planning
2) Interview Preparation
3) Pre-interview
4) Opening
5) Mid-Interview
6) Closing
7) Post-Interview
Project Reviews, Assurance, PMO
Dec 2012 40

28. Interview Protocol
 Questions
 Closed
 Open
 Clarifying
 Metaquestions
 Demonstrations
 Use a mix of question types!
 Directive or non-directive interviewing?
Project Reviews, Assurance, PMO
Dec 2012 43

29. Interviewing
 Don’t be afraid of pauses and silences
 LISTEN
 Use note-taking to create space
 Don’t let the protocol dominate
(I throw it away)
 Have a plan, but follow the energy
 Check opinions, assumptions and assertions: “what
leads you to believe that?”
Project Reviews, Assurance, PMO
Dec 2012 47

30. Observation
 The ability to gather data is essential to assessment. Data comes in
a variety of forms other than documents & interviews:
 Interaction
 Body language
 The environment
“You can observe a lot just by watching”
- Yogi Barra
Project Reviews, Assurance, PMO
Dec 2012 50

31. Checklists
 Guide your activities – help avoid gaps, prioritise
 Help capture learning
 Checklists on disk
 Project review
 Sponsor Interview
 Business Case
 Project Process / Policy
 Boundary Definition
 Plan Review
 Project Manager
 Diagnostics
(These are starting points. They’re not complete – adapt for your context)
Project Reviews, Assurance, PMO
Dec 2012 53

32. Assurance as an information conduit
 Project teams often have
problems communicating key messages:
 Lack savvy
 Lack credibility
 Lack access
 Assurance teams can help them frame their messages and
communicate them effectively
 Likewise, sponsors often have trouble picking the critical information
from the mass of stuff that comes their way
 Assurance teams can help them recognise key information and
frame appropriate interventions
Project Reviews, Assurance, PMO
Dec 2012 67

33. Thank you
Project Reviews, Assurance, PMO
Dec 2012 69

34. Graham Oakes Ltd
 Making sense of technology…
 Many organisations are caught up in the
complexity of technology and systems.
 This complexity may be inherent to the
technology itself. It may be created by the pace of technology change. Or it may arise from
the surrounding process, people and governance structures.
 We help untangle this complexity and define business strategies that both can be
implemented and will be adopted by people throughout the organisation and its partner
network. We then help assure delivery of implementation projects.
 Clients…
 Dover Harbour Board – Systems and architecture review
 Council of Europe – Defined systems for monitoring compliance with international treaties
 Sony Computer Entertainment – Defined common product approval process for global units
 National Savings & Investments – Helped NS&I and BPO partner develop joint IS strategy
 Amnesty International – Defined ECMS strategy for researchers, activists and campaigners
 Cisco Worldwide Education – Researched competitive marketplace for e-learning assets
 The Open University – Defined enterprise architecture, CRM and product development strategies
 Oxfam – Helped defined strategy for content management, CRM, e-Commerce
 MessageLabs – Implementation assurance for customer service portal
 Sapient Ltd – Risk management strategy for customer billing solution
Project Reviews, Assurance, PMO
Dec 2012 70
Some images & clipart in this presentation are © JupiterImages Corporation