Your SlideShare is downloading. ×
itSMF Scottish Regional Meeting - project review simulation - 5 Mar 2013
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

itSMF Scottish Regional Meeting - project review simulation - 5 Mar 2013

232

Published on

Slides from my session with the itSMF Scottish Regional meeting on 5 Mar, 2013.

Slides from my session with the itSMF Scottish Regional meeting on 5 Mar, 2013.

Published in: Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
232
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Establishing context for regular reviews.  • Providing a pool of expertise  • Maintaining review checklists and guidelines  • Monitoring effectiveness  • Tracking project-level actions  • Supporting portfolio- and programme-level actions  • Embedding lessons learned into organisational standards  • Providing administrative suppor
  • Transcript

    • 1. Why Do IT Projects Fail? Project Review SimulationWhy do IT projects failMar 2013 1
    • 2. Agenda10 min -- Introductions60 min -- Project review simulation45 min -- Debrief (& models)Why do IT projects failMar 2013 2
    • 3. Objectives Think a little about why IT projects fail Explore use of project reviews to help identify failures Share experience and expertiseWhy do IT projects failMar 2013 3
    • 4. Introductions Who are you? Why are you here?Why do IT projects failMar 2013 4
    • 5. People lose touch with reality The single underlying cause for most project failures…Why do IT projects failMar 2013 5
    • 6. Role of the reviewer… … is to help people keep in touch with reality PMI Research About 75% of the things that go wrong on projects, someone knew about the issue but didn’t know how to deal with / who to tell about it.Why do IT projects failMar 2013 6
    • 7. Assessing project Airfix  24:1 scale model (1 hour = 3 working days)  6 teams  Conduct a health check of project AirfixWhy do IT projects failMar 2013 7
    • 8. Assessing project Airfix - Rules 1) You have  Project briefing  Team / resource list  Planning framework 2) Read briefing & identify who / what you want to see over the 3 days 3) I will give document outlines & interview notes in line with your plan 4) Analyse notes and present findings (2 mins) 5) Describe how you got these findingsWhy do IT projects failMar 2013 8
    • 9. Findings 2 minutes (= 45 mins in real time)1. Present your key findings2. Describe how you got to these findingsWhy do IT projects failMar 2013 9
    • 10. Debrief What was common to your findings? Different? Why? What was common to your plans? Different? Why? How did you feel during the exercise?  What was challenging?  What was confusing?  What was interesting? How was this like real life? How did it differ? What would help make reviews easier?Why do IT projects failMar 2013 10
    • 11. Some possible learnings Planning  Analysis  Knowing who to talk to is tough  Knowing what to look for is tough  You can’t cover everyone in  You may need technical skills limited time  People will say conflicting stuff –  You will probably identify more you need to probe people, documents, etc  You can’t uncover every issue –  You need to time to analyse you have to focus and frame an actionable report  Balancing breadth versus depth  You have to pace yourself or is tough you’ll burn out  It’s easy to make inferences based on limited information, or to be overwhelmed with data (And you didn’t have to deal with all the negotiating and so on that happens when managing stakeholders) There is never enough time!Why do IT projects failMar 2013 11
    • 12. Some possible learnings Data Gathering  Getting stakeholders together  You may need depth for results may mean they spark off each to be actionable – evidence other adds to credibility and specific details help action.  Two people may make slower progress together initially (e.g.  There’s always a risk that you’ll interview coverage), but they miss stuff. This damages the project and your credibility. may also fill gaps in each Don’t let this freeze you. Be other’s coverage organised and open minded to  There’s always noise that you’ll minimize the risk. need to untangle (if it was obvious, others would have covered it)  Organisations & teams have inconsistent terminologyWhy do IT projects failMar 2013 12
    • 13. Some thoughts Which sources will give you a quick overview?  Review & project sponsors, programme manager  Risk register  Business case  Plans  Are the documents still current? Are they well maintained? What does your experience tell you to look for?  Patterns in this company or industry?  Bad smells? How do you hit the ground running?  Define charter up front  Clear process for data gathering, recording and analysis  Checklists to help with planning (what to look for, what to ask)  Heuristics and models to help you plan will buy time for data collection and analysisWhy do IT projects failMar 2013 13
    • 14. Review process (Chapters 3, 4, 5) Control parameters Criteria Baseline Reference Feedback Models to improve reference modelsInputs Review execution OutputsArtefacts & other Analysis Loopitems to review, plus Go / No -go Improvedsupporting details. decision. artefacts. Recommendations to improve review artefacts Why do IT projects fail Mar 2013 14
    • 15. Analysis Loop (Chapter 6) Raw Data (e.g. Documents Structured Data & Interviews) (e.g. Issues Log) Analysis drives additional data gathering (e.g. to buy information on key risks) Analysis & Hypothesis GenerationWhy do IT projects failMar 2013 15
    • 16. Establishing effective reviews Establish context for regular reviews Maintain review checklists and guidelines Monitor effectiveness Track project-level actions Support programme- and portfolio-level actions Embed lessons learned into organisational standards Provide administrative supportWhy do IT projects failMar 2013 16
    • 17. Assurance as an information conduit Project teams often have problems communicating key messages:  Lack savvy  Lack credibility  Lack access Assurance teams can help them frame their messages and communicate them effectively Likewise, sponsors often have trouble picking the critical information from the mass of stuff that comes their way Assurance teams can help them recognise key information and frame appropriate interventionsWhy do IT projects failMar 2013 17
    • 18. Thank Yougraham@grahamoakes.co.uk@GrahamDOakesWhy do IT projects failMar 2013 18
    • 19. Useful ModelsWhy do IT projects failMar 2013 19
    • 20. Types of Project Review Timing  Attributes  Event-based  Objectives  Periodic  Status  Ad hoc  Risk  Quality Degree of Independence  Process  Independent assurance  Compliance  Peer  Self-assessment  Summative vs Formative Degree of formality  SpectrumWhy do IT projects failMar 2013 20
    • 21. Gate or Regular Review? Gate Review Regular Review Depth • Tend to go deep into • Tend to be more project status and issues lightweight Trends • Infrequent • Regular reviews can track trends Reviewer • Takes time to get up to • Get to know the project Overhead speed Project • Can be disruptive • Small but frequent Overhead disruption Objectivity • Reviewers are well • Risk of going native separated Specialist • May be needed for deep • Can often spot trends skills review without themWhy do IT projects failMar 2013 21
    • 22. Peer review or independent assurance? Peer Review Independent Assurance Availability of • Must find time from own • Dedicated team Reviewers projects Skills of • Non-specialist • Rigorous approach. Develop Reviewers skills in gathering evidence, conducting interviews, etcUnderstanding • Reviewers are managing • Reviewers risk losing touch. of Projects similar projects themselves.Understanding • See many projects of IssuesRelationship to • Open, friendly relationship • Risk being adversarial Proj TeamOrganisational • Good way to share • Risk focusing on assessing Learning experiences projects Match to • Organisational learning • Executive info & assurance Overall Objectives • Formative reviews • Summative reviewsWhy do IT projects failMar 2013 22
    • 23. Independent Assurance or Peer Review? Peer review  Independent Assurance  Project team’s peers provide  Dedicated team focused on reviews outside perspective (people may rotate through it)  Probably take time off own projects  Work across multiple projects in a to do it (so it can be hard to get programme or portfolio (so need to time from reviewers) manage availability for each project)  Provide advice and reality check to  Probably focused on checking status project manager (& sponsor?) (but may provide advice), reporting to sponsor or external executives  Open and friendly, with emphasis  May become adversarial – “project on 2-way learning from their peers police” – and lose learning focus  Non-specialists may be  Can develop review skills and unstructured in approach, and methods, but need to overcome reluctant to challenge peers team resistance  Dependent on team goodwill  Have authority (if dangerous to use!) May mix roles, but helps to be clear which hat you wearWhy do IT projects failMar 2013 23
    • 24. Independent Assurance or Peer Review? Peer review  Independent Assurance  Project team’s peers provide  Dedicated team focused on reviews outside perspective (people may rotate through it)  Probably take time off own projects  Help across multiple projects in a Work executives manage Help the project team to do it (so it can be hard to get programme or activities need to manage the project external portfolio (so time from reviewers) manage availability for each project)  Provide advice and reality check to  Probably focused on checking status project manager (& sponsor?) (but may provide advice), reporting to sponsor or external executives  Open and friendly, with emphasis  May become adversarial – “project on 2-way learning from their peers police” – and lose learning focus  Non-specialists may be  Can develop review skills and Help sponsors manage interventions unstructured in approach, and methods, but need to overcome reluctant to challenge peers team resistance  Dependent on team goodwill  Have authority (if dangerous to use!) May mix roles, but helps to be clear which hat you wearWhy do IT projects failMar 2013 24
    • 25. Types of Project Review You can mix up the types  Formal, independent gateways at key points during initiation  Regular peer reviews during execution  Occasional health checks  Self-assessments for low-risk projects or high-capability teams  Shift from objectives, risk and process to quality and status How much to budget?  ½ to 2%?Why do IT projects failMar 2013 25
    • 26. Review process (Chapters 3, 4, 5) Control parameters Criteria Baseline Reference Feedback Models to improve reference modelsInputs Review execution OutputsArtefacts & other Analysis Loopitems to review, plus Go / No -go Improvedsupporting details. decision. artefacts. Recommendations to improve review artefacts Why do IT projects fail Mar 2013 26
    • 27. Control parameters Baseline Criteria ReferenceOutputs Feedback Models to improve reference models Review execution Inputs Outputs Artefacts & other Analysis Loop items to review, plus Go / No -go Improved supporting details. decision. artefacts. Recommendations to improve review artefacts It helps to be clear about the outputs and how they will be used The review will generally  Produce recommendations to improve the inputs.  Provide input to go/no-go and other decisions (by giving assurance (or otherwise) as to the fitness-for-purpose of work done to date and proposed) In defined circumstances, the independent assurance team may also trigger escalation processes to other stakeholders.Why do IT projects failMar 2013 27
    • 28. Control parameters Baseline Criteria ReferenceInputs Feedback Models to improve reference models Review execution Inputs Outputs Artefacts & other Analysis Loop items to review, plus Go / No -go Improved supporting details. decision. artefacts. Recommendations to improve review artefacts What documents, interviews, etc will we be reviewing? What supporting materials may be needed? (e.g. historical context, detailed specifications, etc) We can’t review everything!  Many review teams try to review all elements of a project.  This diffuses effort across low-risk or low-impact items, reducing the overall effectiveness and efficiency of the review.  By clearly defining the inputs, we ensure that effort is focused where it will do most to reduce project risk. (Breadth then depth iterations may also help.)Why do IT projects failMar 2013 28
    • 29. Control parameters Baseline Criteria ReferenceCriteria Feedback Models to improve reference models Review execution Inputs Outputs Artefacts & other Analysis Loop items to review, plus Go / No -go Improved supporting details. decision. artefacts. Recommendations to improve review artefacts Criteria define what aspects of the inputs to focus on. Can review against many criteria. (Budget, time, process compliance, security, performance, usability, …) Reviews that try to cover every angle end up skating across the surface. What is most important to this review? For example, if project is constrained by tight budget, let’s focus energy on costs. Typically 2-3 criteria per review. Schedule additional reviews to cover more criteria.Why do IT projects failMar 2013 29
    • 30. Control parameters Baseline Criteria ReferenceWhat doesn’t need to Feedback Models to improve reference models Review execution Inputs Outputsbe reviewed? Artefacts & other items to review, plus supporting details. Analysis Loop Go / No -go decision. Improved artefacts. Recommendations to improve review artefacts Baseline defines what’s been agreed prior to the review.  For example, if we’re reviewing a project plan, we measure it against it’s likelihood of delivering the objectives.  Input artefacts are “fit for purpose” if they meet their objectives against the baseline.  The baseline will often have been reviewed in a stage prior to the current review. Reference Models define external practices agreed for our context.  The input artefacts conform to best practice to the extent that they align to these reference models.  By agreeing reference models up front, we can avoid many of the debates that occur between review and project teams as to whether designs or implementations conform to someone’s perception of “best practice”.Why do IT projects failMar 2013 30
    • 31. Control parameters Baseline Criteria ReferenceFeedback loops Feedback Models to improve reference models Review execution Inputs Outputs Artefacts & other Analysis Loop items to review, plus Go / No -go Improved supporting details. decision. artefacts. Recommendations to improve review artefacts First feedback loop is to improve the inputs (plans, designs, project processes, etc). Second feedback loop updates our database of good practices, building in organisational learning:  Improvements to organisational standards and processes  Improvements to review checklists (e.g. new questions, retired questions) Project reviews can be a key contributor to organisational learningWhy do IT projects failMar 2013 31
    • 32. Control parameters Baseline Criteria ReferenceExecution Feedback Models to improve reference models Review execution Inputs Outputs Artefacts & other Analysis Loop items to review, plus Go / No -go Improved supporting details. decision. artefacts. Recommendations to improve review artefacts One time vs. Ongoing Review (Oversight)? What time, resources & expertise are available for the review team? (Will you need subject matter experts to assist with details?) What access to the project team, documents, etc is available? How about administrative support? (Scheduling interviews and meeting rooms can take a lot of time.) Preserving confidentiality of interviews* Security requirements for review results or project artifacts? May preliminary findings be shared to allow confirmation/correction?Why do IT projects failMar 2013 *You can’t promise confidentiality if you discover illegal activity 32
    • 33. Analysis Loop (Chapter 6) Raw Data (e.g. Documents Structured Data & Interviews) (e.g. Issues Log) Analysis drives additional data gathering (e.g. to buy information on key risks) Analysis & Hypothesis GenerationWhy do IT projects failMar 2013 33
    • 34. Analysis Loop Raw Data (e.g. Documents & Interviews) Structured Data (e.g. Issues Log) Iterations may be based on Analysis drives additional data gathering (e.g. to buy information on key risks) Analysis  Breadth then depth & Hypothesis Generation  Risk  Dependencies  … Gather information into “issues log” for analysis and clustering Hence build models of potential problems, and of information we might gather to refine our understanding and confirm / disconfirm Build traceability from problem to issues cluster to detailed issues and their supporting notes You’ll often start with some sort of hypothesis: be aware of biasesWhy do IT projects failMar 2013 34
    • 35. Interviewing (Chapter 6)1) Planning2) Interview Preparation3) Pre-interview4) Opening5) Mid-Interview6) Closing7) Post-InterviewWhy do IT projects failMar 2013 35
    • 36. Interviewing1) Planning  How do interviews contribute to review objectives?  Number and scope of interviews  Type of interview – individual, pair or workshop?  Who to interview  Establish relationships  Set up logistics  Quiet, private space to interview in  Timing, maps, water, etcWhy do IT projects failMar 2013 36
    • 37. Interviewing2) Interview preparation  Review project documentation  Briefing pack (explain objectives and process)  Self-assessment questionnaires  Agree roles (e.g. if interviewing in pairs)  Develop interview protocolWhy do IT projects failMar 2013 37
    • 38. Interview Protocol Questions  Closed  Open  Clarifying  Metaquestions  Demonstrations Use a mix of question types! Directive or non-directive interviewing?Why do IT projects failMar 2013 38
    • 39. Interviewing3) Pre-interview  Review objectives  Review background to interviewee  Manage the interview space  Be on time  Be alert & ready to listenWhy do IT projects failMar 2013 39
    • 40. Interviewing4) Opening  Set people at ease!  Introductions  Names  Organisational roles  Role on this review & interview  Objectives (or review, of interview)  Confirm timing  Confirm note taking, confidentiality & other protocols  Do they have any questions?Why do IT projects failMar 2013 40
    • 41. Interviewing5) Mid-Interview  Early questions build rapport  When did you join the project? What is your role?  Gather basic facts  Give people space to expand  Confirm you’ve heard correctly – listen actively, recap  Confirm evidence for opinions, assumptions & assertions: “What makes you believe that?”  Record notes (What are you recording? How?)  Listen & observe (e.g. body language & emotional issues)Why do IT projects failMar 2013 41
    • 42. Interviewing Don’t be afraid of pauses and silences LISTEN Use note-taking to create space Don’t let the protocol dominate (I throw it away) Have a plan, but follow the energy Check opinions, assumptions and assertions: “what leads you to believe that?”Why do IT projects failMar 2013 42
    • 43. Interviewing6) Closing  Respect interviewee’s time – reschedule if necessary  Confirm you’ve heard main messages  Final chance to gather new info  Metaquestions  Is there anything else I should be asking?  What else did you expect me to ask?  Is there anyone else you should see?  Exchange contact details  Confirm next steps and protocol for checking / follow-up  Thank them for their timeWhy do IT projects failMar 2013 43
    • 44. Interviewing7) Post-interview  Debrief  Record notes  I quickly lose ability to read my handwriting…  It helps me think about follow-up activities  Log new documents and other artefacts  Record issues  Confirm notes  This may gather new info.  What people change is often interesting.  Schedule any follow-up activitiesWhy do IT projects failMar 2013 44
    • 45. Observation The ability to gather data is essential to assessment. Data comes in a variety of forms other than documents & interviews:  Interaction  Body language  The environment “You can observe a lot just by watching” - Yogi BarraWhy do IT projects failMar 2013 45
    • 46. Graham Oakes Ltd Making sense of technology…  Many organisations are caught up in the complexity of technology and systems.  This complexity may be inherent to the technology itself. It may be created by the pace of technology change. Or it may arise from the surrounding process, people and governance structures.  We help untangle this complexity and define business strategies that both can be implemented and will be adopted by people throughout the organisation and its partner network. We then help assure delivery of implementation projects. Clients…  Dover Harbour Board – Systems and architecture review  Council of Europe – Defined systems for monitoring compliance with international treaties  Sony Computer Entertainment – Defined common product approval process for global units  National Savings & Investments – Helped NS&I and BPO partner develop joint IS strategy  Amnesty International – Defined ECMS strategy for researchers, activists and campaigners  Cisco Worldwide Education – Researched competitive marketplace for e-learning assets  The Open University – Defined enterprise architecture, CRM and product development strategies  Oxfam – Helped defined strategy for content management, CRM, e-Commerce  MessageLabs – Implementation assurance for customer service portal  Sapient Ltd – Risk management strategy for customer billing solutionWhy do IT projects failMar 2013 46 Some images & clipart in this presentation are © JupiterImages Corporation

    ×