Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Chief Skeptical Officer – A New C-Level Role


Published on

Published in: Business, Education, Technology
  • Be the first to comment

The Chief Skeptical Officer – A New C-Level Role

  1. 1. Chief Skeptical Officer – ANew C-Level RoleAlan McSweeney
  2. 2. Why Does This Happen? Not What Is Wanted Or Too Late What Was The Big Idea Required/ Envisaged Reduced Functionality Lack Of Requiring Integration Manual Workarounds And Continue To The Big Failure Happen Time and Again? Requires Too Expensive Rework Or Replacement And What Can Be Done to Stop It? High Cost Of Not Scalable And/Or Poor Operation Performance October 18, 2011 2
  3. 3. Spectrum Of Failures Performance and/or Specified Operational Business Benefits Problems and Savings Not Functionality Delivered Delivered Does not More Expensive Meet Business to Operate Than Requirements Planned Significant ReworkProject Late Requiredand/or Over Budget Solution Largely Unused And/Or Unusable Complete Project Success: Increasing Frequency/ Complete ProjectOn-time, On-budget and Delivering Probability of Failure: Cancelled, Unused, Rejected Specified Benefits Occurrence October 18, 2011 3
  4. 4. What Is The Role Of The Chief Skeptical Officer?• Role with end-to-end view and responsibility from idea definition to delivery• Provide positive skepticism around idea definition and solution specification• Be an independent voice• Be an advocate for and champion of standards• Maintain a repository of learning and standards• Embed learning into the organisation October 18, 2011 4
  5. 5. The Key Functions Of The Chief Skeptical Officer Remove Arbitrariness, Provide Independent Relativity And Oversight of Decisions Subjectivity Require Assertions To Be Well Supported By Evidence Ensure Rational Decision-Making Avoid Systematic Distortion and Identify and Eliminate Misrepresentation Behavioural And Other Ensure Evidence-Based Biases On Decision- Judgements Making Question Assumptions, Attitudes Of Track End-To-End View Knowledge, Opinions From Concept To and Beliefs Stated As Delivery Facts October 18, 2011 5
  6. 6. The Journey From Idea To Successful Operation Idea, Business Need, Business Benefits Process Definition and Solution Design Costing - Implementation and Operational Solution Implementation and Delivery Solution Operation Benefits Realised October 18, 2011 6
  7. 7. The Journey From Idea To Successful Operation Idea, Business Need, Business Benefits But Promised Process Definition and Benefits All Too Solution Design Frequently Not Costing - Delivered Implementation and Operational Solution Implementation and Delivery Solution Operation Benefits Realised October 18, 2011 7
  8. 8. Frequently Too Many Handoffs On The JourneyFrom Idea To Successful Operation Idea, Business Handoff and Need, Business Benefits Information Loss Process Handoff and Definition and Solution Design Information Loss Costing - Handoff and Implementation Information Loss and Operational “Chinese Whisper” Solution Handoff and Implementation Information Loss Effect as Initial and Delivery Concept Moves to Implementation Handoff and Solution Operation Information Loss Benefits Realised October 18, 2011 8
  9. 9. What Causes the Disconnect From Idea To Delivery Mi sco Idea, Business mm Too Need, Business u n Ma Benefits ica ny tio Ha n, nd Iss o Process To ues ffs B -En Le etw Definition and d V ft U ee Solution Design iew nre n S or solv tage En ed s, Costing - d-T , A To Implementation o-E ssu o O and Operational nd mp ften Re tio Se sp ns pa on M ra Solution sib ad te ilit e, N Te Implementation y o O ams and Delivery ne , W ith Solution En d- Operation Benefits Realised October 18, 2011 9
  10. 10. What Can Go Wrong Along The Journey From IdeaTo Successful Operation Poorly Defined Need, Benefits Exaggerated Idea, Business Need, Business Benefits Inadequate Analysis, Poor Or Incomplete Process Design Definition and Solution Design Overly Optimistic Initial and Operational Cost Costing - Estimates Implementation and Operational Poor Requirements Promised/ExpectedAnalysis, Poor Design, Solution Implementation Benefits Not Delivered Poor Management and Delivery Slow, Unreliable, Solution Expensive To Operate, Operation Not Integrated, Requires Manual Workarounds Benefits Realised October 18, 2011 10
  11. 11. Cost of Fixing Errors During Project Lifecycle Errors/ gaps/ omissions become significantly more Benefits Realised expensive to fix at later stages of the solution Solution Operation Solution Implementation and Delivery Costing - Implementation and Operational Process Definition and Solution Design Relative cost to remediate Idea, Business errors at the end can be 50- Need, Business 100 (or more) times more Benefits expensive than at the start October 18, 2011 11
  12. 12. Role Of The Chief Skeptical Officer Is To Take End-To-EndView, Enforce Discipline and Perform Reviews Idea, Business Need, Business Benefits Getting It Right Here … Process Definition and Solution Design … Reduces/Ch Avoids Problems iefEn Ske Costing - Here su pt re ica Implementation Th l O and Operational at Th ffice e S r’s tar Ro Solution t D le Implementation eli Is T and Delivery ve o rs Ta An ke d C En an d-T Solution De o-E Operation liv nd er Th View eE T nd o Benefits Realised October 18, 2011 12
  13. 13. What Is The Chief Skeptical Officer Trying To ProtectYou Against?• Cognitive Bias – Poor or inaccurate judgements, illogical interpretations and decisions, characterised by patterns of behaviour• Strategic Misrepresentation – Deliberate misrepresentation in budgeting caused by distorted incentives• Planning Fallacy – Systematic tendency to underestimate how long it will take to complete a task even when there is past experience of similar tasks over-running• Optimism Bias – Systematic tendency to be overly optimistic about the outcome of actions• Focalism – Systematic tendency to become inwardly focussed and to lose situational awareness and appreciation of wider context during times of stress October 18, 2011 13
  14. 14. Cognitive Bias - Types• Many classifications and types of cognitive bias• Can be very difficult to avoid because of their embedded nature and emotional/irrational basis − Decision-Making and Behavioural Biases - affecting belief formation and business decisions − Probability and Belief Biases - affecting way in which information is gathered and assessed − Attributional Biases - affecting the determination what was responsible for an event or action• Cognitive biases are very real and can have damaging effects October 18, 2011 14
  15. 15. Decision-Making And Behavioural Biases• Relying too heavily on one piece of • Placing too much importance on one information when making a decision - aspect - Focusing Effect Anchoring • Looking to reduce a small risk to zero• Believing things because many others rather than a greater reduction of a larger believe the same – Bandwagon risk – Zero-Risk Bias• Assigning greater weight to apparently • Rejecting new evidence that contradicts dominant factors – Attention Bias an established paradigm – Semmelweis• Interpreting information so as to that Effect confirms preconceptions – Confirmation • Making decisions based to what is pleasing• Seeing oneself as less biased than others – to imagine instead basing decisions on Blind Spot evidence and rationality – Wishful Thinking• Strong preference for immediate payoffs • Assigning a higher value to disposal/loss relative to later ones – Hyperbolic compared with cost of acquisition – Sunk Discounting Cost Effect• Greater preferences just because of • Viewing a harmful action as worse than an familiarity - Exposure Effect equally harmful omission or inaction –• Paying more attention and giving more Omission Bias weight to the negative rather than the • Justifying increased investment based on positive – Negativity Bias the cumulative prior investment despite• Looking for information even when it new evidence suggesting that the decision cannot affect action – Information Bias was wrong - Irrational Escalation October 18, 2011 15
  16. 16. Probability And Belief Biases• Excessive or inflated belief ones • Perceiving patterns where none exist performance, ability – Overconfidence – Clustering Effect • Selecting an options for which the• Belief gaining plausibility through probability of a favorable outcome is increasing repetition - Availability known over an option for which the Cascade probability of a favorable outcome is• Assigning greater weight to initial or unknown - Ambiguity Effect recent events more than subsequent • Considering information to be correct or later events – Serial Position Effect if it has any personal meaning or• Assigning a lower probability to the significance – Subjective Validation whole than the probabilities of the • Overestimating the likelihood of parts – Subadditivity Effect positive rather than negative• Avoidance of risk or the negative by outcomes – Valence Effect pretending they do not exist – Ostrich • Failure to examine all possible Effect outcomes when making a judgment –• Judging future events in a more Attentional Bias positive light than is warranted by actual experience – Optimism Bias October 18, 2011 16
  17. 17. Attributional Biases• Where skilled underrate their abilities and unskilled overrate their abilities – Dunning–Kruger Effect• Defending the status quo – System Justification• Overestimation of agreement – False Consensus Effect October 18, 2011 17
  18. 18. Strategic Misrepresentation• Deliberate misrepresentation in planning and budgeting caused by distorted incentives• Response to how organisations structure rewards and give rise to motivations• Systematic (and predictable) misrepresentation − Deliberately understand costs to gain acceptance with understanding that costs will increase − Not willing to face reality of high costs − Overstatement or understatement of requirements − Competition for scarce funds or jockeying for position − Inclusion of ideology into planning• Underlying system and processes need redesign to eliminate October 18, 2011 18
  19. 19. What Are The Key Disciplines The Chief SkepticalNeeds To Promote• Disciplines and Skills − Benefits Management and Realisation − Capital Planning and Investment Management − Business and Process Analysis − Solution and Process Analysis and Architecture Design• Practices − Reference Class Forecasting − Audits and Checklists October 18, 2011 19
  20. 20. Reference Class Forecasting• Technique to improve accuracy in plans and projections by basing them on actual performance in a reference class of comparable actions• Based on knowledge about actual performance − Analyse distributions and probability of cost overruns − Compare the proposed project with the reference class distribution to establish the most likely outcome• Used to avoid Planning Fallacy, Strategic Misrepresentation and Optimism Bias October 18, 2011 20
  21. 21. Benefits Management And Realisation• To increase the likelihood of success from IT and other investments, organisations must identify the different causes of benefits before developing any project implementation plan• Types of IT projects with very different approaches to benefits management − Fixing or improving something that exists • Resolve problem • Improve integration − Implementing a new initiative • New system • New processes• Need to focus on the changes needed to achieve results and take full advantage of new facilities offered rather than on just IT features• Effective change management is crucial to achieving benefits October 18, 2011 21
  22. 22. Achieving Benefits – Some Questions To Ask• Why is there a need to improve?• What improvements are needed? What improvements are possible or achievable? Have the improvements been agreed by all stakeholders?• What benefits will be realised by each stakeholder if the business objectives are achieved? How can each benefit be measured?• Who owns each of the benefits and will be accountable for its delivery?• What business changes are needed to achieve each benefit? Have the explicit links between each benefits and required business changes been identified?• Who will be responsible for ensuring the business changes are made successfully?• How and when can the changes be made? Who will make the changes? Does the business have the ability and capacity to make the changes? October 18, 2011 22
  23. 23. Benefits Management And Benefits DependencyNetwork• Benefits Dependency Network (BDN) is an approach to linking: − Information Technology Enablers, Changes and New Capabilities – enabling technologies and functions and facilities needed to support the realisation of the identified benefits and to allow the necessary changes to be undertaken − Business Changes – business activities and new ways of working that are required to ensure that the desired benefits are realised − Enabling Changes - prerequisites for achieving the business changes or that are essential to bring the new system into effective operation − Business Objectives - high level priorities in relation to the drivers, outcomes and improvements to be delivered on completion of the project − Business Benefits - outcomes of a change that are deemed to be positive by a stakeholder and that are valuable to the organisation and are measureable − Stakeholders - individuals or groups who will benefit from the project and are either affected by or directly involved in making the changes needed to realise the benefits October 18, 2011 23
  24. 24. Types Of Changes• Business Changes - Permanent changes to working practices, processes, procedures, interactions and relationships that will cause the benefits to be delivered − Generally need new IT system to be in place − May require enabling changes to be implemented• Enabling Changes - Typically one-time changes that are pre-requisites for making the business changes or are necessary to bring the new system into effective operation − Can be made in advance of system implementation − Training − Processes redesign − New work practices − Changes to job roles and responsibilities October 18, 2011 24
  25. 25. Benefits Dependency Network Provide for Require Allow Enable Delivery of Information Technology Enablers, Enabling Business Business Business Changes and Changes Changes Benefits Objectives New Capabilities• BDN provides a framework for explicitly linking the overall investment objectives and the requisite benefits with the business changes which are necessary to deliver those benefits and the essential IT functionality to both drive and enable these changes to be made.• BDN forms part of the benefits realisation plan• Helps keep the focus on benefits realisation during the program execution• Allows variations of the project or program to be assessed for their impact on benefits realisation October 18, 2011 25
  26. 26. Benefits Dependency NetworkMeans to Ways to Achieve Changes Results of ChangesAchieveChangesInformationTechnology Enablers, Enabling Business Business BusinessChanges and Changes Changes Benefits Objectives NewCapabilities October 18, 2011 26
  27. 27. Benefits Dependency NetworkUnderstand Why the Business Will Use the Information Technology System What What the Why the What Users What Information Business Business Do With Information Technology Gains From Uses The Information Technology Capabilities Information Information Technology Capabilities and Features Technology Technology Capabilities Achieve Offer Capabilities CapabilitiesUnderstand What Information Technology System is Needed to Deliver on Requirements What What the Information What the What the What the Business Technology Business Has Business Business Needs to Capabilities to Do Needs To See Wants Provide Are Required October 18, 2011 27
  28. 28. Benefits Dependency Network• Shows the link from the solution, through business activities, outcomes and benefits to the organisations overall drivers• Used to confirm that the solution being introduced will actually provide the results you are seeking• Any function within the proposed solution that is not linked to a benefit is potentially of doubtful value• Functions with many link or links to key benefits can be identified and given extra attention• BDN is a complex approach that requires benefit identification, assessment, validation and realisation maturity within the organisation• Imposes a rigour on the organisation in analysing benefits from projects October 18, 2011 28
  29. 29. Capital Planning And IT And Other Investment CoreRequirements• Determine the scale, scope, and sources of funding for IT and other areas• Assign financial resources to competing activities within the IT portfolio• Establish a balance between capital expenditure (new projects) and operating expenditure (running systems delivered by past projects)• Optimise the total cost of ownership• Manage IT and other portfolios for value and not just cost• IT and other areas need to implement a process for justifying its costs and be seen to be taking these steps October 18, 2011 29
  30. 30. Capital Planning And IT and Other InvestmentManagement• Aligns IT and other investments to organisation strategy (scoring)• Prioritises investments (ranking)• Provides strategic criteria for investment analysis• Conduct annual IT and other portfolio management reviews• Provides recommendation to stop, slow, maintain or accelerate program funding• Identifies redundant/inefficient systems• Integrates IT and other architectures within investments• Ensures compliance with funding standards October 18, 2011 30
  31. 31. Characteristics Of Credible Cost Estimates• Clear identification of requirements of the ultimate deliverable• Broad participation in preparing estimates• Availability of valid data for performing estimates – historical, experience, benchmarks• Standardised and comprehensive estimate structure that includes all possible sources of cost• Provision for uncertainties – include known costs explicitly and allow for unknown costs• Recognition of inflation• Recognition of excluded costs• Independent review of estimates for completeness and realism• Revision of estimates for significant changes in requirements October 18, 2011 31
  32. 32. Challenges Of Developing Good Cost Estimates• Requires detailed, stable, agreed requirements• Agreed assumptions• Access to detailed documentation and historical data for comparison• Trained and experienced analysts• Risk and uncertainty analysis• Identification of a range of confidence levels• Adequate contingency and management reserves October 18, 2011 32
  33. 33. Reasons for Good And Bad Cost Estimates Ineffe and U ctive Risk Effect ncer i ve Unce Risk and Unfa Analy tainty r ta Techn miliar s is Ident ificat Analy inty First- ology or s is Rang ion of a Time Use Probl em Confi e of De Acces s Getting dence Level Docu tailed s to D m ata Unre s and H entation Unre Proje asonable Adeq ua istor Train Unre alistic or ct Bas Conti ngen te Data ical e Exper d and liable Data Unre elin e Mana cy an ien geme d Detai led, S Analy ced Assumalistic Reser nt sts No o ption ves Agree table, Comp r Limited s Overo Requ d ariso ptimi ireme Agr Avail n Data sm n ts Assum eed able New ption Pr ocess s e s Untra Proje Inexp ined and ct Ins er ie t abilit y Comp Analy nced le sts or Te x Project chnol Unre ogy alistic Savin Project gs• Lost of reasons for and causes of inaccurate cost estimates October 18, 2011 33
  34. 34. Sources of Risk and Uncertainty In Estimating Costs• Lack of understanding of the project requirements• Shortcomings of human language and differing interpretations of meaning of project• Behaviour of parties involved in the cost estimation process• Haste• Deception• Poor cost estimating and pricing practices October 18, 2011 34
  35. 35. IT Investment• IT is commonly seen as failing to deliver value for money• Benefits and value must be actively managed for• Realising and assessing business benefits from IT-enabled investments involves more than simply assessing Total Cost of Ownership for IT-related projects and managing the IT budget• Key requirements − Ability to get lifetime costs right − Ability to define benefits correctly and effectively − Ability to manage the benefits management process − Ability to increase and sustain benefits management maturity• Use existing methodologies and frameworks to implement key requirements quickly − ITIM − Benefits Dependency Network − ValIT − Organisational change and commitment• Effective benefits management enables organisations to clearly and consistently articulate IT’s contribution to achievement of business objectives October 18, 2011 35
  36. 36. Business and Process Analysis And Design BuildsBridge From Business To Solution Problem Requirement Business Analysis: Elicit Requirements Current State Analyse Communicate Business Analysis Validate and Solution Design Solution Design: Translate Requirements into Solution Solution Desired Future State• Business analysis is a key driver of business value• Solution delivery start with business analysis October 18, 2011 36
  37. 37. Weaknesses In Business Analysis Capabilities And Competencies At The Root of Many Project Failures Poor Size/Capacity/ Poor Strategic Requirements Complexity Alignment Poor Analysis Large Project, Inadequate Business Practices Complex, Difficult Case, Undefined Changes and Processes Problem/NeedBusiness Requirements Large Project Team and Business Benefits Business Needs Not Captured Multiple Stakeholders Not Measured Not Met Opportunities Lost Investment Inadequate Resource Unproven Technology Inadequately Explored Wasted Allocation and Solution Options PrioritisationInadequate Business Dynamic, Changing Solution Design Not Involvement Environment Aligned to Business Needs Poor Focus on Uncertainly/ Poor Solution Business Needs Ambiguity Design October 18, 2011 37
  38. 38. Analysis-Related Causes Of Failures Poor Size/Capacity/ Poor Strategic Requirements Complexity Alignment Poor Analysis Large Project, Inadequate Business Practices Complex, Difficult Case, Undefined Changes and Processes Problem/NeedBusiness Requirements Large Project Team and Business Benefits Business Needs Not Captured Multiple Stakeholders Not Measured Not Met Opportunities Lost Investment Inadequate Resource Unproven Technology Inadequately Explored Wasted Allocation and Solution Options PrioritisationInadequate Business Dynamic, Changing Solution Design Not Involvement Environment Aligned to Business Needs Poor Focus on Uncertainly/ Poor Solution Business Needs Ambiguity Design October 18, 2011 38
  39. 39. Taking A Complete View of Systems And Processes IsEssential to Solution Success• Overall solution operates with a mix of automated and manual processes in a structured or ad-hoc manner o deliver the required results• Understanding the overall set of processes and their operation is crucial to successful results• Need to see the entire picture to understand how a solution should operate − Systems/applications are just one part of this universe• Unambiguous definition of processes is required• Processes that are to be automated define the scope of the development and implementation work October 18, 2011 39
  40. 40. Complete View of Systems and Processes Is Essentialto Solution Success External External Manual ManualInteraction Interaction External Automated External Component Process Component Manual Manual Process Process System System Component Component Automated Automated Process System Process Manual Manual Process Component Process External External External Manual Component ManualInteraction Interaction October 18, 2011 40
  41. 41. Solution Design and Implementation Sequence Defines where the business wants to goBusiness Plan Business need identifies solutions that will allow Business Need delivery of plan Defines the benefits to be Business Benefits achieved by the solution Defines the detailed requirements Requirements of the solution Definition Process Design Defines the processes that will be implemented by the solution Solution Architecture and Design Defines the solution design to implement the processes Technical and Detailed Design Creates a detailed technical design for implementation Implementation Implements the detailed design October 18, 2011 41
  42. 42. Consistent Approach to Business And ProcessAnalysis• Adopt a consistent and robust framework in business analysis − Enables effective benefits realisation through a solution which meets the business need• Key elements − Establish enterprise standards, procedures and governance − Standardise on infrastructure, analysis methods and operational procedures − Develop competence and skills• Key benefits − Implement solutions that meet business needs − Increase the ability of the business to adapt quickly to changes − Reduce risk, complexity, redundancy and support complexity − Align business and IT − Enable re-use and faster time-to-market − Present one face to the business (customer) − Increase business value October 18, 2011 42
  43. 43. More Information Alan McSweeney October 18, 2011 43