Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Governance in Ultra-Large-Scale Systems

584 views

Published on

What kind of Governance is appropriate to the distributed collaborations across ULS systems?
How are the changing services provided by the operational components of ULS systems to be enabled to achieve levels of cooperation and collaboration that can satisfy requirements for fast system evolution?
How are we to enable the distributed collaborations across ULS systems to be edge-driven?
How are the processes of orchestration and synchronisation to work in ULS systems?
How are people to be taken into consideration as first-class components of ULS systems?

Published in: Business
  • Be the first to comment

Governance in Ultra-Large-Scale Systems

  1. 1. Governance in Ultra-Large-Scale (ULS) Systems Philip Boxer BSc MBA PhD 1Copyright © Boxer Research Ltd 2015
  2. 2. Contents • The Policy, Acquisition and Management Research Agenda – What kinds of challenges are these? • On Orchestration and Synchronisation – What insight can be derived from this experience? • Practical Consequences – What is it possible to do for clients? • In Conclusion – This leaves a lot of unresolved issues… 2Copyright © Boxer Research Ltd 2015
  3. 3. THE POLICY, ACQUISITION AND MANAGEMENT RESEARCH AREA What kinds of Challenges are these? 3Copyright © Boxer Research Ltd 2015
  4. 4. The Governance framework is crucial – Given the scope and scale of ULS systems, technical, organizational, and operational policies will emerge as principal vehicles for ensuring harmonious operations at all levels. – The size and highly distributed nature of ULS systems will limit global visibility and decentralize system management within an overall framework of organizational, technical, and operational policies. – Research is required in how to define ULS system policies that specify organizational, technical, and operational constraints for global system integrity and freedoms for flexible adaptation. – Much of the evolution of ULS systems will occur in situ, thereby imposing requirements to maintain critical operational capabilities while adding or improving other capabilities in place − What kind of Governance is appropriate to the distributed collaborations across ULS systems (sometimes described as having virtual* characteristics)? * For example, “Virtual systems of systems lack a central management authority and a centrally agreed upon purpose. Large- scale behavior emerges and may be desirable, but this type of SoS must rely upon relatively invisible mechanisms to maintain it.” Systems Engineering for Systems of Systems, August 2008, Office of the Under Secretary of Defense (Acquisition, Technology and Logistics) 4Copyright © Boxer Research Ltd 2015
  5. 5. The supporting systems and infrastructures have to be understood differently – Research is needed to understand how supply-chain organizations could be integrated as first-class operational components into ULS ecosystems… – In such an environment, supply-chain organizations ranging from established vendors to open-source collaborations could undergo periodic assessment of capabilities, participate in joint training and readiness exercises with the forces, and come and go as needs and capabilities change. – The supply chains of vendors and integrators that will populate ULS ecosystems must be organized, and incentives must be provided to evolve ULS system capabilities at a rapid pace in response to changing operational needs. * For example in the need for Through-Life Capability Management (TLCM): “a greater proportion of our overall business is available to industry than in any other major defence nation, and growing expertise in the combination of systems engineering skills, agility and supply chain management required to deliver TLCM gives the UK defence industry a comparative advantage.” UK Defence Industrial Strategy December 2005 p6. − How are the changing services provided by the operational components of ULS systems to be enabled to achieve levels of cooperation and collaboration that can satisfy requirements for fast system evolution*? 5Copyright © Boxer Research Ltd 2015
  6. 6. Demand issues are of a different kind – The pervasive application of ULS systems to support global operations in many simultaneous strategic and tactical situations will generate many requirements for rapid evolution to meet changing threats and environments. – ULS systems will experience and should create incentives for substantial local adaptation and bottom-up evolution – ULS systems will be designed to support dynamic coalitions and management of tactical and strategic operations through linkage of field units with command-and-control functions on any scale necessary. – The overarching requirement for ULS systems is operational readiness at all times under all conditions… ULS systems will be required to adapt to changing missions and unanticipated circumstances encountered by warfighters. * An ‘edge’ is any particular situation/context where a threat-demand is encountered that the ULS system must respond to. An early formulation of this was in Power to the Edge: command and control in the information age, Alberts & Hayes, DoD CCRP June 2004. − How are we to enable the distributed collaborations across ULS systems to be edge- driven*? 6Copyright © Boxer Research Ltd 2015
  7. 7. Collaborative processes are critical – Policies must support both local and global operations in such a way that people and the computational actions they initiate can achieve cooperative and even competing objectives without impairing the viability of the system as a whole. – It may often be the case that, to meet immediate needs, local users of ULS systems will be forced to engage in ad hoc acquisition of components whose functionality and quality properties are not well understood or trusted. Because these components address unforeseen problems, an opportunity will arise to improve and generalize their application across similar environments. – ULS systems engineering development and operational use will generate knowledge that can be preserved and analyzed to guide future evolution. * “Orchestration is the set of activities needed to make the elements of a ULS system work in reasonable harmony to ensure continuous satisfaction of the mission objectives”. p25 ULS Systems − How are the processes of orchestration and synchronisation to work in ULS systems?* 7Copyright © Boxer Research Ltd 2015
  8. 8. Taking the human factor into consideration – People are key participants in ULS systems. Many problems in complex systems today stem from failures at the individual and organizational level. We therefore need research on user-centered specifications and on modeling users and user communities. – While some models of human interaction are inspired primarily by economic factors and competitive forces to drive improvements, research is needed to understand other models, such as open source, that involve fostering non-competitive social collaboration. – We cannot fully anticipate the context within which ULS systems will operate and necessarily evolve, as the socio-cultural practices of many different groups (stakeholders, users) will, in fact, be constructing this real-world computational environment. The challenge is thus to design and support systems using an accurate model (scientific understanding) of this ULS/social-context interaction. − How are people to be taken into consideration as first-class components of ULS systems?* Collaborative processes are critical Demand issues are of a different kind The supporting systems and infrastructures have to be understood differently The Governance framework is crucial N-S are limiting E-W are enabling 8 * Boxer, P. J. (2014). "Leading Organisations Without Boundaries: 'Quantum' Organisation and the Work of Making Meaning." Organizational and Social Dynamics 14(1): 130-153. Copyright © Boxer Research Ltd 2015
  9. 9. Client Situations generating learning about these challenges Client Instances The problem motivating their interest • Thales/Bosch • Unable to take advantage of and manage the opportunities emerging among multiple related customers with differing interests. • UK NHS/BT • Unaddressed client needs because of limitations to existing business models • Create/JFSP • Difficulty in managing complexity of modeling frameworks within acceptable timeframes • NATO/MilSatcom • Systematic identification and structuring of risks to deployment and sustainment of operational capabilities • MOD NiteWorks/Army Software Blocking • Unaffordable cost trends plus lack of ability to support rates of operational change, and failure to meet time deadlines supporting deployment • Raytheon/MoD • The customer changing from an equipment-based to a capability-based approach to acquisition and its effects on how the supplier’s market is defined 9Copyright © Boxer Research Ltd 2015
  10. 10. ON ORCHESTRATION AND SYNCHRONISATION What insights can be derived from this experience? 10Copyright © Boxer Research Ltd 2015
  11. 11. The Supplier is supporting the Client Enterprise managing three diverging tempos… Customers of the Client Enterprise Supplier Socio-technical Client Enterprise Socio-technical Ecosystem Demand Tempo The rate at which new forms of demand need to be satisfied Supplier 1 Supplier 2 sub-contract sub-contract Acquisition Tempo The rate at which new requirements can be met Client Enterprise users users Readiness Tempo orchestration synchronization The rate at which the client enterprise is able to support new forms of value proposition Demanded effects in the customer’s context-of-use The client enterprise aligns to the demand of the customer The supplier responds to the client enterprise aligning to the demands of its customers Projective analysis 11Copyright © Boxer Research Ltd 2015
  12. 12. Opportunistic (marginal/ incremental costs) Niche-based (focus where Positional advantage can be sustained) Client’s Economies of Alignment No Yes Supplier’s Economies of Scale or Scope Yes No Effects-based (focus where Relational advantage can be sustained) The challenge for the business is to be able to include effects-based forms of competition Digitalisation The processes of digitalisation change the economics of alignment Over time, innovations in response to demand are imitated and become generally available from suppliers The Supplier has to support the Client’s effects-basing, adding the need to generate Economies of Alignment for the Client 12Copyright © Boxer Research Ltd 2015
  13. 13. Defence Enterprise Acquisition Tempo Readiness Tempo Adapted from: “Appropriate Collaboration and Appropriate Competition in C4ISTAR Transformation”, Dr Nicholas Whittall RUSI 2007 Campaign Tempo Gap = Need Acquisition Effect Threat-Demands Composite Capability Capability Capability Capability Orchestration Doctrine Organization Training Materiel Leadership Personnel Facilities Suppliers Capability Requirement Traditional Defense Companies await Requirements expressed in Programmes. Competitive advantage to be gained in aligning the Need to the Demand. Competitive advantage to be gained from dynamic alignment of Composite Capabilities to the Threat-Demands. Divergence of tempos increases costs of alignment Divergence of tempos increases Costs of Alignment associated with Readiness The divergence of tempos creates an agility challenge 13Copyright © Boxer Research Ltd 2015
  14. 14. With effects-basing, Value comes from managing a Double ‘V’ Requirement Solution Components Design decomposition Design integration The cycle creating Value Orchestrations Military Effects Composite Capabilities Joint Command Gaps in Force Command Structure and Composite Capabilities Scenarios and Campaign Plans Capability = Solution plus DOTMLPF Requirement = Capability gap minus DOTMLPF demand-side supply-side Boxer, P.J. (2007) Managing the SoS Value Cycle, January 2007, http://www.asymmetricdesign.com/archives/85 Synchronisations 14Copyright © Boxer Research Ltd 2015
  15. 15. The unanticipated need for agility can be very expensive Level of short-term savings – the cost of the option to satisfy a significant part of the UAV III need The cumulative costs of sourcing UAVs I & IIUAV I – over-the- horizon targeting (Phoenix) UAV II – CCIR ISTAR asset (Watchkeeper) Level of additional expenditure incurred through Urgent Operational Requirements to deliver UAV III UAV III – extending ‘edge’ capability (+Nimrod, Lydian & Desert Hawk) How should this option have been valued? 15 Without an ability to analyze cohesion it is difficult to establish the costs of alignment and identifying options for increasing agility Copyright © Boxer Research Ltd 2015
  16. 16. a’b’ 2. Change in variance in levels of expenditure, based on the difference between the two curves ‘a’ and ‘b’ Probability Levels of expenditure meeting Customer Demands The cost of Force Structure ‘b’ across the variety of demands b ‘Real Option’ pricing allows a value to be assigned to these changes in variance a The cost of Force Structure ‘a’ across the variety of demands 1. Reduction in average level of expenditure through impact of capability trade. The value of an incremental investment in some new capability is the impact of both the trade and the change in agility* * Agility = property of the Force Structure enabling it do more things with the same underlying capability set. The Value of Agility for the Client Enterprise is to reduce the variance in expenditures Two kinds of benefit: 16Copyright © Boxer Research Ltd 2015
  17. 17. Value comes through creating Real Options that increase the agility of force structures 2: Real Option Valuations (agreed perceptions of likelihood of future scenario mix => distribution of demand across options for composite capabilities) 4: Engineering for Flexibility (maximise mitigation of interoperability risks) 1: Varieties of Geometries of use x Decisive Points (across all possible scenarios. Assumes technical feasibility) What is valued politically => where to develop agility Determines the underlying context in which new demands have to be met Determines forms of cohesion demanded Requirement for flexibility in capabilities 3: Type III Agility (defining new capabilities for distributed collaboration and requisite granularity of supporting capabilities) 17Copyright © Boxer Research Ltd 2015
  18. 18. PRACTICAL CONSEQUENCES What is it possible to do for Clients? 18Copyright © Boxer Research Ltd 2015
  19. 19. Ultra-Large-Scale Socio- Technical Ecosystems Monolithic Systems Analysis of Requisite Variety – establishing the variety of geometries needed In practice we are always operating between the ultra-large-scale and the monolithic Invariances resulting in Quality Attributes Emergent (orchestrations) Static (architectures) Effects anticipated on Demand Situation Unanticipatable (Effects Ladders) Planned (Mission Threads) The client enterprise is operating somewhere between ULS socio-technical ecosystems in one direction, and monolithic systems in the other 19 QAW/ATAM – establishing the quality attributes of the architecture Analysis of stratification and granularity, identifying requirements of lower strata SoS Mission Threads Analysis Copyright © Boxer Research Ltd 2015
  20. 20. Analysis of socio-technical systems needs to include relationships with three new (types of) view… Constrains what is possible Shapes granularity and stratification Example Analyses Functional Architecture Description Data Architecture Description Accountability Hierarchies Description Social Synchronization/ Data Fusion Description Description of Heterogeneity of Demand Organization Stand-alone Systems  () - - - Stand-alone Software ()  - - - Complex SoS/EA  () () - - Case Examples JFSP II: Framework   ()  - NATO AWACS: SoS     () Thales: C4ISTAR      Projective Analysis Views Structure- function Trace Hierarchy Synchronization Demand Example Analyses Functional Architecture Description Data Architecture Description Stand-alone Systems  () Stand-alone Software ()  Complex SoS/EA  () Case Examples JFSP II: Framework   NATO AWACS: SoS   Thales: C4ISTAR   Projective Analysis Views Structure- function Trace 20Copyright © Boxer Research Ltd 2015
  21. 21. Stratification and granularity enabling the supply-side to be aligned to the demand-side 4: Organisational Interoperability (shared understanding of organizational processes) 3: Semantic Interoperability (shared understanding of behavioral meaning) 2: Syntactic Interoperability (communication is possible) 1: Machine Level Interoperability (stuff works) 5: Situational Interoperability (the way a situation is engaged with) 6: Effects Environment (the contexts-of-use in which effects are being created) The layers read into a theatre-of-operations context decisive points mission command (agile) force structure force elements fielded capability equipment capability Engineering constraints ‘supply-side’ pragmatic constraints ‘demand-side’ The size of this overlap depends on the engineering constraints being under- determining 21Copyright © Boxer Research Ltd 2015
  22. 22. The Governance of Alignment can be approached across eight dimensions of agility Driven from ‘center’ Driven from ‘the edge’ Doctrine & operational concepts The principles and operational methods underlying the approach to generating effects Facilities & infrastructure The facilities and infrastructure that are the context within which the enterprise does its work Leadership & education The ability to lead the enterprise creatively and effectively within the context of the its chosen domain of action Materiel & technology The tools and technologies that the enterprise needs to be effective within its chosen domain of action Edge organisation The particular orchestration and synchronisation of capabilities needed to meet a demand Force composition & collective learning The people needed with the appropriate know-how and ability to work together collaboratively Situational understanding The way data is able to be fused to provide a composite picture of what is going on in the particualr situation Personnel & culture The people with the socialisation, background and mutual knowledge and trust to be able to work together 22Copyright © Boxer Research Ltd 2015
  23. 23. Four diagnostics shape how the need of the Client Enterprise can be engaged with Alignment of Organization & Infrastructure to Demand 2 How does the client enterprise align itself to its customers’ demands? White: how we must do what we do Blue: what we do Internal External Internal External Red: particular demands Black: the contexts from which the demands emerge The way things work What determines shape The way things work What determines shape Establishing the Key Stakeholders 1 Who are the key stakeholders in the performance of the client enterprise? Influence Maps and beyond Internal Alignment of DOTMLPFS 4 How are the internal processes of the client enterprise aligned? Alignment of Governance Processes Alignment of Service Layers to Demand 3 What strategy and economics are driving how the client enterprise relates to its customers’ demands? Alignment of Economics SoS Mission Threads/ Quality Attributes and interoperability risks pragmatic constraints Engineering constraints 1: Machine Level Interoperability (lexis) 2: SyntacticInteroperability (command syntax) 3: Interoperabilityof Component Behaviors (shared understanding of behavioral semantics) 4: OrganizationalInteroperability (shared understanding through organizational semantics) 5: SituationalInteroperability (pragmatics of theway a situation is engaged with) 6: Effects Environment (thecontext-of-use in which effects are created) Alignment of Architectures 23Copyright © Boxer Research Ltd 2015
  24. 24. This leads to a different kind of analysis of interoperability risks… Source: Anderson, Boxer & Browsword (2006) An Examination of a Structural Modeling Risk Probe Technique, Special Report, Software Engineering Institute, Carnegie Mellon University, CMU/SEI-2006-SR-017, October 2006. http://www.sei.cmu.edu/publications/documents/06.reports/06sr017.html Special permission to use PAN in this Technical Probe was granted by Boxer Research Limited. Identifying Interoperability Gaps in the different strata Analysis of Granularity Socio-technical SoS in relation to Demand Functional/ Data Coupling Demand cohesion Accountability Hierarchies Distinguishing three different kinds of pattern 1 services know-how 7 drivers 7b problem domains 6 demand situations mission situations 5b 5 composition of orchestrated constituent capabilities orchestrationsof constituent capabilities 4b constituent capabilities 4 2b 3b 2outcomes 3 1c super- structure 1b direct organisation 0processes events 6bdata fusion platforms Analyzing alignment of strata to demand 24Copyright © Boxer Research Ltd 2015
  25. 25. Modeling socio-technical systems Analyzing alignment of strata to demand 5-6col1x5 unitorderborder_isr_cell unitorderborder_reaper_strike_cell unitorderborder_sf_cell unitorderafghan_border_strike unitorderborder_caoc_atc_sync unitorderborder_hale_bm unitorderborder_male_bm traceventborder_male_outputs traceventborder_hale_on_station traceventborder_male_on_station traceventborder_male_strike traceventafghan_report traceventborder_sf_on_station traceventindividual_in_afghan-pakistan_border channelborder_hale_bm channelborder_male_bm channelborder_isr_cell channelborder_reaper_strike_cell channelborder_sf_cell c_sitnindividual_in_afghan-pakistan_border orchnafghan_border_strike 1 1 1 1 1 1 1 1 1 1 1 outcomeborder_hale_on_station 1 1 1 1 1 1 outcomeborder_male_on_station 1 1 1 1 1 1 outcomeborder_male_strike 1 1 1 1 1 1 outcomeborder_sf_on_station 1 1 1 1 khowborder_sf 1 1 1 1 1 1 khowborder_male_strike 1 1 1 1 1 1 1 khowborder_hale_global_hawk 1 1 1 designborder_hale_global_hawk 1 1 designborder_male_operator 1 1 1 1 1 1 1 capyborder_hale_global_hawk 1 1 capyborder_male_reaper 1 1 1 1 1 capyborder_sf 1 1 1 1 1 systemborder_hale_global_hawk 1 1 systemborder_male_reaper 1 1 1 1 1 1 systemborder_sf 1 1 processborder_hale_global_hawk 1 1 processborder_male_reaper 1 1 1 1 1 processborder_sf 1 1 1 1 1 dprocessborder_hale_global_hawk 1 dprocessborder_male_reaper 1 1 1 1 Cohesion-based Costing Defence Expenditure Scenario 1 Alternative Large Scale Small Scale enduring Scenario 2 Scenario 3 Medium Scale enduring Small Scale enduring Small Scale one-off Medium Scale enduring Small Scale limited Small Scale one-off Monte Carlo Simulation of impact of Variations in Demand 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0 75 150 225 300 375 450 525 600 675 750 825 900 975 1050 1125 1200 1275 1350 1425 Scaled Cost1 Scaled Cost2 Scaled Difference The resultant Cost Distributions Value for Defence: Value of reduced Costs of Alignment … and to a different kind of analysis of value * See Boxer, P.J. (2008) What Price Agility? Managing Through-Life Purchaser-Provider Relationships on the Basis of the Ability to Price Agility, Navigator White Paper, Software Engineering Institute, Carnegie Mellon University, September 2008 25Copyright © Boxer Research Ltd 2015
  26. 26. Case Examples Case Root Customer Problem Outputs of Analysis Duration Raytheon Need to create and position new value propositions Redefined the relationship to the market, making it possible to capture new kinds of opportunity 2-3 day workshop for each market Thales Unable to cost alternative forms of alignment and establish the value of introducing new system flexibilities. Established the baseline cost of current levels of agility to support the value propositions of alternative forms of flexibility Three 1-day workshops + analysis of cost data over 2-3 weeks NHS Collaboration platform needed to support roles and accountability structure. Provided a framework for through-life accountability to the customer, changing the basis on which customers can acquire services 2-3 days initial workshop, 3-4 weeks analysis + 1-day feedback workshop JFSP II/ CREATE Horizontal and vertical scope of data and scientific model collaboration platform. Providing dynamic support to collaboration amongst users through analysis of data fusion across chains of scientific models. Two 1-day workshops + analysis over 2-3 weeks JFSP I Gaps between profiles of strategic intent, organizational plans and their execution. Established the ability to determine the lack of alignment between systems and the uses they are ultimately supporting, facilitating investment targeting ‘n’ interviews + interpretation over 2-3 weeks + 1-day feedback workshop. NATO Gaps in the different layers of alignment between the underlying systems and their ultimate contexts of use. Identified the risks to sustaining particular organizations of interoperation, creating an explicit focus on mitigating SoS risks Three or four 2-day workshops over 2-3 weeks MoD Existing methods ‘blind’ to the required variety of configurations of interoperating capabilities Extended the definition of capability gaps to include organizations of interoperation, making it possible to direct resources to SoS capabilities Three 2-day workshops over 2-3 weeks 26Copyright © Boxer Research Ltd 2015
  27. 27. IN CONCLUSION This leaves a lot of unresolved issues… 27Copyright © Boxer Research Ltd 2015
  28. 28. Affordable Capability Development Plan Balance of Investment Resource Constraints Defence Priorities Force Development Options Identify Capability Mismatches Capability Assessment Current and Planned Capability Capability Goals Scenarios Defence Priorities Capability Partitions Government Guidance Future Environment (Threat, Tech’y etc) optional Operational Concepts 2. Needs addition of capability goals for orchestration and synchronisation 1. Needs partitioning superstructure made independent of stratification of interoperability 3. Needs stratified analysis of requisite interoperabilities across DOTMLPFs, changing basis of assessment, analysis of gaps and pricing of options Source: Capability-Based Planning – Developing the Art, 2007 Critique of existing Capability Engineering 28Copyright © Boxer Research Ltd 2015
  29. 29. Bridging between the medium and the longer term • The Challenges – What kind of Governance is appropriate to the distributed collaborations across ULS systems? – How are the changing services provided by the operational components of ULS systems to be enabled to achieve levels of cooperation and collaboration that can satisfy requirements for fast system evolution? – How are we to enable the distributed collaborations across ULS systems to be edge-driven? – How are the processes of orchestration and synchronisation to work in ULS systems? – How are people to be taken into consideration as first-class components of ULS systems? • Unresolved Issues – How are the questions of authorization and ‘in whose interests’ to be approached? – If stratification is driven as much by the contexts of use as it is by what is computationally feasible, how are these two ‘axes’ of engineering to be held in relation to each other in ULS systems? – How is the nature of demand on ULS systems to be understood? – What is the place of accountability, transparency and risk in these processes? – What assumptions are being made about the relationships between person, enterprise and ULS system? 29Copyright © Boxer Research Ltd 2015
  30. 30. Balance of Investment Capability Assessment Capability Goals Scenarios Capability Partitions Future Environment (Threat, Tech’y etc) optional Operational Concepts Force Development Options Stratified Analysis Interoperabilities across all DOTMLPFs Analysis of Alignment Costs Pricing of options across all DOTMLPFs Force Geometries Orchestration & Synchronisation Risk Analysis + Identify Capability Mismatches Agility types I, II & III interoperability risks Requisite variety of geometries Force Geometries ‘envelope’ Capability Engineering for Agility 30Copyright © Boxer Research Ltd 2015
  31. 31. END 31Copyright © Boxer Research Ltd 2015
  32. 32. UAV II – CCIR ISTAR asset (Watchkeeper) UAV I – over-the-horizon targeting (Phoenix) Without an ability to analyze cohesion it is difficult to establish the costs of alignment… The asset moved from being organic to the MLRS capability to providing a capability to Divisional Command 32Copyright © Boxer Research Ltd 2015
  33. 33. … and identifying options for increasing agility UAV III – extending ‘edge’ capability (+ Nimrod, Lydian & Desert Hawk) Afghanistan needs a much greater layering (and fusion) of feeds and their much greater availability at the ‘edge’ in support of a greater campaign tempo with a commensurately greater readiness tempo 33 The unanticipated need for agility can be very expensive Copyright © Boxer Research Ltd 2015

×