Yves caseau@md day2011

1,040 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,040
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
43
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Yves caseau@md day2011

  1. 1. Complexity, Modularity, Abstraction:Designing Architecture-Oriented Services Model-Driven Day November 24th, 2011 (v0.0) Yves Caseau Bouygues Telecom – Académie des TechnologiesYves Caseau – Complexity, Modularity, Abstraction – November, 2011 1/31
  2. 2. Outline1. Complexity the 21st century challenge for enterprises and their information system2. Enterprise Architecture Anticipation in a complex world is not forecasting, but building a « situation potential »3. SOA and sustainability repeatable long-term performance requires discipline / governance4. Cloud-Ready Architecture Distributed, scalable, massively parallel computing resource for the whole information systemYves Caseau – Complexity, Modularity, Abstraction – November, 2011 2/31
  3. 3. Companies are Facing a Complex WorldPart I: Complexty A Complex World:  Hyper-competition, globalization, time is shrinking  The power has shifted to the consumers (F. Dupuy)  T. Friedman : « All that is easy has been done, what’s left is the hard stuff » Complicated problems require specialists, Complex problems require everyone  Diversity of skills and viewpoints …  … organized into teams Complex problems are solved “on the gemba”, where they occur, one at a time  Abstractions hide too much, decomposition does not work !  “Reproducible conditions” … do not always exist (isolation is impossible)  Communication is hard (cf. IT when specifying is harder than coding) Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 3/31
  4. 4. An Enterprise is a « Complex System »Part I: Complexty  Numerical complexity (number of things)  Multi-scale  Temporal complexity  Richness of interactions with environment  Symptom examples:  Costs (Information Systems evolution)  Example: non-linear evolution of project costs w.r.t. project size  Rate of errors & failures  Difficulty to « guarantee » robustness and fault-tolerance  Ross Ashby « regulation of a (complex) system requires a control system that is as complex as the system itself »  Time-to-market  The first complexity-related issue  Time-to-integrate-a-new-component depends on the host size : – Human complexity (organization) – Modularity failure (unforseen impacts & interaction between components)  Law of unintended consequences – Feature Interaction Problem Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 4/31
  5. 5. A systemic view of the enterprise & its ISPart I: Complexty Finality Political Aspects Business Model (CEISAR) Evolution Changes Structure PRAXEME Organisation (CEISAR) “pragma” aspect Functional Process Objets Semantic aspect Model Model Logic aspect Operating System Decision System “Pragmatic” aspect operation procedures Information System Geographical Software aspect / localization Precesses IS IS is a complex Hardware system itself Input flows environnement Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 5/31
  6. 6. Consequences of a « systemic vision »  Features / properties « emergence »Part I: Complexty  from « Performance by Design » …  .. to « Performance though simulation & prototypes »  Humility and Continuous improvement Complexité  Complexity Governance  Acknowledge the problem ! Exécution dans le  Attack it with method  Monde Réel Agilité  Cf. CEISAR’s cube Modèle Eléments Partageables ou Réutilisables Eléments Spécifiques (three dimensions) Operations Transformations Synergie  Explicit policies are a key governance tools  SLA, service contacts, data ownership policies, …  “Enterprise Architecture” as a corporate practice  Stakeholders alignment  The “external environment” is critical to IS strategic planning Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 6/31
  7. 7. Measuring ComplexityPart I: Complexty  Measuring the numerical complexity   Sizing the objects (TPMC, FP, Tb)  Complexity (« relationship intricacy ») Metrics  DSM (design structure matrix)  Cyclomatic complexity  (G,  )    ( x). ( y) ( x , y )G  Scalar Euclidian Complexity  The only scale-invariant metric that applies to an architecture map  Bottom line : metrics offer a small amount of help Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 7/31
  8. 8. Taming Information System Complexity (I)Part I: Complexty  Simple approach:  maps and architecture schemas (UML2 helps ).  Common sense approach:  Energetic standardization  Reducing heterogeneity reduces complexity (Printz).  Technological approach:  Infrastructure (middleware)  Introduction of physical de-coupling, sharing and intermediation Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 8/31
  9. 9. Taming Information Systems Complexity (II)Part I: Complexty  Systematic approach (« Urbanisation »):  Enterprise Architecture, seen as a company-wide practice, since it helps to align everyone towards a common target and a transformation path.  Architectural approach (hardest, at the structure level):  modularity (semantic de-coupling).  Strategic Approach (Service-Oriented Architecture):  SOA as a governance practice, reduces complexity: Cf. Parts  share & reuse II & III  Common understanding  Reduces the temporal complexity Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 9/31
  10. 10. Lessons from Bouygues TelecomPart II Back-office re-engineering 2001-20051. Complexity the 21st century challenge for enterprises and their information system2. Enterprise Architecture Anticipation in a complex world is not forecasting, but building a « situation potential »3. SOA and sustainability repeatable long-term performance requires discipline / governance4. Cloud-ready architecture Distributed, scalable, massively parallel computing resource for the whole information systemYves Caseau – Complexity, Modularity, Abstraction – November, 2011 10/31
  11. 11. Enterprise ArchitecturePart II: Architecture IS Re-engineering Architecture: why ?  « Urbanisation »  Communicate a vision  Component-based  Key transformation tool  Process-oriented  Favor reuse (business logic extraction)  Reduce complexity and costs  Temporal de-coupling (asynchronous message)  Support standardization  Functional de-coupling  Align stakeholders (intermediation)  Architects need to avoid  « Enterprise Architecture » complex tools & formalisms  Coherence of three domains  Sensitive to each  Strategy : goals enterprise’s maturity level  operations: processes and data  Asynchronous / diachronous  Information Systems: applications  Acts as a corporate memory and services  Visual change management  Similar topic / two views  EA is more concerned with the target  Re-engineering is a process Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 11/31
  12. 12. Data ArchitecturePart II: Architecture  Data Model  Semantic reference: what does a customer, a bill, … mean ?  Conceptual model: how is a customer defined ?  Ontology: class hierarchy (UML)  Major business asset & key alignment tool  Data Architecture  Distribution  Formats (ex: XML)  Life cycle  Dynamic Management of Business Objects  Distribution / synchronization  Save / restore (disaster recovery)  Data flow management  E.g., network capacity planning Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 12/31
  13. 13. Functional Architecture and Object-Oriented DesignPart II: Architecture  Functional Architecture means to decompose:  Into functions and sub-functions, in a top-down manner  Descriptive normalization: (input, output, transformation, roles, …)  Functional Architecture is no longer self-contained(vs. 20 years ago)  A narrow focus on functional architecture leads to take business process and data distribution issues into account too late.  When functional analysis is carried too deeply, one gets “”silos”  Functional top-down is poorly suited to the efficient use of off-the-shelf software packages  Object-Oriented Design : mixing functional & data model at the IS level  Functional Architecture feeds :  Business roadmaps & « level 0 » decomposition: descriptive analysis of activities/jobs within the company  Service definition within SOA  Contributes to data architecture and business object model Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 13/31
  14. 14. Service ArchitecturePart II: Architecture  Service = Function + Interface + Contract  Service Architecture  Design a structure (clean up the call graph)  Provide meaning (to simplify change management and reuse)  “local” SOA = service-based architecture  Often linked to technology, such as « Web Services »  A new look for an old best practice   The goal is the system (and its architecture), services are a mean  “global” SOA = build a service catalog, whose rich structure is “an architecture”  Independent from technologies (Tuxedo, procedures, …)  Reuse of previous software component reuse theory, applied to large components which are pieces of the IS  The goal is to design a reusable service catalog (the lasting asset), architecture (organization) is a mean (and varies across time) Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 14/31
  15. 15. Process ArchitecturePart II: Architecture  Design a structure on top of temporal interactions:  Events  Chaining & dependencies => business logic  Goals => processes  Describe « fractal/recursive » structure for processes  Processes / sub-processes  Process families  Sharing resources: data, GUI, …  Roles (organizational alignment)  Business process mapping is the best alignment tool between organization and information system  Process description -> services, functions, … (cf. previous slides)  Normalize / Standardize  Share / reuse / outsource  Best approach to integrate packaged software Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 15/31
  16. 16. Designing a Target ArchitecturePart II: Architecture Functions Processes Data Lexicography Métiers Référence Objets (ontology) externe Macro-functions Macro-processes (draft) Object Lifecycle External Level 0 Reference Macro-processes External Reference Exchanges - Content Events Services Catalog Processes Update protocol Service Archi. v1 Process Archi v1 Data Archi. v1 Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 16/31
  17. 17. Designing a Modular ArchitecturePart II: Architecture  Goal: minimize impact dispersion for new services  “Definition”: modularity is the correlation  « Distance in the code » and frequency of interaction  « Distance in the code » and « co-evolution »  Good practices :  Layered architecture (define abstraction levels)  Process Architecture (define a composition grammar)  Sharing/reuse & modularity go hand-in-hand : sub-process identification  Event-Oriented Architecture  Pub/sub is still a one of the best modular patterns  Model-Driven Architecture: careful design of « future-proof » data model  Service Architecture reduces unmanaged interactions Cf. Part III  Reification of functional architecture  Abstraction/ encapsulation Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 17/31
  18. 18. Part III Image blog1. Complexity the 21st century challenge for enterprises and their information system2. Enterprise Architecture Anticipation in a complex world is not forecasting, but building a « situation potential »3. SOA and sustainability repeatable long-term performance requires discipline / governance4. Cloud-ready architecture Distributed, scalable, massively parallel computing resource for the whole information systemYves Caseau – Complexity, Modularity, Abstraction – November, 2011 18/31
  19. 19. Strategic Issues : Tomorrow’s Information SystemPart III: Sustainability & SOA Complexity does not mean that some challenges are not clearly visible  Information Systems 2.0 – our customers …  want their voice heard: they need a feedback mechanism  collaborate with other customers or prospects  expect Information Systems to be always on, any time on any device   Information Systems need to be personalized & predictive  Social networks analytics, Bayesian learning, …  Semantic processing : making sense from customer & partner input  Time acceleration generates more data - need to master « big data » analytical skills  Customers and end-users demand to be more autonomous  End-users have control over their own processing , write their own « apps »  IS must follow the customer wherever she goes, not the opposite !  Customer are « the architect of their own experience » → IS must become interoperable platforms Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 19/31
  20. 20. Sustainable Information Systems »Part III: Sustainability & SOA  « develop today’s information services without preventing the ability to develop those for tomorrow, through the overuse of resources, or the production of un-manageable complexity».  Freely inspired from « sustainable » as defined by Brundtland Commission   (global) SOA is the only method for sustainable IS development  Not the only re-engineering method  (other approaches exist which reduce IS complexity),  but the best method to do so continuously, as a company-wide practice (with all stakeholders), a long-term progressive & measured effort, which generates its own reward  Clean-up : learn to remove/trip (classic from asset management)  Cf. Extreme programming (Agile Manifesto) :  Leveling the effort, continuous integration, favor simple designs  Continuous simplification, not a last-resort heroic attempt Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 20/31
  21. 21. SOA & Governance :Part III: Sustainability & SOA Three steps:  Service Définition: build the business service catalog. Starts like a functional analysis (from business processes) – but reusability and composability is engineered during this step, trough the dialog between business and IS.  Architecture de Service: Transform a list into a hierarchical and modular structure. Classical difficulties and solutions (ex: refactoring) … to avoid a « Web Services spaghetti».  Service Intégration : the « technical » step – often confused (SOA SOI). Integration/middleware technology is quite mature nowadays  “Think globally, act locally” SOA starts with the periphery of the system and ends with the core. Data architecture is critical from day one.  Beware of the “proof of concept” which is much harder to integrate than to build   « Something you do, not something you buy » - David Linthicum  SOA Governance  its first goal is to establish the existence of shared artifacts (architecture schemas, service catalogs, roadmaps) et everyone’s role (rights and duties) Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 21/31
  22. 22. Sustainability’s EquationPart III: Sustainability & SOA  For a company with expected growth g, the IT budget is defined as the sum of the project budget P and the Operation budget O.  An application portfolio of size S is built, with life expectancy A  a given share (d) of the application portfolio is removed/destroyed  We suppose that the operation cost for an application that costs 1k€ is w k€  We also suppose that there is a productivity gain over the years of p%.  The usual two ratios that are of interest :  r = (P / O) = ration of build / run - usual measure R = (P / P + O)  n = Pn / P = % of the project budget spent on creating new applications  Sustainability Theorem : stable IT budget relative to company growth  n = [A (g + d) ] / (1 + (g+d) A) & r = [g + d + p +(1-d)/A] / [w (1 - d + A(g+d))]  Corollary  Easy to plug into an Excel spreadsheet  Example: w = 25%, d = 5%, p = 4%, A = 10, we get R = 42% and n = 33%. (no surprise )  How to improve r ?  Clean-up old applications  Increase productivity for operations Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 22/31
  23. 23. SOA as a discipline : Architecture-oriented ServicesPart III: Sustainability & SOA  How get reusability, across the enterprise (sharing) and across time (reuse) ?  Abstraction  Trade-off between too generic and too specific  Modularity  Relies on process and event architecture  Composability  Horizontal (Process) : Common (Pivotal) Object Model  Vertical (Functional) : Parametric Polymorphism   API Model Discipline  Manage versions !  API (meta) data model : worth some effort !  Each CIO becomes a software editor  This is more an art than a science  Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 23/31
  24. 24. MDA: where Abstraction and Modularity startPart III: Sustainability & SOA  Data model is the key factor for modularity & flexibility  A few rules drawn from experience:  Reify links  Reify roles  Hierarchy products vs. complex ontologies  Group/individual substitution  Work on the meta-model  Think « object »   Ontology before description, encapsulation (hiding vs. transparency)  The data model defines the IS « situation potential »  Difficulty of strategic planning in a complex world   scenario practices: what if …  … we were … one of our competitor ?  … we outsourced this activity ?  … we provided this service to other companies (SaaS) Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 24/31
  25. 25. Part VI1. Complexity the 21st century challenge for enterprises and their information system2. Enterprise Architecture Anticipation in a complex world is not forecasting, but building a « situation potential »3. SOA and sustainability repeatable long-term performance requires discipline / gouvernance4. Cloud-ready architecture Distributed, scalable, massively parallel computing resource for the whole information systemYves Caseau – Complexity, Modularity, Abstraction – November, 2011 25/31
  26. 26. Cloud Computing : Scalable Resources & ServicesPart IV: Cloud-ready Architecture  Two angles :  Cloud as a computing resource : a CIO issue  Cloud as a service platform : joint ownership  A « new » kind of resource  The heir of grid and « commodity servers »  Fast provisioning  A new kind of business model  CAPEX to OPEX (rent)  Full scalable (up and down) Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 26/31
  27. 27. The Innevibilaty of Cloud Computing Part IV: Cloud-ready Architecture  Better TCO  Commoditization  economy of scale (operations)  Utilization rate (from sharing)  PUE (Power Usage Effectiveness)  Parallel computing performance  Example from the movie industry  MapReduce & Hadoop   Distributed computing reliability  Massive redundancy …  State-of-the-art availability (99.9% to 99.99%)  Flexibility  Scalability  Pay-as-you-grow Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 27/31
  28. 28. Don’t Confuse Clouds with Smoke Part IV: Cloud-ready Architecture  Data privacy & security  A major concern for most citizens  Not all data centers are equal in front of hackers  Data architecture must distinguished hot/warm/cold  Brewer’s Theorem (Distributed computing is still hard )  Consistency (all views obtained through different process represent the same information)  High-availability (relies on massive replication)  Fault-tolerance (disaster recovery)  Cloud, Caching and Peer-to-Peer Architecture  « Cloud projection over the edges »  Digital homes future architecture Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 28/31
  29. 29. SOA is not scale-free / speed of light is still a constraint Part IV: Cloud-ready Architecture  Latency is still a constraint  Tens of ms to access the cloud  Performance is not guaranteed (or price is no longer cheap)  Service « scale » (size) is crucial  SOA is not scale-free  A classic rule of « performance management » for SOA  Even more important with a cloud-architecture  RCA: Rich Client Architecture  Used for SaaS and MMORPG … for a reason   Applies to heavy transactional contexts Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 29/31
  30. 30. Cloud-ready ArchitecturePart IV: Cloud-ready Architecture  A cloud-ready architecture supports progressive migration of both front-office & back-office services to the cloud  SaaS for front-office is easier : way to start …  Portal integration, then mash-up  Variation of a service-oriented architecture  événements Applications  Requires to: interactives Processus Batchs  Leverage Web technology (ex: portail)  “think platform” Infrastructure (ex: ESB synchrone) Annuaire cf. “What would Google Do” UDDI service service Propriété clés:  Build catalogs of • Stateless •Gestion Web APIs Mash-ups explicite du contexte Infrastructure (ex: ESB asynchrone) •Contrat de service Applications Ressources « Cloud » Référentiels (Internet) (données) Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 30/31
  31. 31. Conclusion Models are key for:  agility  situation potential (seizing opportunities)  cost management Innovation in the digital world happens in the source code  agile development (end user / designer / developer)  Fail often to succeed early (Eric Riess)  Lean IT: no delays, quality in, fast delivery, less code, short intervals (small batches), customer participation, VSM & « muda » removal You should prepare to the advent of massively parallel computing  Cloud or multi-core  Even for traditional back-office functions  From VNG to DCG Yves Caseau – Complexity, Modularity, Abstraction – November, 2011 31/31

×