Soa Runtime Governance Practices


Published on

SOA-based systems are more quickly and efficiently deployed and more effectively managed through a disciplined approach to SOA runtime governance. In this presentation we outline key problems addressed by SOA runtime governance and current practices for effectively implementing runtime governance in SOA environments.

SOA runtime governance problems include:
1. Understanding the service network topology described by the services that participate in the service network and the message traffic flowing among those services.
2. Actively maintaining established availability and performance service levels.
3. Enforcing authentication, authorization, privacy and integrity security constraints in an application-independent fashion.
4. Managing the business transactions supported by the service network including active management of transaction performance and availability as well as detection, diagnosis and correction of business transaction faults.
5. Validating the correct operation of the service networking on a continual basis in order to actively manage dynamic changes to the service network.
The current practices for effective runtime governance are then outlined based on experiences captured in over 100 SOA implementations. The practices focus on governance architectures, capabilities and processes proven to be effective in these SOA implementations.
The presentation also outlines areas in which some established practices are less effective than expected in support of effective runtime governance specifically focusing on processes in which development organizations must proactively participate in the governance activities.
The presentation also outlines the benefits of various approaches in terms of reduced development cost, more responsive system changes, improved operational management and faster and more cost-effective service network evolution.

Published in: Technology, Business

Soa Runtime Governance Practices

  1. 1. SOA Runtime Governance Practices Paul Butterworth Chief Technology Officer AmberPoint, Inc October 2008
  2. 2. Agenda <ul><li>SOA Topologies </li></ul><ul><li>SOA Runtime Governance Practices </li></ul><ul><ul><li>Discover </li></ul></ul><ul><ul><li>Manage Service Quality </li></ul></ul><ul><ul><li>Manage Business Transactions </li></ul></ul><ul><ul><li>Prepare for greater scale </li></ul></ul><ul><ul><li>Validate changes </li></ul></ul>Based on experiences with ~200 users
  3. 3. Typical Service Network Topology <ul><li>Shared Services </li></ul><ul><li>External Services </li></ul>firewall Order Entry Accounting Partner Internal Services Credit <ul><li>Services not applications </li></ul><ul><li>Shared </li></ul><ul><li>Dynamic </li></ul><ul><li>Federated </li></ul>
  4. 4. Typical Service Network Infrastructure Java Service Mainframe Application Web Service DBMS Biz Application Biz Application Appliance In all but the newest of environments, “SOA” ≠ “Just Web Services & XML” Network Service Bus
  5. 5. Keys to Successful Governance and Management of SOA Applications <ul><li>Continuous SOA Discovery </li></ul><ul><li>Service Management & Security </li></ul>
  6. 6. Keys to Successful Governance and Management of SOA Applications <ul><li>Business System Validation </li></ul><ul><li>Closed Loop Governance </li></ul><ul><li>Continuous SOA Discovery </li></ul><ul><li>Service Management & Security </li></ul><ul><li>Business Transaction Management </li></ul> <ul><li>Business </li></ul><ul><li>Architects & Development </li></ul><ul><li>Operations </li></ul>
  7. 7. Agenda <ul><li>SOA Topologies </li></ul><ul><li>SOA Runtime Governance Practices </li></ul><ul><ul><li>Discover </li></ul></ul><ul><ul><li>Manage Service Quality </li></ul></ul><ul><ul><li>Manage Business Transactions </li></ul></ul><ul><ul><li>Prepare for greater scale </li></ul></ul><ul><ul><li>Validate changes </li></ul></ul>Based on experiences with ~200 users
  8. 8. Discovery and Application Mapping <ul><li>Dynamic Discovery of your SOA environment… </li></ul><ul><ul><li>Application Flow & Transactions </li></ul></ul><ul><ul><li>Dependencies </li></ul></ul><ul><ul><li>Services </li></ul></ul><ul><ul><li>Consumers </li></ul></ul><ul><ul><li>Runtime Policies & Metadata </li></ul></ul><ul><li>… across Heterogeneous Infrastructure </li></ul><ul><ul><li>Containers </li></ul></ul><ul><ul><li>ESBs & Process Engines </li></ul></ul><ul><ul><li>Appliances </li></ul></ul><ul><ul><li>Registries / Repositories </li></ul></ul><ul><li>No application, message or header modifications </li></ul><ul><li>Closes the loop with design time governance </li></ul>A complete accounting of your SOA application environment Intended Design Running Reality Repositories Service Registries Home-grown Databases Messaging
  9. 9. Hybrid Discovery Model <ul><li>Approved Services </li></ul><ul><li>Intended Usage </li></ul><ul><li>Policies </li></ul>Runtime Repository Policies Data / Results service contract <ul><li>Services (discovered, changes) </li></ul><ul><li>Scorecard Information </li></ul><ul><li>Policies (new, changes) </li></ul>Discovers Publishes <ul><li>Publishes </li></ul><ul><ul><li>Changes to services, endpoints and policies </li></ul></ul><ul><ul><li>Scorecard metrics – availability, performance, etc. </li></ul></ul><ul><ul><li>Dependencies </li></ul></ul><ul><li>Detects discrepancy between intentions (design/dev) and reality (runtime) </li></ul>Reality Design vs. Service Management Xact Management System Validation Closed Loop Governance Ensures Closed Loop Governance Software Development Tools Development Tools Repositories/ Registries Home-grown Databases Enterprise Service Bus ? ? ?
  10. 10. Detailed Metadata of Your SOA Environment <ul><li>Operational Info: </li></ul><ul><ul><li>When service was discovered </li></ul></ul><ul><ul><li>Availability </li></ul></ul><ul><ul><li>Type of service </li></ul></ul><ul><ul><li>Type of container </li></ul></ul><ul><ul><li>Link to WSDL </li></ul></ul><ul><li>Business Info: </li></ul><ul><ul><li>Business owner </li></ul></ul><ul><ul><li>Division </li></ul></ul><ul><ul><li>Version </li></ul></ul><ul><ul><li>Etc. </li></ul></ul><ul><li>Custom: </li></ul><ul><ul><li>Chargeback info </li></ul></ul><ul><ul><li>Risk assessment </li></ul></ul><ul><ul><li>Links to URL’s </li></ul></ul><ul><ul><li>Etc. </li></ul></ul>Operational Info Business Info
  11. 11. Agenda <ul><li>SOA Topologies </li></ul><ul><li>SOA Runtime Governance Practices </li></ul><ul><ul><li>Discover </li></ul></ul><ul><ul><li>Manage Service Quality </li></ul></ul><ul><ul><li>Manage Business Transactions </li></ul></ul><ul><ul><li>Prepare for greater scale </li></ul></ul><ul><ul><li>Validate changes </li></ul></ul>
  12. 12. Service Quality Management <ul><li>Monitor Performance & Availability </li></ul><ul><ul><li>Trends, thresholds, varying intervals, etc. </li></ul></ul><ul><li>Isolate areas of interest </li></ul><ul><ul><li>Recent additions </li></ul></ul><ul><ul><li>“ Rogue” services </li></ul></ul><ul><ul><li>Problem areas </li></ul></ul><ul><ul><li>Specific application groups </li></ul></ul>Filters Detail Graphical View Table View <ul><li>Monitor Security </li></ul><ul><li>Respond to anomalies </li></ul>
  13. 13. Service Level Management Service- and Business-level Visibility <ul><li>Enforce agreements based on business criteria </li></ul><ul><ul><li>Flexible calendars, multiple objectives </li></ul></ul><ul><li>Granular visibility – groups, users, services, operations </li></ul><ul><li>Preventative and corrective actions </li></ul>Service View Alerts User Summary and Objectives Historical Reporting
  14. 14. Security First- and Last-Mile Enforcement Firewall Identity Management Systems <ul><li>First Mile Security </li></ul><ul><li>Client-side agent </li></ul><ul><li>Automatic enforcement of out-bound security </li></ul><ul><li>Last Mile Security </li></ul><ul><li>Plug-ins provide endpoint protection </li></ul><ul><li>No ability to circumvent </li></ul><ul><li>Extensive Integration </li></ul><ul><li>Identity Management Systems </li></ul><ul><li>Security Appliances </li></ul><ul><li>App Server / ESB / OS Security </li></ul><ul><li><?xml version='1.0'?> </li></ul><ul><li><PaymentInfo xmlns=''> </li></ul><ul><li><Name>John Smith</Name> </li></ul><ul><li><EncryptedData </li></ul><ul><ul><li>Type='' xmlns=''> </li></ul></ul><ul><ul><li><CipherData> </li></ul></ul><ul><ul><ul><li><CipherValue>A23B45C56</CipherValue> </li></ul></ul></ul><ul><ul><li></CipherData> </li></ul></ul><ul><li></EncryptedData> </li></ul><ul><li></PaymentInfo> </li></ul><ul><li>Complete Policy Library </li></ul><ul><li>Authentication </li></ul><ul><li>Authorization </li></ul><ul><li>Credential Mapping </li></ul><ul><li>Censorship </li></ul><ul><li>Crypto </li></ul><ul><li><Name> </li></ul><ul><li><Encrypted </li></ul><ul><ul><li>Type='http </li></ul></ul><ul><ul><li><CipherDa </li></ul></ul><ul><ul><ul><li><Cipher </li></ul></ul></ul><ul><ul><li></Ciphe </li></ul></ul>
  15. 15. Agenda <ul><li>SOA Topologies </li></ul><ul><li>SOA Runtime Governance Practices </li></ul><ul><ul><li>Discover </li></ul></ul><ul><ul><li>Manage Service Quality </li></ul></ul><ul><ul><li>Manage Business Transactions </li></ul></ul><ul><ul><li>Prepare for greater scale </li></ul></ul><ul><ul><li>Validate changes </li></ul></ul>
  16. 16. Business Transaction Management Managing Individual Services is Not Enough <ul><li>Real business value is associated with complete, end-to-end transactions </li></ul><ul><ul><li>Order management </li></ul></ul><ul><ul><li>Claims processing </li></ul></ul><ul><ul><li>Sales lead qualification </li></ul></ul><ul><ul><li>On-line reservations </li></ul></ul><ul><li>Common Issues... </li></ul><ul><ul><li>No overall view into transaction status </li></ul></ul><ul><ul><li>Minimal business visibility </li></ul></ul><ul><ul><li>Slow end-to-end response times </li></ul></ul><ul><ul><li>Transactions &quot;disappear&quot; </li></ul></ul><ul><li>Business Impact </li></ul><ul><ul><li>Internal fire drills and finger pointing </li></ul></ul><ul><ul><li>Unhappy customers </li></ul></ul><ul><ul><li>Lost revenue </li></ul></ul>Process Engine Service Bus End-to-End Technical Challenges <ul><li>Transactions flow through both service and non-service based components </li></ul><ul><ul><li>Services </li></ul></ul><ul><ul><li>Applications </li></ul></ul><ul><ul><li>ESBs </li></ul></ul><ul><ul><li>Process Engines </li></ul></ul><ul><ul><li>Databases </li></ul></ul><ul><li>Variety of architectures </li></ul><ul><ul><li>Synchronous and asynchronous messaging </li></ul></ul><ul><ul><li>Long running transactions – hours, days, ... </li></ul></ul>
  17. 17. Business Transaction Management Monitoring Performance, Availability & Service Level Agreements <ul><li>Enforces agreements in real time </li></ul><ul><li>Enables preventative and corrective actions </li></ul><ul><ul><li>Not just reporting violations after its too late </li></ul></ul>Transaction Performance & Availability Service Level Violations Consumer SLA’s Historical Reporting <ul><li>Business Groups </li></ul><ul><li>Platinum, Gold, etc. </li></ul><ul><li>Accounting, Shipping, etc. </li></ul>Process Engine Service Bus End-to-End
  18. 18. Business Transaction Management Business Instrumentation Consumer SLA’s Business Groups Business Instrumentation <ul><li>Track business value flowing through the system </li></ul><ul><ul><li>Track revenue, total orders, etc. </li></ul></ul><ul><ul><li>Can customize instrumentation and dashboards </li></ul></ul>
  19. 19. Business Transaction Management Real-time Detection of Exceptions <ul><li>Handles Technical and Business Exceptions </li></ul><ul><ul><li>Stalled transactions, missing steps, error messages </li></ul></ul><ul><ul><li>Incorrect data values, boundry conditions, etc. </li></ul></ul><ul><li>User-defined Exception Policies </li></ul><ul><ul><li>What to look for – leverage message content </li></ul></ul><ul><ul><li>Action to take – notify, intervene, etc </li></ul></ul>Rejected Order Alert
  20. 20. Agenda <ul><li>SOA Topologies </li></ul><ul><li>SOA Runtime Governance Practices </li></ul><ul><ul><li>Discover </li></ul></ul><ul><ul><li>Manage Service Quality </li></ul></ul><ul><ul><li>Manage Business Transactions </li></ul></ul><ul><ul><li>Prepare for greater scale </li></ul></ul><ul><ul><li>Validate changes </li></ul></ul>
  21. 21. Runtime Policy Enforcement: Service Virtualization <ul><li>Abstracts service changes and versions behind a published ‘façade’ (a ‘virtual’ service) </li></ul><ul><li>Enables endpoint routing, load-balancing, failover, transformations etc. </li></ul><ul><li>Sees simpler interface </li></ul><ul><li>Service changes don’t show through. </li></ul>Before After Virtual Svc (PEP) <ul><li>Load balance </li></ul><ul><li>Route </li></ul><ul><li>Transform </li></ul><ul><li>Version </li></ul>Service B OrderLookup ChangeDate ChangeQty ScheduleShip ChangePrior LookupETA Service B OrderLookup ChangeDate ChangeQty ScheduleShip ChangePrior LookupETA Service A Service A
  22. 22. Automatic Policy Provisioning <ul><li>Policies with a “where clause” </li></ul><ul><li>Automatically applies policies based on dynamic attributes and message content . </li></ul><ul><ul><li>All production services </li></ul></ul><ul><ul><li>All services in Accounting application </li></ul></ul><ul><ul><li>All services deployed in WebLogic containers </li></ul></ul><ul><li>User-defined attributes for services, containers & policies </li></ul><ul><li>Assignments are reevaluated as attributes change </li></ul>s1 s5 s4 s2 s6 s3 where “ Accounting” Security Encryption all services One-at-a-Time Approach where deployed on .NET app servers Logging Profile Based Approach s1 p1 s2 s3 s100 p1 p1 p50 100 svcs x 50 policies 5,000 policy points Load-Bal Weighted <ul><li>Can manage system on “autopilot” where policies are automatically assigned as appropriate. </li></ul><ul><li>Eliminates production mistakes by reducing manual steps. </li></ul>
  23. 23. Agenda <ul><li>SOA Topologies </li></ul><ul><li>SOA Runtime Governance Practices </li></ul><ul><ul><li>Discover </li></ul></ul><ul><ul><li>Manage Service Quality </li></ul></ul><ul><ul><li>Manage Business Transactions </li></ul></ul><ul><ul><li>Prepare for greater scale </li></ul></ul><ul><ul><li>Validate changes </li></ul></ul>
  24. 24. Business System Validation Distributed Components and Reuse Puts Business Systems at Risk <ul><li>Impact of any changes ripple throughout the system </li></ul><ul><ul><li>Real impact of planned changes is hard to predict </li></ul></ul><ul><ul><li>Impact of unplanned or unannounced changes can be devastating </li></ul></ul><ul><li>Yet, most SOA environments find it impossible to setup and replicate all dependent systems for testing elsewhere </li></ul><ul><li>And, new use and reuse creates blind spots in preproduction procedures </li></ul>“ Approved” Design Development QA Development Staging Production Process Engine Service Bus Need to Validate Integrity of the Entire System Before Installing Changes
  25. 25. Validate Impact on Dependent Systems Development Staging Production Process Engine Service Bus The “Preflight Check” for SOA Systems <ul><li>Acceptance testing of pending changes to SOA environment </li></ul><ul><ul><li>New Versions of Services </li></ul></ul><ul><ul><li>Policy Changes </li></ul></ul><ul><ul><li>Bug Fixes </li></ul></ul><ul><ul><li>Infrastructure Patches, etc. </li></ul></ul><ul><li>Uses knowledge of dependencies and observed interactions </li></ul><ul><li>Simulates services that can’t be replicated in pre-production environments </li></ul><ul><ul><li>External services </li></ul></ul><ul><ul><li>Fee-based services </li></ul></ul><ul><li>Gives Staging and Operations a final check before deploying changes </li></ul> : Security Policies Functioning Unexpected Deviation for B2B Partner Usage  : WS-I Compliant  : Capacity Adequate Validation Checklist
  26. 26. Q&A Paul Butterworth [email_address] 510.663.6300