The Story of How an Oracle Classic Stronghold successfully embraced SOA


Published on

The Story of How an Oracle Classic Stronghold successfully embraced SOA

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Story of How an Oracle Classic Stronghold successfully embraced SOA

  1. 1. The Story of How an Oracle Classic Stronghold Successfully Embraced SOA AMIS ODTUG Preview 15 th June 2009 Lucas Jellema SOA
  2. 2. Overview <ul><li>What is an Oracle stronghold? </li></ul><ul><li>Triggers to start moving towards Services </li></ul><ul><li>Levels of embracing Services and SOA </li></ul><ul><ul><li>Objectives, benefits, costs & challenges </li></ul></ul><ul><ul><li>Demonstration </li></ul></ul><ul><li>Pitfalls, Lessons Learned & Best Practices </li></ul><ul><li>Summary </li></ul>
  3. 3. What is an Oracle stronghold? <ul><li>The typical Oracle stronghold </li></ul><ul><ul><li>Using Oracle RDBMS & Oracle Development tools </li></ul></ul><ul><ul><ul><li>Lot of SQL and PL/SQL </li></ul></ul></ul><ul><ul><ul><li>Probably Oracle Forms and maybe APEX as well </li></ul></ul></ul><ul><ul><ul><li>Possibly Oracle Designer, tools for BI & Reporting </li></ul></ul></ul><ul><ul><li>Several databases with many years of essential corporate data </li></ul></ul><ul><ul><li>IT staff has Oracle veterans – 5-15 years or more </li></ul></ul><ul><ul><li>Internet development may have taken place largely separate from the Oracle technology stack </li></ul></ul>
  4. 4. Then wat happened… <ul><li>Job Agency offered Oracle Forms application for entering timesheets </li></ul><ul><ul><li>Business partners requested a programmatic interface to load multiple timesheets </li></ul></ul><ul><ul><li>Marketing wants to expose job details from the back office through an internet website </li></ul></ul><ul><li>Oracle Forms application for assigning homes offered SaaS-style </li></ul><ul><ul><li>SaaS Customers want the details on houses and their availability published on a website </li></ul></ul>
  5. 5. Then wat happened… (2) <ul><li>Car Lease company has various custom applications and databases per department </li></ul><ul><ul><li>Business requires IT to support processes that go across those applications and databases </li></ul></ul><ul><ul><ul><li>through a single, unified User Interface </li></ul></ul></ul><ul><ul><ul><li>involving a legacy database and a 3 rd party ERP system </li></ul></ul></ul><ul><ul><ul><li>with eventually some self service web modules </li></ul></ul></ul><ul><li>Insurance company sells policies through agents using a Forms application </li></ul><ul><ul><li>New direct channel: On-line policy selling </li></ul></ul>
  6. 6. Then wat happened… (3) <ul><li>Agricultural company supports ‘cow insemination’ process with Forms application </li></ul><ul><ul><li>Farmers and inspectors need to be able to record data anytime and anywhere through PDAs (that run a .Net application) </li></ul></ul><ul><li>Mid-sized chemical pharmaceutical company uses BoB stand-alone systems and databases </li></ul><ul><ul><li>To allow for faster (near real-time) responses to customer demands and logistical challenges, tighter integration between the systems is needed </li></ul></ul>
  7. 7. Common Characteristics
  8. 8. Objectives <ul><li>Business Agility </li></ul><ul><ul><li>(Faster) responses to changing demands </li></ul></ul><ul><ul><li>Creating new business from existing resources </li></ul></ul><ul><li>Lower costs </li></ul><ul><ul><li>Through reuse, better integration, decoupling </li></ul></ul><ul><li>Higher Quality and Faster Process execution </li></ul><ul><ul><li>Automated data exchange cross boundaries </li></ul></ul><ul><ul><li>Workflow and task orientation </li></ul></ul><ul><ul><li>Business Event driven interaction </li></ul></ul>
  9. 9. SOA = BAD
  10. 10. SOA = B usiness A gility through D ecoupling
  11. 11. Decoupling ≈ Managing Dependencies minimize impact of change while maximizing reusability
  12. 12. Types of decoupling <ul><li>Functional </li></ul><ul><ul><li>Interface - Encapsulation of implementation </li></ul></ul><ul><ul><ul><li>Design by Contract, Implement by Design </li></ul></ul></ul><ul><li>Technical </li></ul><ul><ul><li>No proprietary technology, protocol, message format </li></ul></ul><ul><ul><li>Standards based (XML, HTTP, RSS, WSDL…) </li></ul></ul><ul><li>Temporal </li></ul><ul><ul><li>Asynchronous communication (separate response) </li></ul></ul><ul><li>Development </li></ul><ul><ul><li>Separate teams working in parallel based on mutually agreed interface definitions </li></ul></ul>
  13. 13. Decoupling Applications & Data Data Application (User Interface) Application (User Interface) Data
  14. 14. Decoupling Applications & Data Application (User Interface) Data Application (User Interface)
  15. 15. Decoupling Applications & Data (User Interface) Data Application
  16. 16. Decoupling Applications & Data (User Interface) Application Workflow Engine Email IM Fax CMS
  17. 17. Data Ownership <ul><li>Data no longer exclusively owned by a single application </li></ul><ul><li>Data (query and manipulation) available via APIs, (web)services and open standards </li></ul><ul><ul><li>For example based on XML, XSD, WSDL, SOAP, HTTP </li></ul></ul><ul><li>Data Hubs are formalized, structured approach where data is completely separated from applications </li></ul><ul><ul><li>All access is through services </li></ul></ul><ul><ul><li>No data duplication at all in the enterprise </li></ul></ul><ul><ul><li>Data ownership is separate process </li></ul></ul>
  18. 18. Decoupling from Table to ESB+ http WEBDAV FTP http WS/SOAP WS* WS* WS* WSRP
  19. 19. Tables in Database <ul><li>SQL for retrieval and manipulation </li></ul><ul><li>Data Model in plain “view” </li></ul><ul><li>Decoupling between DML and Retrieval </li></ul><ul><li>select e.empno, e.ename , d.dname from emp join dept using (deptno) </li></ul><ul><li>Insert into dept; insert into emp; </li></ul>
  20. 20. View in Database <ul><li>Hide (encapsulate) Data Model </li></ul><ul><ul><li>Manage access privileges </li></ul></ul><ul><li>SQL for retrieval and manipulation </li></ul><ul><li>Instead Of trigger decouples DML operations </li></ul><ul><li>select id, name, department from emp_vw </li></ul><ul><li>Insert into emp_vw </li></ul><ul><li>Use case: new UI on top of ‘legacy’ data model </li></ul>
  21. 21. Package in Database <ul><li>Hide (encapsulate) SQL </li></ul><ul><li>Procedure calls for retrieval and manipulation </li></ul><ul><ul><li>Potentially complex data structures using Object Types and (nested) Cursors </li></ul></ul><ul><li>HRM_MGR.get_emp( id) return emp_t </li></ul><ul><li>HRM_MGR.create_emp( emp_t); </li></ul><ul><li>Use case: tailor made business services to support (Java) client </li></ul>
  22. 22. Package in Database (2) <ul><li>Hide (encapsulate) SQL and Oracle and user defined Types </li></ul><ul><li>Procedure calls for retrieval and manipulation </li></ul><ul><ul><li>Input and output parameters standard types only (string and number) </li></ul></ul><ul><ul><li>Complex datastructures: XML passed as string </li></ul></ul><ul><li>HRM_MGR.get_emp( id) return string </li></ul><ul><li>HRM_MGR.create_emp( string); </li></ul><ul><li>Use case: packaged business services to support any client (that can access the DB) </li></ul>
  23. 23. Handling XML <ul><li>Structured, multi-level data in a string: XML </li></ul><ul><li>Oracle Database has XMLType </li></ul><ul><ul><li>Can be created from a String, will parse XML </li></ul></ul><ul><ul><ul><li>Can validate against a schema definition (the XML data design) </li></ul></ul></ul><ul><ul><li>Support XPath operations to retrieve specific bits and pieces from the XML document </li></ul></ul><ul><ul><li>Can do XSLT transformations of the incoming or outgoing XML </li></ul></ul>
  24. 24. AQ for Asynchronous <ul><li>Decouple consumer and provider in time – asynchronuous processing </li></ul><ul><li>Consumer is registered on the AQ </li></ul><ul><ul><li>Usually a package that processes User Defined Type that is sent as payload in the AQ message </li></ul></ul><ul><li>Use case: asynchronously processed one-way (fire-n-forget) requests </li></ul><ul><ul><li>Potentially lenghty requests </li></ul></ul><ul><ul><li>High volume of requests </li></ul></ul>
  25. 25. Hiding the database http WEBDAV FTP http WS/SOAP WS* WS* WS*
  26. 26. Publish package as http-based API using dbms_epg <ul><li>Hide database protocol </li></ul><ul><ul><li>Not its physical location nor the schema, and user authentication </li></ul></ul><ul><li>HTTP communication is truly cross technology </li></ul><ul><ul><li>Browser, Java, .Net, JavaScript & RIA clients, … </li></ul></ul><ul><ul><li>Approximation of RESTful services (very du jour) </li></ul></ul><ul><li>Can publish in various formats </li></ul><ul><ul><li>Text, HTML, CSV, JSON, XML, RSS </li></ul></ul><ul><li>Use case:cross-technology, internal no WS*/ESB </li></ul>http
  27. 27. Publish static resources on various protocols with XMLDB <ul><li>Hide database protocol </li></ul><ul><ul><li>support FTP, WEBDAV, HTTP(S) </li></ul></ul><ul><li>Run scheduled batch jobs (PL/SQL) to periodically create & expose resources </li></ul><ul><ul><li>Can also consume and process resources </li></ul></ul><ul><li>Use case: cross technology need for retrieving slowly changing resources (CSV, XML) </li></ul><ul><ul><li>Possibly uploading resources for batch processing </li></ul></ul><ul><li>Use case: </li></ul>WEBDAV FTP http
  28. 28. SOAP based WebServices http WEBDAV FTP http WS/SOAP WS* WS* WS* WSRP
  29. 29. SOAP WebServices <ul><li>All messages (input and output) are XML </li></ul><ul><li>The message consists of two parts inside an envelope (a SOAP XML wrapper) </li></ul><ul><ul><li>The header with meta-data </li></ul></ul><ul><ul><li>The body with the contents to be handled by or returned by the service </li></ul></ul><ul><li>The WebService Definition Language (WSDL) document describes the service </li></ul><ul><li>An XML Schema Document (XSD) describes the structure of the XML messages </li></ul><ul><ul><li>XSD is like an ERD or Table Design </li></ul></ul>
  30. 30. Oracle RDBMS 11g - Native Database WebServices <ul><li>Schema can be published through native database web services </li></ul><ul><ul><li>Each package corresponds with a WSDL </li></ul></ul><ul><ul><li>Every program unit with an operation </li></ul></ul><ul><ul><li>WSDL and XSD are dynamically generated </li></ul></ul>WS/SOAP <ul><ul><li>http and https is supported </li></ul></ul><ul><ul><li>Limited control over WSDL & XSD </li></ul></ul><ul><li>Use case: internal, cross technology, WS enabled client, no ESB or Application Server available </li></ul>
  31. 31. WebService in App Server based on PL/SQL package <ul><li>Hide database </li></ul><ul><ul><li>Protocol, location, authentication: everything handled by the application server </li></ul></ul><ul><li>Use JPublisher (embedded in JDeveloper) to publish a PL/SQL package as WebService </li></ul><ul><ul><li>JPublisher creates JAX-WS-annotated class, utility classes and possibly helper types in the database </li></ul></ul><ul><li>Alternatively: create ADF BC Application Module & publish it as a WebService (Bulldog) </li></ul><ul><ul><li>With support for SDO (Service Data Objects) </li></ul></ul><ul><li>Use case: WS (SDO!) enabled client, no ESB </li></ul>WS*
  32. 32. Enterprise Service Bus Data Service Service Service App 2 Data Service External Partner Service App 1
  33. 33. The Enterprise Service Bus <ul><li>Virtualizes services – hides the real service from consumers </li></ul><ul><ul><li>Deals with the physical location of the services </li></ul></ul><ul><li>Allows callers to use a generic, canonical message structure that it will transform to the service contract </li></ul><ul><ul><li>It may even allow callers to use their own “lingo” </li></ul></ul><ul><li>Adapts synchronous to a-synchronous and vv. </li></ul><ul><li>Can use multiple real services to offer one virtual (composite) service </li></ul>
  34. 34. The Enterprise Service Bus (2) <ul><li>Handles various QoS & SLA aspects </li></ul><ul><ul><li>sometimes in concert with tools like OWSM </li></ul></ul><ul><ul><li>Encryption, signing, authentication </li></ul></ul><ul><ul><li>Retry and fallback </li></ul></ul><ul><ul><li>“ Throttle” (prevent peak loads) </li></ul></ul><ul><li>Does monitoring, tracking & auditing, reporting, notification and escalation </li></ul><ul><li>Works with Adapters to access technologies </li></ul><ul><ul><li>like RDBMS (SQL, PL/SQL), AQ and JMS, File System, FTP, Java, E-Business Suite </li></ul></ul>
  35. 35. Publish PL/SQL Package through ESB <ul><li>Use Database Adapter to create Service, combine with ESB Routing Service </li></ul><ul><li>Use case: </li></ul><ul><ul><li>external access to services </li></ul></ul><ul><ul><li>virtualize location of service – </li></ul></ul><ul><ul><ul><li>route to service based on content of the request </li></ul></ul></ul><ul><ul><li>virtualize part of contract of service </li></ul></ul><ul><ul><ul><li>Package-derived XSD not suitable for consumers </li></ul></ul></ul><ul><ul><li>handle peak loads </li></ul></ul><ul><ul><li>monitor service levels and trace service access </li></ul></ul>WS*
  36. 36. Introducing BPEL for Service <ul><li>BPEL adds to service </li></ul><ul><ul><li>Long running (stateful) ‘service instances’ </li></ul></ul><ul><ul><li>Composite services that include </li></ul></ul><ul><ul><ul><li>Multiple service calls (including asynchronous) </li></ul></ul></ul><ul><ul><ul><li>Exception handling including retry and compensation </li></ul></ul></ul><ul><ul><ul><li>Human Task for manual steps & Integration Rule Engine </li></ul></ul></ul><ul><ul><ul><li>Process flow logic </li></ul></ul></ul><ul><li>Use case: </li></ul><ul><ul><li>data request must be fulfilled by various services; </li></ul></ul><ul><ul><li>dml impacts several systems and/or requires human approval </li></ul></ul>WS*
  37. 37. Publish Business Events <ul><li>Extreme Decoupled Architecture </li></ul><ul><li>Any system – including database – reports events that may be interesting to other parties </li></ul><ul><li>The Event backbone (could be the ESB) </li></ul><ul><ul><li>Defines Event Types (name, structure of payload) </li></ul></ul><ul><ul><li>Registers Event Listeners (“please call me when the event occurs and send the details”) </li></ul></ul><ul><ul><li>Receives events – instances of the predefined event type with payload and timestamp </li></ul></ul><ul><ul><ul><li>Propagate events to all registered listeners </li></ul></ul></ul><ul><ul><ul><li>Without blocking the event producer </li></ul></ul></ul>
  38. 38. Event Driven Architecture (EDA) Data Service Service Service App 2 Data Service External Partner Service App 1
  39. 39. Database publishing events <ul><li>Table Trigger intercepts DML </li></ul><ul><ul><li>Checks for Business Events such as new employee </li></ul></ul><ul><ul><li>Sends them to package EVENT_PRODUCER </li></ul></ul><ul><li>Package EVENT_PRODUCER sends events </li></ul><ul><ul><li>Via UTL_HTTP to a WebService </li></ul></ul><ul><ul><li>Via AQ to a listener (ESB AQ Adapter) </li></ul></ul>Event Producer EMP WS*
  40. 40. Publishing Service with UI <ul><li>Instead of only publishing a programmatic service interface </li></ul><ul><ul><li>A service can be published with a User Interface; the service-with-UI is called: Portlet </li></ul></ul><ul><li>The standard approach: </li></ul><ul><ul><li>Portlet Container in Application Server exposes WSRP services for the portlets </li></ul></ul><ul><ul><li>The Portlet produces (X)HTML and handles HTTP requests </li></ul></ul><ul><ul><li>A Portal consumes the WSRP Portlets in a web page </li></ul></ul>WSRP
  41. 41. Decoupling from Table to ESB+ http WEBDAV FTP http WS/SOAP WS* WS* WS* WSRP
  42. 42. Increasingly decoupled <ul><li>More hiding of the implementation </li></ul><ul><li>More Formal Interface Contract </li></ul><ul><li>Less (proprietary) technology & more standards for interacting </li></ul><ul><li>Less exposure of (legacy) data model </li></ul><ul><li>More support for asynchronous interaction </li></ul><ul><li>More reuse potential </li></ul><ul><li>Pervasive throughout enterprise </li></ul><ul><li>More suitable for external consumption </li></ul>
  43. 43. Comes at a cost… <ul><li>More run time overhead </li></ul><ul><ul><li>Additional tiers </li></ul></ul><ul><ul><li>XML serialization and deserialization </li></ul></ul><ul><li>More infrastructure </li></ul><ul><ul><li>Burden of Administration </li></ul></ul><ul><ul><li>License Costs </li></ul></ul><ul><ul><li>Hardware </li></ul></ul><ul><li>Broader skills palette – more stuff to master </li></ul><ul><li>Harder to get started </li></ul>
  44. 44. Pitfalls <ul><li>Inconsistent, illegible, unstructured namespaces and (XML) data model </li></ul><ul><ul><li>Having the database (table and column names) shine through in the canonical data model </li></ul></ul><ul><li>Introducing new unmanaged dependencies </li></ul><ul><ul><li>Hard coded endpoints (service URLs) </li></ul></ul><ul><ul><li>Calling external services without proper SLA or fallback option </li></ul></ul><ul><ul><li>Using complex technology without proper skills </li></ul></ul>
  45. 45. Pitfalls <ul><li>Lack of Balance between reusability and usefulness (Fine grained vs. coarse grained) </li></ul><ul><li>The greedy clutches of enterprise architects </li></ul><ul><ul><li>Think, talk, (high level) design, draw & write, present, think, talk, …. (no real action) </li></ul></ul><ul><li>Inappropriate use of the SOA infrastructure </li></ul><ul><ul><li>Web applications retrieving each indivual record (or even field) through a separate service call </li></ul></ul><ul><ul><ul><li>Running Forms on top of the ESB! </li></ul></ul></ul><ul><ul><li>Sending debug and trace messages via the ESB </li></ul></ul><ul><ul><li>Package calls other package via ESB </li></ul></ul>
  46. 46. Pay close attention to… <ul><li>Handling errors, exceptions, time out etc. </li></ul><ul><ul><li>Early detection of service unavailability </li></ul></ul><ul><li>SOA and Services Governance </li></ul><ul><ul><li>Involve business in managing services and canonical data model </li></ul></ul><ul><ul><li>Implement process of life cycle management </li></ul></ul><ul><ul><li>Ensure services are well known, findable & trusted </li></ul></ul><ul><li>Involving DBAs/Administrators early on </li></ul><ul><li>Implementing DTAP process & environment </li></ul>
  47. 47. Useful <ul><li>Publish actual XSDs very visibly </li></ul><ul><li>Use Mock Service implementations during development and test </li></ul><ul><li>Automated Functional and Performance Test of individual Services </li></ul><ul><li>Automatic Service ‘ping’ utility </li></ul><ul><li>Quality of Service – SLAs and Monitoring </li></ul><ul><li>Canonical Model also includes Domains, Dictionary and “Identity Repository” </li></ul>
  48. 48. Summary <ul><li>Objective: agility through decoupling </li></ul><ul><ul><li>Managing dependencies </li></ul></ul><ul><ul><li>Crossing boundaries – functional, technology, time </li></ul></ul><ul><li>Just do it! </li></ul><ul><ul><li>Get started – at the right level for your situation </li></ul></ul><ul><ul><ul><li>Do not go off and buy BPEL just like that </li></ul></ul></ul><ul><ul><li>Even though it won’t be perfect the first time round – you will learn (only) through experience </li></ul></ul><ul><li>Do it explicitly, visibly and with all involved </li></ul>SOA