Service Oriented Architecture for  Order Processing in the  IBM Supply Chain Dr. Germán Goldszmidt ( [email_address] ), STSM, IBM Software Group Carl Osipov ( [email_address] ), Software Engineer, IBM Software Group 2166A  Architecting the On Demand Enterprise  October 16, 2006
Agenda COATS/OMCS project business case for the On Demand transformation Technical Demo OMCS transformation to a service-oriented architecture design conceptual framework: business processes, workflows, services enterprise service bus: a messaging and mediation pattern deployment architecture development  roles and responsibilities best practices deployment WBI SF clustering for increased throughput and availability messaging workload balancing using WebSphere MQ clustering Migration to WebSphere Process Server Migration Wizard WebSphere Integration Developer and WebSphere Business Modeler v6 Conclusion Video Q&A
Situation COATS application is a shared order entry   service for more than 20 IBM manufacturing plants worldwide.   It fields hardware orders from IBM customers, IBM business partners, IBM sales professionals and other internal organizations. The l egacy application was  a complex batch system   running on a mainframe. Challenge Rigid legacy code difficult to modify, slowing ability to address new business opportunities and requirements PL/I programmers difficult to find Batch bottlenecks and conflicting data delayed orders and shipments  “ Big bang” total replacement unaffordable and disruptive Customer Order Analysis and Tracking System order-2-manufacturing z-, p-, i- series, storage, printers, retail, …
Business Benefits to COATS reduced analysis, design and code time 25% reduction in the cost of deploying a new application release concurrent, collaborative and iterative development practices enabled by an integrated tools suite improved error tolerance reduced discrepancies in delivery scheduling real-time processing of transactions, from 4 minutes to 10 seconds potential for increased capacity though vertical/horizontal scaling improved application adaptability  to changing business requirements improved existing business processes streamlined the transformation of business processes to executable workflows introduced business rules adaptable in real time by a business analyst introduced sense-and-respond metrics  into business processes average throughput, success ratio, problem resolution time, invalid orders automatic generation of events marked as business relevant at development time Win: IBM Supply Chain Other Associated Wins for Asset:  IBM e-care
Use Processes, Workflows, Services, and Components   Process Models &  Modules Services Workflows Distributed Components transform use expose
DEMO
Development of an SOA solution WebSphere  Business Modeler Rational  Software Architect WebSphere Integration Developer File System export export WebSphere Process Server deploy BPEL XSD WSDL Java Classes BPEL XSD WSDL Java Classes import J2EE ear J2EE ear XSD import Repository export Potential candidates Reusable Assets Asset Governance Analyst Architect Integration Specialist SME Describe “as-is”  Explore ‘to-be”  Assign resources Simulate process  Establish system requirements Specify services to be implemented Choreograph process workflow Iterate and generate executable  Identify and create reusable assets Deployment Manager Deploy application Developer Create adapters, service providers Add details, service implementations
Create Business Process Model Business Analyst
Business Process Modeling expect a learning curve for transitioning from traditional process  modeling tools processes in WebSphere Business Modeler are visually more complex than what business people expect from Visio equivalents processes nested in while loops are not easy to understand the change of nomenclature and modeling artifacts is often confusing adopt a convention on how to use colors  nested vs. simple processes external business rules leave the implementation details out of the business process, focus on services that generate business value create the first iteration of the business process design with the groups outside of IT  deliver a detailed business vision to the IT danger of mixing “as-is” with “to-be” Business Analyst
Configure the generated BPEL workflow Resolve partner links Resolve staff activities Modify transaction boundaries (if required) Specify compensation activities (if required) Add systems fault handling Generate events for business performance monitoring Add code to java snippets Workflow Developer
Staff Activities Steps: create staff activity interface add activity to process  add staff roles, verbs and parameters integrate UI pages select staff resolution plug-in and parameters deploy the process with appropriate staff resolution plug-in Workflow Developer
Modify transaction boundaries Long running processes persist business data for each activity/step in the workflow Persistence of large objects adversely impacts workflow performance # workflow steps * (DB connectivity overhead + Factor of the object size) 10^2 order of magnitude performance difference is typical Long running processes can often be  partitioned into microflows (short running processes) Pass “heavyweight” business objects as parameter to invoke new microflows Order Validation and Topology Generation Order Processing
Workflow Development core Java/EJB skills are important problems when integrating with existing services, existing schemas need to be updated or modified to stay compatible with WID limitations InfoCenter->Reference->Limitations absence of the round-trip modeling (WBM-Rational-WID) lack of explicit support for defining rules, metrics, role, and deployment requirements and exporting to runtime artifacts (eg: J2EE) Workflow Developer
Capture Business Policies Example: SelectManufacturingPlant Select where to route orders based on changing business criteria Analysts specify intended policies using WebSphere Business Modeler Documented through annotations of a task Policies need to be enforced  implemented as one or more rules external to the workflow.   WB Modeler WID WPS Document Policies Specify enforcement points in a business process Implement Rules for identified policies Create Rule service façade Connect workflow to rule services Deploy Rules Administer Rules Deployment Manager Workflow Developer Business Analyst
Business Rules Implementation  choose a business rule approach early in the project WebSphere Business Modeler cannot export business rules in isolation BPEL export tangles the business rules and the flow BPEL does not provide a syntax for externalized business rules consider a WSDL interface to a rule base wsdl:operation  per business rule approach room to grow from a simple lookup to a service-based implementation flexibility of switching a rules engine behind the interface in line with WebSphere Integration Developer/SCA approach
Oneida Lifecycle Adapt CEI CBE CBE WPS Dashboard Monitor Executive Operator End Users
Deployment Topology Process Scalability
Scale Out/Horizontal Clustering early phases of the project used a single server deployment environment embedded WebSphere MQ MQ GET/ PUT QM MQ the production environment leverages WBI SF horizontal clustering external WebSphere MQ  WBI SF specific queues: GET/PUT JMS client, TCP/IP connection WebSphere Network Deployment centralized administration clustering additional WBI SF nodes can be added to the cluster as needed load balancing pSeries p615, 1 CPU, 2 GB RAM WBI Server Foundation v5.1.1 WAS component for agent for remote server management and monitoring built-in WAS  messaging engine: embedded JMS Provider central queue manager can be shared by multiple WBI SF servers WAS Network Deployment manages individual WAS servers via the node agent WBI SF CLUSTER WASND Deployment Manager WBI SF JMS Node Agent WBI SF JMS Node Agent
Clustering Considerations: WBI SF properties of a clustered WBI SF environment increased workload capacity improved resource utilization workload sharing high availability centralized administration of servers drawbacks of centralized messaging no balancing of the  messaging  workload single point of failure limit to the scalability of WBI SF MQ GET/ PUT QM JMS WBI SF CLUSTER WASND Deployment Manager WBI SF JMS Node Agent WBI SF JMS Node Agent
Balance the Messaging Workload/MQ Clustering The OMCS production environment takes advantage  of decentralized WebSphere MQ servers the central messaging server eliminated an instance of WebSphere MQ is installed on each physical server in the cluster send/receive are handled by separate queue managers each MQ server can be added to a cluster shared by WBI SF improved workload balancing  improved performance though local binding  high availability though redundancy MQ MQ PUT QM GET QM PUT QM Deployment Manager Cluster Repository Backup Repository WBI SF JMS Node Agent WBI SF JMS Node Agent GET QM WASND Deployment Manager MQ GET/ PUT QM WBI SF CLUSTER MQ CLUSTER
collocate MQ and WBISF to reduce I/O bottlenecks cluster the MQ servers  to balance the messaging workload minimize use of  the key process bottlenecks process instantiation invoke activity assign reference the capacity planning and tuning guides https://w3.opensource.ibm.com/docman/index.php?group_id=1706&selected_doc_group_id=1524&language_id=1 https://w3quickplace.lotus.com/QuickPlace/wasperf/PageLibrary852569AF00670F15.nsf/h_0DD72A0FDA0EFC0785256E010040EEC1/19A28B673F96C2E085256FB000782BBC/?OpenDocument Optimize for Performance MQ MQ PUT QM GET QM PUT QM Deployment Manager Cluster Repository Backup Repository WBI SF JMS Node Agent WBI SF JMS Node Agent GET QM WASND Deployment Manager WBI SF CLUSTER MQ CLUSTER
Best Practices even with automated code generation from the model, basic software engineering principles have to be followed change management and tracking  WebSphere Business Modeler model -> BPEL and vice versa XSD data model and the Java-based implementation ensure an appropriate skills base for the project auto generated code requires solid Java/EJB foundations initiate a proof-of-concept as needed establish milestones to interlock development artifacts ensure that integration specialists sign off on the business process model before it is converted to BPEL “ bite the bullet” and migrate from Workbench to WebSphere Business Modeler derive visually appealing diagrams as needed plan for performance hit of using BPEL vs. Java implementation typically 3-5x slower design with web service support limitations in mind schema, custom serializers/deserializers
Migrate to WebSphere Process Server 6.0
WebSphere Process Server WPS v6.0 requires BI applications to be created by WID Migration Wizard GUI tool in WID to migrate from BPEL4WS 1.1 to WS-BPEL 2.0 each BPEL process becomes an SCA component uses source code only don’t need to migrate the EAR, EJB and Web projects created by “Generate Deploy Code” in WSAD IE Rulelogic Migration Tool ftp://rolo.torolab.ibm.com/com.ibm.wbit.br.tools_1.0.2.zip Manual verification common pitfalls Multiple replies per BPEL operation  Java snippets http://publib.boulder.ibm.com/infocenter/dmndhelp/v6rxmx/topic/com.ibm.wbit.help.migration.ui.doc/pdf/migrate.pdf Assembly diagram of the SCA components WSADIE “Generate Deploy Code” equivalent optionally define a binding
WebSphere Integration Developer Migration Wizard  migrate non-WBI projects first ensure J2EE 1.4 compliance documented in the Rational Software Architect InfoCenter http://publib.boulder.ibm.com/infocenter/rtnl0600/topic/com.ibm.etools.rad.migration.doc/topics/tmigratefrom51x.html separate the BPEL logic from the service definition use a Business Integration Library project to store WSDL & XSD files migrate one service/BPEL project at a time for each project, follow the steps in the InfoCenter http://publib.boulder.ibm.com/infocenter/dmndhelp/v6rxmx/topic/com.ibm.wbit.help.migration.ui.doc/topics/tprepsrcart.html
WebSphere Business Modeler WebSphere Business Modeler Version 5 export to WSAD IE 5.1 requires the Migration Wizard to convert BPEL4WS 1.1 to WS-BPEL 2.0 WebSphere Business Modeler Version 6 Enhancements BPEL Export ability to export multiple input and output sets and correlations ability to export process models as BPEL4WS 1.1 or WS-BPEL 2.0  Workplace migration:  run Modeler 6.0 against a Modeler 5.1.x workspace.  automatic backup of the workspace prior to migration import a Modeler 5.1.x project  check out a project from the team repository on the fly migration during the check out
Conclusion
VIDEO
Lessons learned/best practices Reinforce basic software engineering principles for auto generated code Change management and tracking  Ensure an appropriate skills base for the project Initiate a proof-of-concept as needed Establish milestones to interlock development artifacts Ensure that integration specialists sign off on the business process model before it is converted to BPEL Design and validate web services against a common data model Use document/literal Powered by Software:  WebSphere MQ, Portal, WAS, WBI-SF, DB2 Migrating to WebSphere Process Server and WebSphere Business Monitor Tools: Rational XDE, WBI Modeler, WSADIE, Rational Software Architect, WebSphere Integration Developer Patterns: IBM patterns for e-business, SOA, ESB Solution Flexible architecture that supports the ongoing transformation of COATS into highly adoptable application Real time transaction processing  Legacy business logic converted to externalized business rules and workflows of reusable services to improve adaptability to changing requirements Business process modeling and workflow generation -  reduced analysis, design and coding time Generation of sense-and-respond metrics Results Affordable incremental migration from legacy Order transaction processing time reduced from  4 minutes to   10 seconds   Ability to make   on demand changes to the run time workflow ,  through easily selectable rules 25+% reduction in development time/cost  SOA Reference architecture  and  best practices  that are being replicated across IBM and clients Customer Order Analysis and Tracking System order-2-manufacturing z-, p-, i- series, storage, printers, retail, …
On Demand Business Process Lifecycle developerWorks articles series 1.  Create the foundation for your on demand business processes   2.  Patterns for e-business recipe   3.  Business process modeling using WebSphere Business Integration Modeler   4.  Integrate artifacts from Rational XDE and WebSphere Business Integration Modeler   5.  Workflow development, deployment, and testing   6.  Apply customization policies and rules   7.  Monitor business processes and emit events using CEI   8.  Business process monitoring -- Create key performance indicators   9.  Involve people   10.  Develop message adapters for CICS transaction servers   11.  Integrate business processes with CICS transaction servers   12.  Implement a compensation service   13.  Deploy in a clustered environment 14.  Use a clustered WebSphere MQ deployment to balance messaging workload 15.  Deploy a scalable, secure and stable foundation for a Service-Oriented Architecture
Conclusion Built a business case of an SOA transformation of a legacy IBM supply chain application demonstrated value to IBM Designed and developed the OMCS application using WebSphere Business Integration tooling Refactored the deployment architecture to improve throughput, availability and performance WBI SF and MQ clustering Began migration to WebSphere Process Server
Click to edit Master title style QUESTIONS?
Click to edit Master title style THANK YOU!

A Service Oriented Architecture For Order Processing In The I B M Supply Chain

  • 1.
    Service Oriented Architecturefor Order Processing in the IBM Supply Chain Dr. Germán Goldszmidt ( [email_address] ), STSM, IBM Software Group Carl Osipov ( [email_address] ), Software Engineer, IBM Software Group 2166A Architecting the On Demand Enterprise October 16, 2006
  • 2.
    Agenda COATS/OMCS projectbusiness case for the On Demand transformation Technical Demo OMCS transformation to a service-oriented architecture design conceptual framework: business processes, workflows, services enterprise service bus: a messaging and mediation pattern deployment architecture development roles and responsibilities best practices deployment WBI SF clustering for increased throughput and availability messaging workload balancing using WebSphere MQ clustering Migration to WebSphere Process Server Migration Wizard WebSphere Integration Developer and WebSphere Business Modeler v6 Conclusion Video Q&A
  • 3.
    Situation COATS applicationis a shared order entry service for more than 20 IBM manufacturing plants worldwide. It fields hardware orders from IBM customers, IBM business partners, IBM sales professionals and other internal organizations. The l egacy application was a complex batch system running on a mainframe. Challenge Rigid legacy code difficult to modify, slowing ability to address new business opportunities and requirements PL/I programmers difficult to find Batch bottlenecks and conflicting data delayed orders and shipments “ Big bang” total replacement unaffordable and disruptive Customer Order Analysis and Tracking System order-2-manufacturing z-, p-, i- series, storage, printers, retail, …
  • 4.
    Business Benefits toCOATS reduced analysis, design and code time 25% reduction in the cost of deploying a new application release concurrent, collaborative and iterative development practices enabled by an integrated tools suite improved error tolerance reduced discrepancies in delivery scheduling real-time processing of transactions, from 4 minutes to 10 seconds potential for increased capacity though vertical/horizontal scaling improved application adaptability to changing business requirements improved existing business processes streamlined the transformation of business processes to executable workflows introduced business rules adaptable in real time by a business analyst introduced sense-and-respond metrics into business processes average throughput, success ratio, problem resolution time, invalid orders automatic generation of events marked as business relevant at development time Win: IBM Supply Chain Other Associated Wins for Asset: IBM e-care
  • 5.
    Use Processes, Workflows,Services, and Components Process Models & Modules Services Workflows Distributed Components transform use expose
  • 6.
  • 7.
    Development of anSOA solution WebSphere Business Modeler Rational Software Architect WebSphere Integration Developer File System export export WebSphere Process Server deploy BPEL XSD WSDL Java Classes BPEL XSD WSDL Java Classes import J2EE ear J2EE ear XSD import Repository export Potential candidates Reusable Assets Asset Governance Analyst Architect Integration Specialist SME Describe “as-is” Explore ‘to-be” Assign resources Simulate process Establish system requirements Specify services to be implemented Choreograph process workflow Iterate and generate executable Identify and create reusable assets Deployment Manager Deploy application Developer Create adapters, service providers Add details, service implementations
  • 8.
    Create Business ProcessModel Business Analyst
  • 9.
    Business Process Modelingexpect a learning curve for transitioning from traditional process modeling tools processes in WebSphere Business Modeler are visually more complex than what business people expect from Visio equivalents processes nested in while loops are not easy to understand the change of nomenclature and modeling artifacts is often confusing adopt a convention on how to use colors nested vs. simple processes external business rules leave the implementation details out of the business process, focus on services that generate business value create the first iteration of the business process design with the groups outside of IT deliver a detailed business vision to the IT danger of mixing “as-is” with “to-be” Business Analyst
  • 10.
    Configure the generatedBPEL workflow Resolve partner links Resolve staff activities Modify transaction boundaries (if required) Specify compensation activities (if required) Add systems fault handling Generate events for business performance monitoring Add code to java snippets Workflow Developer
  • 11.
    Staff Activities Steps:create staff activity interface add activity to process add staff roles, verbs and parameters integrate UI pages select staff resolution plug-in and parameters deploy the process with appropriate staff resolution plug-in Workflow Developer
  • 12.
    Modify transaction boundariesLong running processes persist business data for each activity/step in the workflow Persistence of large objects adversely impacts workflow performance # workflow steps * (DB connectivity overhead + Factor of the object size) 10^2 order of magnitude performance difference is typical Long running processes can often be partitioned into microflows (short running processes) Pass “heavyweight” business objects as parameter to invoke new microflows Order Validation and Topology Generation Order Processing
  • 13.
    Workflow Development coreJava/EJB skills are important problems when integrating with existing services, existing schemas need to be updated or modified to stay compatible with WID limitations InfoCenter->Reference->Limitations absence of the round-trip modeling (WBM-Rational-WID) lack of explicit support for defining rules, metrics, role, and deployment requirements and exporting to runtime artifacts (eg: J2EE) Workflow Developer
  • 14.
    Capture Business PoliciesExample: SelectManufacturingPlant Select where to route orders based on changing business criteria Analysts specify intended policies using WebSphere Business Modeler Documented through annotations of a task Policies need to be enforced implemented as one or more rules external to the workflow. WB Modeler WID WPS Document Policies Specify enforcement points in a business process Implement Rules for identified policies Create Rule service façade Connect workflow to rule services Deploy Rules Administer Rules Deployment Manager Workflow Developer Business Analyst
  • 15.
    Business Rules Implementation choose a business rule approach early in the project WebSphere Business Modeler cannot export business rules in isolation BPEL export tangles the business rules and the flow BPEL does not provide a syntax for externalized business rules consider a WSDL interface to a rule base wsdl:operation per business rule approach room to grow from a simple lookup to a service-based implementation flexibility of switching a rules engine behind the interface in line with WebSphere Integration Developer/SCA approach
  • 16.
    Oneida Lifecycle AdaptCEI CBE CBE WPS Dashboard Monitor Executive Operator End Users
  • 17.
  • 18.
    Scale Out/Horizontal Clusteringearly phases of the project used a single server deployment environment embedded WebSphere MQ MQ GET/ PUT QM MQ the production environment leverages WBI SF horizontal clustering external WebSphere MQ WBI SF specific queues: GET/PUT JMS client, TCP/IP connection WebSphere Network Deployment centralized administration clustering additional WBI SF nodes can be added to the cluster as needed load balancing pSeries p615, 1 CPU, 2 GB RAM WBI Server Foundation v5.1.1 WAS component for agent for remote server management and monitoring built-in WAS messaging engine: embedded JMS Provider central queue manager can be shared by multiple WBI SF servers WAS Network Deployment manages individual WAS servers via the node agent WBI SF CLUSTER WASND Deployment Manager WBI SF JMS Node Agent WBI SF JMS Node Agent
  • 19.
    Clustering Considerations: WBISF properties of a clustered WBI SF environment increased workload capacity improved resource utilization workload sharing high availability centralized administration of servers drawbacks of centralized messaging no balancing of the messaging workload single point of failure limit to the scalability of WBI SF MQ GET/ PUT QM JMS WBI SF CLUSTER WASND Deployment Manager WBI SF JMS Node Agent WBI SF JMS Node Agent
  • 20.
    Balance the MessagingWorkload/MQ Clustering The OMCS production environment takes advantage of decentralized WebSphere MQ servers the central messaging server eliminated an instance of WebSphere MQ is installed on each physical server in the cluster send/receive are handled by separate queue managers each MQ server can be added to a cluster shared by WBI SF improved workload balancing improved performance though local binding high availability though redundancy MQ MQ PUT QM GET QM PUT QM Deployment Manager Cluster Repository Backup Repository WBI SF JMS Node Agent WBI SF JMS Node Agent GET QM WASND Deployment Manager MQ GET/ PUT QM WBI SF CLUSTER MQ CLUSTER
  • 21.
    collocate MQ andWBISF to reduce I/O bottlenecks cluster the MQ servers to balance the messaging workload minimize use of the key process bottlenecks process instantiation invoke activity assign reference the capacity planning and tuning guides https://w3.opensource.ibm.com/docman/index.php?group_id=1706&selected_doc_group_id=1524&language_id=1 https://w3quickplace.lotus.com/QuickPlace/wasperf/PageLibrary852569AF00670F15.nsf/h_0DD72A0FDA0EFC0785256E010040EEC1/19A28B673F96C2E085256FB000782BBC/?OpenDocument Optimize for Performance MQ MQ PUT QM GET QM PUT QM Deployment Manager Cluster Repository Backup Repository WBI SF JMS Node Agent WBI SF JMS Node Agent GET QM WASND Deployment Manager WBI SF CLUSTER MQ CLUSTER
  • 22.
    Best Practices evenwith automated code generation from the model, basic software engineering principles have to be followed change management and tracking WebSphere Business Modeler model -> BPEL and vice versa XSD data model and the Java-based implementation ensure an appropriate skills base for the project auto generated code requires solid Java/EJB foundations initiate a proof-of-concept as needed establish milestones to interlock development artifacts ensure that integration specialists sign off on the business process model before it is converted to BPEL “ bite the bullet” and migrate from Workbench to WebSphere Business Modeler derive visually appealing diagrams as needed plan for performance hit of using BPEL vs. Java implementation typically 3-5x slower design with web service support limitations in mind schema, custom serializers/deserializers
  • 23.
    Migrate to WebSphereProcess Server 6.0
  • 24.
    WebSphere Process ServerWPS v6.0 requires BI applications to be created by WID Migration Wizard GUI tool in WID to migrate from BPEL4WS 1.1 to WS-BPEL 2.0 each BPEL process becomes an SCA component uses source code only don’t need to migrate the EAR, EJB and Web projects created by “Generate Deploy Code” in WSAD IE Rulelogic Migration Tool ftp://rolo.torolab.ibm.com/com.ibm.wbit.br.tools_1.0.2.zip Manual verification common pitfalls Multiple replies per BPEL operation Java snippets http://publib.boulder.ibm.com/infocenter/dmndhelp/v6rxmx/topic/com.ibm.wbit.help.migration.ui.doc/pdf/migrate.pdf Assembly diagram of the SCA components WSADIE “Generate Deploy Code” equivalent optionally define a binding
  • 25.
    WebSphere Integration DeveloperMigration Wizard migrate non-WBI projects first ensure J2EE 1.4 compliance documented in the Rational Software Architect InfoCenter http://publib.boulder.ibm.com/infocenter/rtnl0600/topic/com.ibm.etools.rad.migration.doc/topics/tmigratefrom51x.html separate the BPEL logic from the service definition use a Business Integration Library project to store WSDL & XSD files migrate one service/BPEL project at a time for each project, follow the steps in the InfoCenter http://publib.boulder.ibm.com/infocenter/dmndhelp/v6rxmx/topic/com.ibm.wbit.help.migration.ui.doc/topics/tprepsrcart.html
  • 26.
    WebSphere Business ModelerWebSphere Business Modeler Version 5 export to WSAD IE 5.1 requires the Migration Wizard to convert BPEL4WS 1.1 to WS-BPEL 2.0 WebSphere Business Modeler Version 6 Enhancements BPEL Export ability to export multiple input and output sets and correlations ability to export process models as BPEL4WS 1.1 or WS-BPEL 2.0 Workplace migration: run Modeler 6.0 against a Modeler 5.1.x workspace. automatic backup of the workspace prior to migration import a Modeler 5.1.x project check out a project from the team repository on the fly migration during the check out
  • 27.
  • 28.
  • 29.
    Lessons learned/best practicesReinforce basic software engineering principles for auto generated code Change management and tracking Ensure an appropriate skills base for the project Initiate a proof-of-concept as needed Establish milestones to interlock development artifacts Ensure that integration specialists sign off on the business process model before it is converted to BPEL Design and validate web services against a common data model Use document/literal Powered by Software: WebSphere MQ, Portal, WAS, WBI-SF, DB2 Migrating to WebSphere Process Server and WebSphere Business Monitor Tools: Rational XDE, WBI Modeler, WSADIE, Rational Software Architect, WebSphere Integration Developer Patterns: IBM patterns for e-business, SOA, ESB Solution Flexible architecture that supports the ongoing transformation of COATS into highly adoptable application Real time transaction processing Legacy business logic converted to externalized business rules and workflows of reusable services to improve adaptability to changing requirements Business process modeling and workflow generation - reduced analysis, design and coding time Generation of sense-and-respond metrics Results Affordable incremental migration from legacy Order transaction processing time reduced from 4 minutes to 10 seconds Ability to make on demand changes to the run time workflow , through easily selectable rules 25+% reduction in development time/cost SOA Reference architecture and best practices that are being replicated across IBM and clients Customer Order Analysis and Tracking System order-2-manufacturing z-, p-, i- series, storage, printers, retail, …
  • 30.
    On Demand BusinessProcess Lifecycle developerWorks articles series 1. Create the foundation for your on demand business processes 2. Patterns for e-business recipe 3. Business process modeling using WebSphere Business Integration Modeler 4. Integrate artifacts from Rational XDE and WebSphere Business Integration Modeler 5. Workflow development, deployment, and testing 6. Apply customization policies and rules 7. Monitor business processes and emit events using CEI 8. Business process monitoring -- Create key performance indicators 9. Involve people 10. Develop message adapters for CICS transaction servers 11. Integrate business processes with CICS transaction servers 12. Implement a compensation service 13. Deploy in a clustered environment 14. Use a clustered WebSphere MQ deployment to balance messaging workload 15. Deploy a scalable, secure and stable foundation for a Service-Oriented Architecture
  • 31.
    Conclusion Built abusiness case of an SOA transformation of a legacy IBM supply chain application demonstrated value to IBM Designed and developed the OMCS application using WebSphere Business Integration tooling Refactored the deployment architecture to improve throughput, availability and performance WBI SF and MQ clustering Began migration to WebSphere Process Server
  • 32.
    Click to editMaster title style QUESTIONS?
  • 33.
    Click to editMaster title style THANK YOU!

Editor's Notes

  • #4 The goals of the on demand transformation is to achieve faster time to value, lower development costs, and increased capacity of BT projects. The high-level goals of the Oneida-2 PoC included: to demonstrate value to the business, to prepare and plan towards an on demand transformation, and to document best practices for other engagements. A stretch goal, was to actually move some of the pilot to the real production environment. Then, we want to scale and replicate this on multiple other internal IT projects for IBM. COATS is a shared order entry service for more than 20 manufacturing plants worldwide. It fields hardware orders from IBM customers, IBM Business Partners, IBM sales professionals and other internal organizations. The application sorts and prioritizes these orders, comparing them against manufacturing rules and the customer’s installed hardware base. Then, several times each day, it routes material lists and instructions to appropriate manufacturing facilities. C OATS supports orders for "complex” configured hardware: P-, I-, Z- Series, Storage, Printers, and Retail. Salespersons, business partners, or client buyers indirectly access COATS by introducing orders for new machines, upgrades, customizations, and other changes to existing customer orders. COATS “translates” the flow of customer orders into manufacturing material lists (Bill of Materials) and other instructions, which are then forwarded to IBM manufacturing plants worldwide. The manufacturing plants then fulfill the orders and ship them to the customers’ premises. An example of an order will be for a mainframe with detailed specifications, pre-loaded software, delivery date, etc. An example of the corresponding output will include a configurations to the manufacturing floor of a specific plant, in the corresponding nomenclature, including bill of materials, configuration plan, etc. The original application is a complex batch system, used as a shared “insourced” service. COATS supports many plants, and each plant has its own customization needs and access patterns, e.g. ‘High Volume-Low Price’ vs ‘High price-Low Volume’. Its history goes back over 25 years, and now includes 1.4MLoc of PL/1, OS/390 Assembler, Java, etc. It is used as a shared “insourced” service , with worldwide 365x24 coverage, supporting many plants, each plant has customization needs. It is running close to capacity at peak times. There are many a lterations to orders, including automatic alterations (scheduler system) to meet customer on time date. Multiple databases need to be updated and queried, depending on geography and other parameters, e.g., sold machines history is kept 5-10 years, in multiple history DBs. The application is continually being updated (quarterly versions, each takes 6 months to develop), to support new initiatives, product introductions, business opportunities, and outsourcing requirements. It delivers orders to manufacturing more than 20 times daily. It translates sales orders into manufacturing material lists and instructions, for new machines, upgrades, engineering changes, and delivers orders to manufacturing 20+ times daily.
  • #5 1:30 SOA is here, it works, and is helping IBM save money. In addition to cutting the application development costs per release by 25%, the transformed application is more resilient to errors and hence has better availability. The performance of the application improved following the transformation to SOA and flexibility is designed into the architecture to add capacity to handle higher workloads in the future. Using Modeler did not only improve the existing processes but also given the project stakeholders the flexibility to simulate the system and identify new opportunities for process improvement. The ability to simulate the system is also helpful in analysing scenarios for response to changing business requirements and market conditions. To developers the introduction of the modeler means automated translation of the process changes and requirements into BPEL. As you have seen from the demo and the presentation, the application is easy for business analysts to modify via Modeler and the business rule editor. And the last but certainly not least, business services provided by the application became more visible to executives though sense and respond metrics collected from the business processes.
  • #8 1:30 So how does the transformation actually happen? Who needs to be involved and what products does a team need to transform a legacy application to SOA? Early in a project lifecycle, the business analyst engages with the subject matter experts who are typically business owners of the transformation effort and model the business processes. Throughout this exercise, the business analyst can reuse any of the already modeled processes and business objects. Often some or all of the components that are needed to implement a business process may not be in place. The architect is responsible for the design of the required components and the initial code that should be used for the actual implementation. The article produced by both the architect and the analyst is imported into a programming environment where the developers can continue the work to produce deployable code When the application is running into production, a reuse engineer who is familiar with a particular domain can review the assets and identify what can be reused First the business analysts creates a model using WBI Modeler: Explore ‘to-be” business services, Assign resources, Perform process simulations. Identifying patterns in business solutions and other reusable assets from repositories. IT architects use Rational XDE for use case modeling of the business scenario and to develop an object model for the workflow process and services. Generate data models from UML. The resulting artifacts are used by the development team to produce an executable implementation. Java code is generated from the object model using the XDE code generation facilities. The generated Java code is imported into WSAD-IE. The exported object model is then converted to XML schema elements using WSAD-IE. The development team uses WSAD-IE to integrate the Java code and the XML schema elements with additional components and services to complete an executable workflow implementation. They c horeograph business process workflow, create adapters and services, add technology details and service implementations. They generate Web service code for legacy façade components; Integrate rules (BR Beans) using a rule service façade to integrate with workflow and components; Generate necessary WSDL/XSD and deployment artifacts; Enable Legacy access; Customize process templates (BPEL+) e.g. Java snippets, custom components; Integrate legacy components service interfaces; Integrate Staff activities for decision making, exception handling, etc; Generate (business, system) events for Key Performance Indicators; Integrate a presentation layer through Portal; Store reusable assets back into repository; The resulting application (EAR file) is then deployed on WBI-SF. (Deployment Manager); Reusable assets such as existing XSD schemas can be imported into WBI Modeler before the modeling process. SMEs identify and create reusable assets. TOOLS USED: WBI Modeler 5.1 , Rational XDE, WSAD-IE 5.1, WBI-SF 5.1, +Common Event Infrastructure (Tech Preview) , Websphere Portal Express (Toolkit 5.021), ALSO (not shown) Rational Requisite Pro, DB2, MQ Series, CICS 2.3, IBM Open Bazar CVS repository Development Steps
  • #9 0:45 Lets take a look at the development lifecycle step by step starting with the process model development. The figure on the slide shows a modeled portion of an “as-is” business process. Of course on of the advantages of having a process model is that nce the model is created, the business analyst can make changes to it and immediately simulate the impact of these changes on the KPIs defined by the stakeholders. The goal of exploring various changes to the model is to get to the “to-be” version of the process that the business stakeholders want to implement.
  • #10 Traditionally, business analysts have experience in using Visio or a similar diagramming tool to model business processes. Since a significant aspect of the business analyst’s role is to communicate the process model to the various stakeholders, usually we find that analysts settle on a particular style for appearance of the processes models which makes it easier to communicate the model to the stakeholders.
  • #14 2:30 What are the caveats of the workflow development process? One of the common mistakes that teams make is forgetting that the workflow implementation in WSAD IE is EJB based and not hiring skills with sufficient expertise in Java/EJB. In fact, because the workflow implementation code is auto generated and is not well commented, it is imperative that the developers who are expected to maintain the workflow code should be familiar with the underlying Java/EJB technologies. Another issue that developers forget when designing complex schemas from scratch or adopting schemas from standards organizations such as oasis, is that WSADIE and WBI SF do not support arbitrary schemas out of the box. Before signing off on a schema implementation is it important to cross-reference the schema definition against the list of the limitations that are documented in the WSADIE Infocenter. Same applies to WID/WPS products. If a project is considering to adopt the WBI framework, one of the key questions to answer early in the project lifecycle relates to the change management of the business process modeling and the BPEL workflows. The current tooling does not support round tripping of changes from WSADIE/WID to Modeler or vice versa. Another design decision for a project is to identify strategy to handle the rules, metrics and roles and other requirements embedded in a process model. These definition are lost when exported to BPEL and will not be available in the final deployable artifact: the EAR file. I’ll talk about an approach to handle one of the elements of the process model that I mentioned in the next few slides that describe the business policy and rule implementation. Finally, another feature that was available in the past as a part of WebSphere InterChange Server, WICS – adapters is not yet fully supported in WSADIE, although there is some partial support of the WICS adapters in the WPS and WID 6.0.1.
  • #15 1:30 The previous slide mentioned business policies as one of the examples of an aspect of a business process model which doesn’t not map directly to BPEL syntax. So what is a policy? It is typically a declarative statement that places a restriction on operation of business processes, e.g., " only USA customers can order machine X ". Each policy may require one or more enforcement points in the implementation. An enforcement point may be implemented as an explicit step in a process or a specific location in the code. After the enforcement points are defined in the process, and the process model is converted into BPEL, the workflow developer needs to implement the business rules to support the policies. One of the options for implementing the rule which I’ll describe in more detail in the next slide is to have the rules be services external to the process and to use the decisions made by the rules as variables in the BPEL workflow. This option is essentially an implementation of a façade pattern where a single web service interface façade provides access to the business rules needed by the workflow. Then the business rules are implemented as invoke activities in the flow and are bound to the specific rule implementation through partner links. Finally, after the application is running, business rules can be managed using clients shipped in the WBI SF and WPS products.
  • #17 The Oneida Lifecycle goes beyond the development lifecycle to cover Business Process management (BPM). BPM covers the collection of business process events performance and input during execution, presentation and analysis, followed by adaptation and optimization. We used the Common Base Events (CBE) format to emit the relevant events from the execution. CBEs are retrieved and correlated to create and display Key Performance Indicators (KPIs), which are used to refine the business processes. CBEs are stored and distributed using the Common event Infrastructure (CEI). The CEI accepts events from various sources, and performs a set of tasks: It persists the events for later query and reporting, and publishes the events to interested subscribers. If can filter out selected events based on xPath expression. It can support multiple qualities of service for event transport and delivery. CBE is its native format, but CEI can support multiple event message formats. Events are correlated and monitored by the business executives using a performance dashboard that highlights the Key Performance Indicators. We used portlets to render and aggregate information into composite pages to provide information to users in a compact form. The portal also allows operators to handle assigned work items assigned, such as order exceptions.
  • #23 Add more best practices
  • #30 Learn about how to Build reusable assets to transform an order processing system in the developerWorks series On demand business process life cycle http://www-128.ibm.com/developerworks/ibm/ws-odbp/ Find out more about Oneida and COATS from http://w3.ibm.com/articles/workingknowledge/2005/04/swg_odsd_oneida.html See the latest OMCS/COATS demo at http://w3.webahead.ibm.com/w3ki/display/oneida/Demos Business value: a) increased order volume capacity b) faster turn around time to implement manufacturing requirements, and (on line request from manufacturing used to take now in 10 seconds vs 4 minutes in previous environment!) c) ability to make on demand changes to the run time workflow, through easily selectable business rules. (early added rule). In COATS, they had a daily average of approximately 2500 requests (about 300 at peak hours). Thus, 4 minutes savings, potentially more than 150 hours daily speed-up in handling of orders per day. (At this time, only a selected few are using OMCS in a trial basis, yesterday only 50 such requests). Today, using COATS, a manufacturing plant customer that enters an "order transaction“, (to speed up an order, move the order more quickly through processing to get down to mfg floor) must wait about 4 minutes for its completion. Using OMCS R1 portal a similar "order transaction" now takes approximately 10 seconds. We expect about 200 such transactions in an average day using OMCS R1. Thus, for the aggregate customers of supply chain, the wait time was reduced by 13 hours. . Additional performance benefits are expected in R2, when a significantly larger volume of orders will flow thru OMCS. All Release requests will process in R2 as well as on-line for MFC The manufacturing requests are typically "manual fixes" of orders, in which the staff from the plant update sales order with the right data, (from sales nomenclature to available parts numbers) or attempt to expedite an order in the scheduling system because it has priority, etc. From Feb 7, we feed actual orders ON-LINE through the new OMCS step for NewC validation, and continue with the legacy to manufacturing. CICS front end, WebsPhere to DB2 connection, COBOL instread of Pl/I to use OMCS (Order Management Component Services) project is transforming a subset of the overall COATS system to a real-time order submission system. This project makes extensive use of workflows, services, business rules, enterprise service bus (ESB) and other integration technology provided by WBI-SF. The users are integrated to applications through WPS. We connected systems such as legacy COATS and NEWC Order Validation/Configuration services with business process through CICS and MQ. The development process started with business process modeling using WBI Modeler. The workflows are enhanced using the WSAD-IE tool with business objects from Rational XDE and integration of legacy adaptors. During this work we harvested reusable development assets and best practices: § to connect to legacy systems using SOAP for CICS modules § rule access services and an object framework to integrate BRBeans rule engine § shared business process (e.g., Milestone Manager Process) across business units § used IBM Open Bazaar to share the process/code/workflows across projects § Stored procedures to connect to legacy databases from the workflows Values to OMCS This project helps COATS to implement a method for incremental transformation of their legacy business functions. We demonstrated incremental development methods, proper use of the tools, and middleware components. This results in improved performance, reduced development coat and faster development turn around time. The customizable business process improved the runtime customization of the processes by business owners. A better error handing facility is included with the processes for faster exception processing resulted in huge revenue savings. Values to external market This project demonstrated an on demand transformation of the legacy business through the proper use of elements of IBM middleware components. The assets and best practices demonstrated in this project could be replicated across other business units. For example, the knowledge could be shared about how to transform PL1 modules and integrate with the business processes, how to connect legacy functions from workflow using MQ and how to implement customizable workflows. The SOA based development methods and proper use of tools illustrates faster time to market. The process customization, incremental transformation, and legacy integration methods demonstrated in this project will helps create reusable assets and best practices across industry. From the WEB the customer can request to send from 1 to 30 MFGNO's. That in turn will get all the associated order numbers for thoses MFGNO's and send to NEWC for RELEASE Processing. The returning data from NEWC will update 5 COAT DB2 tables with topology and process information. We had 89 Release batch runs at an average batch run based on 1/09 to 02/09 of 4 minutes. The processed 7000 to Release so average would be approx 80. per run.
  • #32 Add reference to a proof point – add the case in point chart, url. Add slides on the SOA+ collaboration, rowing chart… research, internal accounts. List of the projects where Oneida has been used. Expand on one of the projects in the deck