Integration Approach for MES
Upcoming SlideShare
Loading in...5
×
 

Integration Approach for MES

on

  • 169 views

This approach will help to change the traditional approach of point-to-point communication in Manufacturing Execution Systems (MES) to using BizTalk server as a middleware to Integrate several systems

This approach will help to change the traditional approach of point-to-point communication in Manufacturing Execution Systems (MES) to using BizTalk server as a middleware to Integrate several systems

Statistics

Views

Total Views
169
Views on SlideShare
169
Embed Views
0

Actions

Likes
0
Downloads
5
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft Word

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Integration Approach for MES Integration Approach for MES Document Transcript

  • Table of Contents 1. Asset Bounds ...................................................................................................................................... 3 2. Asset Introduction ............................................................................................................................ 3 3. Advantages of BizTalk ........................................................................................................................... 4 4. Integration Process ............................................................................................................................... 5 4.1 File or Web Service ....................................................................................................................... 6 5. Adapter and Integration Solutions........................................................................................................ 7 6. BIZTALK Best Practices .......................................................................................................................... 8 a. Design Best Practices ........................................................................................................................ 8 b. Schema Creation Best Practices........................................................................................................ 9 c. Map Creation Best Practices ........................................................................................................... 10 d. Orchestration Best Practices ........................................................................................................... 11 e. Best Practices for Performance ...................................................................................................... 15 1
  • Integration Reusable Assets - Systems Integration Approach for MES Date 27/1/2014 Owner Vinod kumar kodatham Description This approach will help to change the traditional approach of point-to-point communication inManufacturing Execution Systems (MES) to using BizTalk server as a middleware to Integrate several systems. Multiple channels of several systems can be plugged into the centralized BizTalk server.Enterprise Service Bus (BizTalk server)with its inbuilt BAM and ESB capabilities will help especially for reporting Application tracking and application failures in respective BAM and ESB portals respectively. These are totally reusable and isolated which can be used by any application across the industry which uses BizTalk as the middle ware platform OBJECT OVERVIEW Technical Name Description Integration Approach for MES. The asset will provide high level design considerations which enable seam less integration cross platform and systemsusing BizTalk framework which developers can modify as per the requirement. 2
  • 1. Asset Bounds This asset can be used for the several simple to complex Integrationscenarios that needs to scale from low to high volumes with low latency and need of fault tolerance that has got redundancy, scalable and message retry featuresand governance over the solution.The conceptual architecture of a solution is done by even taking into account non-functional requirements like performance, availability, extensibility, resilience, robustness, and scalability and ensure they will be properly addressed during the technical design phase. This gives you out of the box features to develop/customize all these functionalities. The asset is independent of Source and Target System(s)/ Application. 2. Asset Introduction This document describes the idea of systems Integration using a centralized BizTalk application i.e. using BizTalk framework as an Integration platform for integrating systems (process in MES System) like POM, RIM, IOM, RCM, QOM, SAP and PCS (Process Control Systems). BizTalk framework orchestrates the integration process by maintaining the performance, robustness and availability.This framework will describe the design decision made and development task required to build the BizTalk application’s components. Any Business scenario which operates with integration has the above typical interface. 1. Business has a need to transfer information from one application to a different application or one system to a different system that are residing in a single or different multiple servers. These applications can be either in a same server or a different servers 2. These applications can be either an application server which uses Webservices or a database server. Now this source data needs to be migrated to the target system meeting the business requirements which is called as transformations. 3. In order to integrate these two different systems so that they can communicate each other business requires a Middleware i.e. BizTalk server. There are lots of Middleware technologies available in market but if you need to go for the best then we say its BizTalk server as it is one of the most successful product in the market. It’s a Microsoft product. The benefits of this tool are mention in the very next section. 3
  • 3. Advantages of BizTalk Figure 2: Orchestrated Data transfer Process ESB Portal(6) Monthly basis(2) BizTalk(4) Receive Port Exception Handling(5) Orchestration Process WCF-Custom Source Webservice(1) Send Port WCF Custom Target Webservice(7) e Retrieve Source Data(3) Receive Pipeline Send Pipeline Message Box While planning for the integration scenario BizTalk takes care about tasks like Gathering information, Defining naming conventions, planning team development, setting up and working with source control. Advantages of using BizTalk are. 1. Recoverable Interchange. In BizTalk, an Interchange can contain two or more messages, such as a batch. Recoverable Interchange was introduced in BizTalk Server 2006, only messages will be suspended if they fail validation, and when corrected theycan be resumed after the error. 2. Failed message routing. This functionality is available to enable failed messages to be subscribed by orchestration and send ports. When used appropriately, failed message routing will be used to notify users about the failed messages or to handle error properly and repairing the messages if required. 3. Lowest total cost of ownership (TCO). BizTalk Server reduces the cost and complexity of managing business processes and its automation with a single, unified solution for Enterprise Application Integration (EAI), Business-to-Business integration (B2Bi), and Business Process Management (BPM). 4. BizTalk Integrated management and development tools will enhance productivity. 5. Using TAPI it also supports Computer Telephony Interface (CTI). 6. Powerful, familiar tools to design, develop, deploy, will enable to manage those processes easily 4
  • 4. Integration Process Figure 3: A typical Integration Scenario Source(ERP,LOB,Thir d Party Vendor etc.,) Web Method1 Target(ERP,LOB,Third Party Vendor etc.,) Integration Layer 1 Web Method2 On a scheduled basis(Daily/ Monthly etc.,) 2 Web Method1 Web Method2 Web Services 2 Web Services 2. Retrieve Data from source Source XML Data Stream 1 3. Map source structure to target structure Send data to Target Target XML Data Stream In this scenario, information will betransferred from multiple source systems like Enterprise Resource Planning (ERP), Line of Business (LOB) systems or any other third party vendor systemsto Target system (ERP, LOB or a third party vendor). Messages inside BizTalk are through the XML documents and defined with the XML schemas in XSD standard. Maps are implemented with the XSLT standard. Orchestrations or mappings are implemented to meet the business needs while transferring data from source to target. Schemas, maps, pipelines and orchestrations are created visually using graphical tools within Microsoft Visual Studio. The additional functionality can be delivered by .NET assemblies that can be called from existing modules—including, for instance, orchestrations, maps, pipelines, business rules. BizTalk allows creating 'send' and 'receive' port through which external applications can exchange messages .The ports can be configured to use a set of transport protocols and application adapters that are shipped with BizTalk Server. With these adapters, BizTalk can exchange messages over multiple channels such as files, MSMQ, HTTP (Hypertext Transfer Protocol), SOAP, IBM MQ, etc. Application adapters, which can directly interact with packaged applications like SAP, Oracle Apps, etc., are also available. 5
  • 4.1 File or Web Service Data is retrieved from a third party web service or a file based where the business rules are applied in and will transform the data to the targetsystem. There are differentways to expose and consume Webservices from BizTalk Server. BizTalk can understand both positional/delimited and XML message formats. Biztalk can transform messages from one format to another. Since BizTalk has HTTP and SOAP Adapters, it can interact with web-based applications, especially web services. The BizTalk Server interacts with services in the following two ways: 1. BizTalk process orchestrations can be exposed as Web services. In such a case, external clients can invoke the orchestration by making SOAP/HTTP calls. 2. BizTalk can consume Web services by using the SOAP Adapter. 6
  • 5. Adapter and Integration Solutions Adapter for BizTalk Server seamlessly integrates BizTalk with enterprise databases, applications, transactions, business packages, EDI, and more. Adapters allowconnecting Microsoft BizTalk server to Enterprise Information Systems, including homegrown applications, databases like DB2, legacy data stores, transaction systems, e-business formats such as EDI, HIPAA and HL7, and packaged applications. The Adapter Offering for BizTalk Server: 1. Provides easy administration and configuration through existing BizTalk Server Administration Console 2. Complete, Proven, High-Performance Adapter Set 3. Relational and legacy database systems 4. Transaction processors 5. Custom applications 6. Object models like COM objects, Enterprise JavaBeans, and others 7. Messaging environments like IBM MQSeries,Oracle AQ,TIBCO EMS Q 8. Terminal-based systems like mainframes and AS/400s 9. Packaged applications such as Enterprise Resource Planning (ERP) systems, Supply Chain Management (SCM) systems, and Customer Relationship Management (CRM) systems 10. Electronic Data Interchange (EDI) systems 11. e-Business XML formats such as SWIFT, FIX, HIPAA, ebXML, BOD, and cXML 7
  • Each adapter masks the complexities of its underlying source, so developers can use industrystandard XML, SQL, or Stored Procedures to work with all types of information. Also, each adapter is highly optimized for its target, so developers don't require specialized knowledge to take full advantage of special features and syntax for unusual data structures, applications, and transactions. All adapters provide standard metadata about available functions and fields, return values, error codes, and more. 6. BIZTALK Best Practices Some of the BizTalk design best practices to be followed are. a. Design Best Practices Category Technique Design Group the projects as per artifacts type. Design Promote only those properties that are required for routing or tracking. Use distinguished property for expression shape usage. Reference It is a best to group the projects as per the kind of artifacts it contains. Schemas are broken into Internal, plus either External, Import and Export, or even separated by adapter: SQL, Oracle, etc. Orchestrations or Processes should be in one project. Avoid using Transforms in Orchestrations. Put maps in the Ports. Pipelines, if you need them, in a separate project. Maps or Transforms in a project. Performance More processing required to find the promoted properties from the message. These properties are loaded in the memory, hence having a lot of them can put the system under extreme loads. Also, they are 8
  • written to the database for tracking taking up disk space and more CPU cycles. b. Schema Creation Best Practices Category Technique Reference Schemas Always Use Separate Internal and External Schemas Always transform messages into canonical schema— regardless of how simple the schema is or who the source is. This can reduce the number of maps you need. Suppose you need to map three types of inbound messages to three types of outbound messages (perhaps not today, but maybe in the future). Applying this technique, you create a map to your canonical schema for each inbound schema and then a map from your canonical schema to each outbound schema. That’s only six maps to build and maintain, not nine. Schemas Always validate xml message against the standard schema. N/A 9
  • c. Map Creation Best Practices Category Technique Reference Transformation Use Transformation on Receive and Send Ports when you have multiple partners sending multiple message formats and you need to convert these messages to a canonical format. Business reason. Embed the map in the schedule and the partner changes the format, not only do you have to rebuild the map, you have to rebuild the schedule to use the new version of the map. Transformation Use transformation in your Performance. orchestration schedule When you need to generate a new message in the schedule and use the modified (mapped) contents of an existing message as the base. When you want to map multiple parts of a message into one outbound message (this type of mapping cannot be done in receive / send port). There are performance gains which come from doing mappings in receive ports sometimes, but they are mostly around how many persisted messages your scenario generates and it is a bit complicated to explain. Transformation Pure XSLT is much faster than XSLT which uses inline script or referenced assemblies. Performance Use xslt as much as possible in mapping. 10
  • Transformation Use external XSLT or serializable classes in case of complex maps with 1000+ connections. Maintainability. Transformation Do not use .NET objects directly inside the map. Use external assemblies instead. Maintainability and Performance. Transformation To minimize memory consumption, we recommend that you put any map that uses these functoids into a small assembly. Then, reference that assembly. Performance. The System.Policy.Security.Evidence object is often used in transforms and can consume a lot of memory. Whenever a map contains a scripting functoid that uses inline C# (or any other inline language), the assembly is created in memory. The System.Policy.Security.Evidence object uses the object of the actual calling assembly. This situation creates a rooted object that is not deleted until the BizTalk service is restarted. d. Orchestration Best Practices Category Orchestration Technique Eliminate orchestrations for messaging only patterns Reference When possible minimize the use of orchestrations to increase overall throughput and reduce the latency of 11
  • business processes. If there is no need to run long running transactions and there is no need to invoke several systems for each request, then consider eliminating orchestrations and moving business logic to Receive and Send Ports to reduce the total amount of roundtrips to the BizTalkMsgBoxDb and decrease the latency due to database access. In this case, implement custom pipelines and reuse helper classes that were previously invoked from orchestrations. Use orchestrations only when strictly needed to implement design patterns as Scatter and Gather or Convoys. Orchestration Always Use Multi-Part Message Types If your logical port and Receive/Send shape are using your multi-part message type, and you can easily change the underlying schema in the future without having to disconnect the port from the receive shape. Orchestration Optimize orchestration latency Best practice by reducing the number of persistence points. Orchestration Minimize orchestration complexity to improve performance Best practice Orchestration Use the Call Orchestration shape versus the Start Orchestration shape to improve performance Best practice Orchestration Use XmlReader with XLANGMessage versus using Performance 12
  • XmlReader with XmlDocument to improve orchestration performance Orchestration minimizing the use of logical ports bound to physical ports Performance Orchestration to extract or set properties used with business logic in an orchestration If you are using a map to extract or set properties used with business logic in an orchestration, use distinguished fields or promoted properties. This practice should be followed because when extracting or setting values with a map the document is loaded into memory however when using distinguished fields or promoted properties, the orchestration engine accesses the message context and does not load the document into memory. Orchestration aggregate several fields into one field If you are using a map to aggregate several fields into one field, use distinguished fields or promoted properties with an orchestration variable to accumulate the result set. Orchestration Use Orchestration Design Patterns when required Performance and maintainability Orchestration Make appropriate use of .NET classes in orchestrations to maximize performance Performance and maintainability Orchestration Orchestration Exception Handler Blocks When using Orchestration exception Handler blocks, include all orchestration shapes in one or multiple scopes (non transactional scopes whenever possible) and create at least the following exception handler blocks: 13
  • An exception handler block for handling a generic System.Exception error. An exception handler block for handling a general exception in order to catch and handle possible unmanaged errors, such as COM exceptions. Orchestration Always Try to Design Orchestrations with DirectBound Ports You’ve got three DirectBound port types: Message Box Routing, Self-Correlating, and Orchestration-toOrchestration (also called Partner Ports). Direct-Bound ports are self-configured, without sacrificing flexibility or performance. A handy benefit of Direct-Bound ports is that neither the developer nor the administrator needs to create corresponding physical ports in BizTalk Explorer or BizTalk Administration Console. So Direct-Bound ports yield orchestrations that are more self-contained, and therefore more reusable and easier to redeploy independently. Orchestration Publish Your External Schema, Not Your Orchestration It provides loosely coupled mechanism for exposing the services. This buys you more freedom to change your orchestration without breaking 14
  • the caller e. Best Practices for Performance Category Technique Performance Optimize the BizTalk Registry for Web Services. Reference If your BizTalk orchestration is published as a Web service, you’ll definitely want to tweak some of the default ASP.NET parameters used by BizTalk. Below are the recommended settings. Windows Registry Editor version 5.00 [HKEY_LOCAL_MACHINESYSTE MCurrentControlSetServicesBTSSvc$ BTSHOSTCLR Hosting] “MaxIOThreads”=dword:00000064 “MaxWorkerThreads”=dword:00000064 “MinIOThreads”=dword:00000019 “MinWorkerThreads”=dword:00000019 Please refer http://msdn2.microsoft.com/enus/library/aa561380.aspx for more details. Performance Configure Parameters Used Please refer below link for more for Low Latency information. Performance Tuning http://msdn.microsoft.com/enus/library/aa475435.aspx 15
  • Performance High Availability Create separate hosts for orchestration, Receive and Send Ports Creating separate hosts allows BizTalk to implement throttle mechanism and help it to load balance the incoming messages. Cluster the resources for high availability. Clustering of MSDTC, SSO, Disk and BizTalk for High availability. http://www.microsoft.com/downloads/d etails.aspx?displaylang=en&FamilyID= eb437722-2828-4cbb-84c317556b4df26b High Availability Use database mirroring for BizTalk Server databases high availability. Database mirroring of BizTalk databases helps to achieve the High Availability for Database servers and helps to recover from Failover scenarios. Appendix Reference used for Best Practices: http://social.technet.microsoft.com/wiki/contents/articles/5210.biztalk-server-2010-orchestration-bestpractices.aspx 16